Significance target detection method and device, electronic equipment and storage medium

Abstract

The application provides a saliency object detection method and device, electronic equipment and storage medium, and relates to the technical field of computer vision. The method comprises the following steps: inputting color intensity values of each pixel point in an RGB three-channel target image into a pulse neuron coding layer to obtain a first coding result of the color intensity values of each pixel point in each channel; inputting the first coding result into a pulse neuron feature processing layer to obtain first analog visual coding information of each pixel point; inputting the first analog visual coding information of each pixel point into a skip coding layer to obtain a second coding result; inputting the second coding result into a self-attention mechanism layer and a multi-layer perceptron layer to realize enhancement processing and obtain second analog visual coding information of each pixel point; and inputting the first analog visual coding information and the second analog visual coding information into an edge mapping layer to obtain an attention edge image (detection result) of the target image. The application can improve the calculation efficiency of saliency object detection.

Description

Technical Field

[0001] This application relates to the field of computer vision technology, and in particular to a salient target detection method and apparatus, electronic device and storage medium. Background Technology

[0002] Salient object detection (SOD) is a key low-level task in computer vision. SOD extracts visually salient parts from images, providing important prior information to support higher-level tasks in computer vision. Currently, SOD is used as supplementary information for these higher-level tasks on edge devices such as mobile phones and surveillance cameras.

[0003] In reality, the computing resources on edge devices such as mobile phones and surveillance cameras are limited, resulting in low computing efficiency of SOD. Summary of the Invention

[0004] This application provides a salient target detection method, apparatus, electronic device, and storage medium to address the shortcomings of existing technologies where the calculation speed for salient target detection is too slow and the calculation efficiency is too low due to limited computing resources, thereby achieving the goal of improving the calculation efficiency of salient target detection.

[0005] This application provides a salient target detection method, including the following steps: The color intensity values ​​of each pixel in the target image in the RGB three channels are input into the spiking neuron coding layer, and the first encoding result of pulse encoding of the color intensity values ​​of each pixel in each of the RGB three channels is obtained from the output of the spiking neuron coding layer. The first encoding result of the color intensity value of each pixel in the RGB three channels is input into the spiking neuron feature processing layer constructed based on multiple spiking neurons and channel splicing layers to obtain the first simulated visual encoding information of each pixel output by the spiking neuron feature processing layer. The first analog visual coding information of each pixel is input into a skip coding layer constructed using a residual structure, and the second coding result of each pixel is obtained from the output of the skip coding layer. The second encoding result of each pixel is input into a self-attention mechanism layer and a multilayer perceptron layer with residual connections constructed based on spiking neurons, and the second simulated visual encoding information of each pixel is obtained after the second encoding result of each pixel is enhanced based on the self-attention mechanism layer and the multilayer perceptron layer. The first and second simulated visual encoding information of each pixel is input into an edge mapping layer constructed based on multiple spiking neurons, multiple interpolation sampling layers, and multiple feature concatenation layers. The attention edge image of the target image output by the edge mapping layer is obtained, and the attention edge image is used as the detection result.

[0006] According to the salient target detection method provided in this application, the spiking neuron feature processing layer includes a first primary color feature processing layer, a second primary color feature processing layer, and a third primary color feature processing layer constructed based on multiple spiking neurons; The first primary color feature processing layer is used to obtain the first primary color encoding result of the color intensity value of each pixel in the first channel of the RGB three channels in the first encoding result, perform a spiking neuron operation on the first primary color encoding result and then perform a max pooling operation to output the first primary color spiking encoding information to the channel splicing layer. The second primary color feature processing layer is used to obtain the second primary color encoding result of the color intensity value of each pixel in the second channel of the RGB three channels in the first encoding result, perform a spiking neuron operation on the second primary color encoding result and then perform a max pooling operation to output the second primary color spiking encoding information to the channel splicing layer. The third primary color feature processing layer is used to obtain the third primary color encoding result of the color intensity value of each pixel in the third channel of the RGB three channels in the first encoding result, perform a spiking neuron operation on the third primary color encoding result and then perform a max pooling operation to output the third primary color spiking encoding information to the channel splicing layer. The channel splicing layer is used to splice the first primary color pulse coding information, the second primary color pulse coding information and the third primary color pulse coding information to output the first analog visual coding information of each pixel.

[0007] According to the salient target detection method provided in this application, the skip coding layer includes: a first feature extraction layer, a second feature extraction layer, and a fusion layer; The first feature extraction layer is used to receive the first simulated visual encoding information of each pixel, extract features from the first simulated visual encoding information of each pixel, and output the first feature map of each pixel to the second feature extraction layer and the fusion layer. The second feature extraction layer is used to perform spiking neuron operations on the first feature map of each pixel and then perform feature extraction, and output the second feature map of each pixel to the fusion layer; The fusion layer is used to sum the first feature map and the second feature map of each pixel to obtain a first summation result, and after performing a spiking neuron operation on the first summation result, output the second encoding result of each pixel.

[0008] According to the salient target detection method provided in this application, the self-attention mechanism layer is used for: Obtain the second feature map of each pixel; The second feature map is bisected along the channel dimension to obtain a first bisected feature map and a second bisected feature map. Based on multiple spiking neurons, residual connections are applied between the first bisecting feature map and the second bisecting feature map to obtain attention values; Feature extraction is performed on the attention values ​​to generate a third feature map; Obtain a second summation result by summing the third feature map and the second encoding result, and determine the second summation result as the self-attention mechanism processing result.

[0009] According to the salient target detection method provided in this application, the multilayer perceptron layer is used for: After performing spiking neuron operations on the self-attention mechanism processing results, the first operation result is output. The first operation result is then normalized and then convolutional to obtain the fourth feature map. After performing a spiking neuron operation on the fourth feature map, a second operation result is output. After performing a convolution operation on the second operation result, a normalization operation is performed to obtain a fifth feature map. The fourth feature map and the fifth feature map are summed to obtain a third summation result. After performing spiking neuron operations on the third summation result, feature extraction is performed to obtain a sixth feature map. The sixth feature map and the self-attention mechanism processing result are summed to obtain a fourth summation result. The fourth summation result is determined as the second simulated visual coding information of each pixel.

[0010] According to the salient target detection method provided in this application, the edge mapping layer includes a first pulse convolutional layer, a second pulse convolutional layer, a third pulse convolutional layer, and a fourth pulse convolutional layer constructed based on multiple spiking neurons; the multiple feature splicing layers include a first feature splicing layer and a second feature splicing layer; the multiple interpolation sampling layers include a first interpolation sampling layer, a second interpolation sampling layer, and a third interpolation sampling layer. The first pulse convolutional layer is used to perform feature extraction and spiking neuron operation on the first simulated visual coding information of each pixel, and output the first processing result to the first feature splicing layer. The first feature splicing layer is used to splice the first processing result and the original feature information of each pixel, and output the second processing result to the second pulse convolution layer among the plurality of pulse convolution layers; The second pulse convolutional layer is used to extract features and perform spiking neuron operations on the second processing result, and output the third processing result to the first interpolation sampling layer; The first interpolation sampling layer is used to perform interpolation sampling on the third processing result based on nearest neighbor interpolation, and output the first interpolation sampling result to the second feature splicing layer; The third pulse convolutional layer is used to perform feature extraction and spiking neuron operations on the second simulated visual coding information of each pixel, and outputs the fourth processing result to the second feature splicing layer. The second feature splicing layer is used to splice the fourth processing result and the first interpolation sampling result, and output the fifth processing result to the fourth pulse convolutional layer; The fourth pulse convolutional layer is used to extract features and perform spiking neuron operations on the fifth processing result, and output the sixth processing result to the third interpolation sampling layer; The third interpolation sampling layer is used to interpolate and sample the sixth processing result and then output the attention edge image.

[0011] This application also provides a salient target detection device, comprising the following modules: The encoding processing module is used to input the color intensity values ​​of each pixel in the RGB three-channel of the target image into the spiking neuron encoding layer, and obtain the first encoding result output by the spiking neuron encoding layer, which performs pulse encoding on the color intensity values ​​of each pixel in each of the RGB three-channel; The feature processing module is used to input the first encoding result of the color intensity value of each pixel in the RGB three channels into the spiking neuron feature processing layer constructed based on multiple spiking neurons and channel splicing layers, and obtain the first simulated visual encoding information of each pixel output by the spiking neuron feature processing layer; The skip coding module inputs the first analog visual coding information of each pixel into the skip coding layer constructed using a residual structure, and obtains the second coding result of each pixel output by the skip coding layer. An enhancement processing module is used to input the second encoding result of each pixel into a self-attention mechanism layer and a multilayer perceptron layer with residual connections constructed based on spiking neurons, and to obtain the second simulated visual encoding information of each pixel after the second encoding result of each pixel is enhanced based on the self-attention mechanism layer and the multilayer perceptron layer. The mapping processing module is used to input the first and second simulated visual coding information of each pixel into an edge mapping layer constructed based on multiple spiking neurons, multiple interpolation sampling layers, and multiple feature splicing layers, to obtain the attention edge image of the target image output by the edge mapping layer, and to use the attention edge image as the detection result.

[0012] This application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the salient target detection method as described above.

[0013] This application also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the salient target detection method as described above.

[0014] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the salient target detection method as described above.

[0015] The salient target detection method, apparatus, electronic device, and storage medium provided in this application input the color intensity values ​​of each pixel in the RGB three channels of the target image into a spiking neuron coding layer to obtain the first encoding result output by the spiking neuron coding layer, which performs pulse coding on the color intensity values ​​of each pixel in each of the RGB three channels; input the first encoding result of each pixel into a spiking neuron processing layer constructed based on multiple spiking neurons and a channel splicing layer to obtain the first simulated visual encoding information of each pixel output by the spiking neuron feature processing layer; and input the first simulated visual encoding information of each pixel into a skip coding layer constructed using a residual structure. The process involves obtaining the second encoding result of each pixel output by the skip coding layer; inputting the second encoding result of each pixel into a self-attention mechanism layer and a multilayer perceptron layer constructed based on spiking neurons with residual connections, and obtaining the second simulated visual encoding information of each pixel after the second encoding result information of each pixel is enhanced by the self-attention mechanism layer and the multilayer perceptron layer; inputting the second simulated visual encoding information of each pixel into an edge mapping layer constructed based on multiple spiking neurons, multiple interpolation sampling layers, and multiple feature concatenation layers, and obtaining the attention edge image of the target image output by the edge mapping layer, and using the attention edge image as the detection result. Therefore, this application can adopt a biomimetic design to apply the SNN architecture to general RGB salient object detection, effectively leveraging its energy efficiency advantages to solve the shortcomings of existing technologies where the computational speed of salient object detection is too slow and the computational efficiency is too low due to limited device computing resources, thereby achieving the goal of improving the computational efficiency of salient object detection. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is a flowchart illustrating the salient target detection method provided in this application.

[0018] Figure 2 This is a schematic diagram of the data processing procedure for the salient target detection method provided in this application.

[0019] Figure 3 This is a schematic diagram of the pyramid structure of the primary color separation module in the salient target detection method provided in this application.

[0020] Figure 4 This is one of the schematic diagrams illustrating the edge mapping layer processing in the salient target detection method provided in this application.

[0021] Figure 5 This is the second schematic diagram of the edge mapping layer processing in the salient target detection method provided in this application.

[0022] Figure 6 This is a comparison chart of salient target detection results obtained through various methods in the evaluation results of the salient target detection method provided in this application.

[0023] Figure 7 This is the evaluation result data of the salient target detection method provided in this application, which comprehensively evaluates the model performance from different perspectives based on four indicators.

[0024] Figure 8 This is the energy consumption assessment data for the salient target detection method provided in this application.

[0025] Figure 9 This is a schematic diagram of the salient target detection device provided in this application.

[0026] Figure 10 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0027] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0028] Among related technologies, Spiking Neural Networks (SNNs) are considered the third generation of neural network models and represent a highly promising solution in the field of neuromorphic computing. SNNs can mimic the firing patterns of biological neurons, using pulses as the carriers of information flow within the network to achieve sparse computation. SNNs have achieved success in various high-level tasks in computer vision technologies such as object recognition, detection, and segmentation, revealing their potential in visual tasks. However, due to the numerous challenges faced by pixel-level regression, the practical application of SNNs in low-level tasks remains insufficient.

[0029] When SNNs are deployed on dedicated neuromorphic hardware, they can reduce energy consumption compared to Artificial Neural Networks (ANNs). In this case, SNNs can be applied to edge computing inference. At the same time, SNNs have achieved performance levels comparable to ANNs. This reveals the potential of SNNs in visual tasks.

[0030] SOD originates from the human visual attention path, and its output binary image is highly consistent with the impulse-based format. Therefore, salient object detection can be achieved through a biomimetic SNN.

[0031] The following is combined with Figures 1-8 This application describes the salient target detection method.

[0032] Figure 1 This is one of the flowcharts illustrating the salient target detection method provided in this application, such as... Figure 1 As shown, the method includes the following steps S110~S150.

[0033] S110: Input the color intensity values ​​of each pixel in the target image in the RGB three channels into the spiking neuron coding layer, and obtain the first encoding result of the spiking neuron coding layer outputting the color intensity values ​​of each pixel in each of the RGB three channels through pulse encoding.

[0034] To more vividly illustrate the ideas in the embodiments of this application, see [link to relevant documentation]. Figure 2As shown in the embodiments of this application, the photosensitive process of S-type cone cells 21-S that perceive blue light, M-type cone cells 21-M that perceive green light, and L-type cone cells 21-L that perceive red light can be simulated first. The three primary colors of each pixel in the target image 201 are separated to obtain the red component (i.e., the color intensity value of the R channel), the green component (i.e., the color intensity value of the G channel), and the blue component (i.e., the color intensity value of the B channel) of each pixel in the target image 201.

[0035] like Figure 2 As shown, cone cells transmit the sensed visual signals to bipolar cells 22 via synapses. Bipolar cells 22 then transmit the integrated and amplified visual signals to horizontal cells 23 and amacrine cells 24 via synapses. Horizontal cells 23 regulate the activity of bipolar cells 22 through lateral inhibition and feedback regulation mechanisms, thereby indirectly affecting the output signals of ganglion cells 25. Amacrine cells 24 exert lateral inhibition on ganglion cells 25 and regulate the activity of bipolar cells through feedback mechanisms. This interaction of longitudinal and lateral transmission is similar to the residual structure commonly used in neural networks.

[0036] Reference Figure 2 After obtaining the input image (i.e., the target image), pulse coding is first performed on the color intensity value of each pixel in the RGB three channels of the input image to obtain the first coding result. After obtaining the first coding result, the channel-by-channel separation step is then performed. The specific implementation of pulse coding and obtaining the pulse coding result (i.e., the first coding result) can refer to existing technologies, and this application does not limit it in this regard.

[0037] S120: Input the first encoding result of the color intensity value of each pixel in the RGB three channels into the spiking neuron feature processing layer constructed based on multiple spiking neurons and channel splicing layers, and obtain the first simulated visual encoding information of each pixel output by the spiking neuron feature processing layer.

[0038] In the early information acquisition stage, if a large amount of information is lost, it will lead to a decline in the performance of subsequent layers. In order to retain valuable information, prevent uncontrollability caused by relying on a single input, and improve the convergence ability during model training, the first primary color feature processing layer, the second primary color feature processing layer, and the third primary color feature processing layer are constructed based on multiple spiking neurons and multiple max pooling layers in the spiking neuron feature processing layer.

[0039] The first primary color feature processing layer is used to obtain the first primary color encoding result of the color intensity value of each pixel in the first channel of the RGB three channels in the first encoding result. After performing spiking neuron operation on the first primary color encoding result, max pooling operation is performed, and the first primary color spiking encoding information is output to the channel splicing layer.

[0040] The second primary color feature processing layer is used to obtain the second primary color encoding result of the color intensity value of each pixel in the second channel of the RGB three channels in the first encoding result. After performing spiking neuron operation on the second primary color encoding result, max pooling operation is performed, and the second primary color spiking encoding information is output to the channel splicing layer.

[0041] The third primary color feature processing layer is used to obtain the third primary color encoding result of the color intensity value of each pixel in the third channel of the RGB three channels in the first encoding result. After performing spiking neuron operation on the third primary color encoding result, max pooling operation is performed, and the third primary color spiking encoding information is output to the channel splicing layer.

[0042] Taking the first channel as the red channel, the second channel as the green channel and the third channel as the blue channel as an example, the expression of the first primary color feature processing layer can be found in formula (1), the expression of the second primary color feature processing layer can be found in formula (2), and the expression of the third primary color feature processing layer can be found in formula (3).

[0043] Formula (1): .

[0044] Formula (2): .

[0045] Formula (3): .

[0046] In formula (1) ~ formula (3), This is the primary color pulse encoding information. This is the second primary color pulse encoding information. For the third primary color pulse encoding information, This is for maximizing pool operations.

[0047] In formulas (1) to (3), T represents the time dimension, which is used to represent the number of time steps in the time series data; H represents the height dimension, which is used to represent the height of the image or feature map; and W represents the width dimension, which is used to represent the width of the image or feature map.

[0048] Finally, the channel stitching layer stitches together the first primary color pulse coding information, the second primary color pulse coding information, and the third primary color pulse coding information to output the first analog visual coding information for each pixel. The expression for the channel stitching layer is given in formula (4).

[0049] Formula (4): .in, For the output of the channel splicing layer, This indicates a splicing operation. That is, the first analog visual encoding information.

[0050] In this application, each pixel is processed according to the above formulas (1)-(4) to obtain the corresponding first simulated visual coding information. In this application, when the spiking neuron feature processing layer performs data processing, it calls the function of the skip coding module. SEM in formulas (1)-(3) represents skip coding processing. The first simulated visual coding information finally output by the spiking neuron feature processing layer will be input into the skip coding layer again later.

[0051] The spiking neuron feature processing layer in this application is... Figure 2 The Primary-color Separation Module (PSM) in the model. Based on the above introduction, the relationships between the layers in the PSM can be obtained as follows: Figure 3 As shown, it resembles a pyramid structure. Figure 3 In the diagram, the first feature extraction layer, the second feature extraction layer, and the fusion layer indicate that the primary color separation module will call the skip coding layers corresponding to three different channels in the skip coding module during data processing.

[0052] Of course, it can be understood that the above-described construction of the first primary color feature processing layer, the second primary color feature processing layer, and the third primary color feature processing layer based on multiple spiking neurons and the multiple max-pooling layers is merely an example, and the expression for the first primary color feature processing layer could also be, for example, In this case, the expression for the second primary color feature processing layer could also be, for example, In this case, the expression for the third primary color feature processing layer could also be, for example, .

[0053] Step 130: Input the first analog visual coding information of each pixel into the skip coding layer constructed using the residual structure, and obtain the second coding result of each pixel output by the skip coding layer.

[0054] In some embodiments, the skip coding layer may include a first feature extraction layer, a second feature extraction layer, and a fusion layer.

[0055] The first feature extraction layer can be used to receive the first analog visual encoding information of each pixel, extract the feature of the color intensity value of each pixel in the RGB three channels, and output the first feature map of each pixel to the second feature extraction layer and the fusion layer.

[0056] Taking all pixels including the target pixel as an example (the target pixel is any one of the pixels), in some examples, in the first feature extraction layer, the first simulated visual encoding information of the target pixel can be convolved to extract feature values, and then batch normalization is performed to obtain the first feature map. The expression of the first feature extraction layer is shown in formula (5).

[0057] Formula (5): .

[0058] In formula (5), Represents the first feature map. This represents the convolution operation. This indicates a batch normalization operation. This represents the feature input.

[0059] The second feature extraction layer is used to perform spiking neuron operations on the first feature map of each pixel and then extract features, outputting the second feature map of each pixel to the fusion layer.

[0060] In some examples, in the second feature extraction layer, the first feature map of the target pixel can be processed by spiking neuron operations, followed by convolution operations to extract feature values, and then batch normalization operations can be performed to obtain the second feature map of the target pixel. The expression for the second feature extraction layer is given in Equation (6).

[0061] Formula (6): .

[0062] In formula (6), This represents the second feature map. This indicates the operation of a spiking neuron.

[0063] The fusion layer is used to sum the first feature map and the second feature map of each pixel to obtain the first summation result. After performing spiking neuron operation on the first summation result, the second encoding result of each pixel is output. The expression of the fusion layer can be found in formula (7).

[0064] Formula (7): .

[0065] In formula (7), That is, the result output by the skip coding layer.

[0066] For RGB three channels There are three types, namely , as well as The fusion layer can output the encoded result corresponding to the R channel. The encoding results corresponding to the G channel The encoding result corresponding to channel B Specifically, in Figure 2 In the hop coding module, the three hop coding layers use the same network structure but different parameters. The first hop coding layer is used to process the data in the R channel to obtain the encoded result. The second skip coding layer is used to process the data in the G channel to obtain... The third skip coding layer is used to process the data in the B channel to obtain... In each skip coding layer, the first convolution and batch normalization correspond to the operations in the first feature extraction layer, the second convolution and batch normalization correspond to the operations in the second feature extraction layer, and max pooling corresponds to the operations in the fusion layer.

[0067] S140: Input the second encoding result of each pixel into the self-attention mechanism layer and multilayer perceptron layer with residual connections constructed based on spiking neurons, and obtain the second simulated visual encoding information of each pixel after the second encoding result of each pixel is enhanced based on the self-attention mechanism layer and multilayer perceptron layer.

[0068] Please continue reading Figure 2 During the formation of vision, after the retina captures and processes light signals, the visual information is transmitted through ganglion cells 25 to the lateral geniculate body 26. The lateral geniculate body 26 is an intermediate region for information integration, where it further enhances the visual information before transmitting it to the cerebral cortex.

[0069] To simulate the above process, this application introduces a Spiking Transformer to further enhance and integrate information. The Transformer structure is inspired by visual attention mechanisms and has various implementations in SNNs. Since the overall computation is based on convolution, the Spiking Transformer is more suitable for hardware deployment. Based on the Spiking Transformer, this application constructs a self-attention mechanism layer and a multi-layer perceptron layer using spiking neurons.

[0070] In some embodiments, the self-attention mechanism layer is used to: obtain a second feature map of each pixel; bisect the second feature map along the channel dimension to obtain a first bisected feature map and a second bisected feature map; apply residual connections between the first bisected feature map and the second bisected feature map based on multiple spiking neurons to obtain an attention value; perform feature extraction on the attention value to generate a third feature map; obtain a second summation result by summing the third feature map and the second encoding result, and determine the second summation result as the self-attention mechanism processing result.

[0071] In specific implementation, the computational structure of the self-attention mechanism layer (represented by ResSSA in formula (11)) is shown in formulas (8) to (11) below.

[0072] Formula (8): .

[0073] Formula (9): .

[0074] Formula (10): .

[0075] Formula (11): .

[0076] In formula (8) ~ formula (11), The input represents the feature, specifically the output of the skip coding module, i.e., the second coding result. s is the second feature map of each pixel obtained based on the second coding result. This is the attention value. Indicates multiplication operation, This indicates an operation that divides the channel along the channel dimension.

[0077] The multilayer perceptron layer is used for: performing spiking neuron operations on the self-attention mechanism processing result and outputting the first operation result; performing a normalization operation on the first operation result and then performing a convolution operation to obtain the fourth feature map; performing spiking neuron operations on the fourth feature map and outputting the second operation result; performing a convolution operation on the second operation result and then performing a normalization operation to obtain the fifth feature map; summing the fourth and fifth feature maps to obtain the third summation result; performing spiking neuron operations on the third summation result and then performing feature extraction to obtain the sixth feature map; summing the sixth feature map and the self-attention mechanism processing result to obtain the fourth summation result; and determining the fourth summation result as the second simulated visual encoding information for each pixel.

[0078] In specific implementation, the computational structure of the multilayer perceptron layer (represented by ResMLP in formula (14)) is shown in formulas (12) to (15) below.

[0079] Formula (12): .

[0080] Formula (13): .

[0081] Formula (14): ).

[0082] Formula (15): .

[0083] In formula (12) ~ formula (15), This represents the feature input, specifically the result of the self-attention mechanism. This represents the output of the multilayer perceptron layer.

[0084] In practice, the self-attention mechanism layer and the multilayer perceptron layer can be integrated into one module, which is named the Enhancement Integration Module (EIM).

[0085] Of course, it is understandable that other forms of attention variants (Spike-Self-Attention, SSA) could be used to replace the above self-attention mechanism layer, or other forms of multilayer perceptrons (MLP) could be used to replace the above multilayer perceptron layer.

[0086] S150: Input the first and second analog visual coding information of each pixel into the edge mapping layer constructed based on multiple spiking neurons, multiple interpolation sampling layers, and multiple feature splicing layers, obtain the attention edge image of the target image output by the edge mapping layer, and use the attention edge image as the detection result.

[0087] In the visual pathway, light signals acquired from the external environment are progressively transmitted to the primary visual cortex 27, where a primary perceptual image is formed. This primary image typically contains noticeable object edges, which aligns with the results of salient object detection. In this embodiment, based on a biomimetic simulation of the formation process of the primary perceptual image, an edge mapping layer (also known as a Map Generation Module (MGM)) is developed to generate a salient map from features, thereby implementing the execution process of S150.

[0088] The entire process of the edge mapping layer can be represented by the following formulas (16) to (18).

[0089] Formula (16): .

[0090] Formula (17): .

[0091] Formula (18): .

[0092] In formula (16) ~ formula (18), This is the output of the pulsed convolutional layer. For the output of the splicing layer, formula (18) represents the upsampling process. Wherein, It includes first analog visual coding information and second analog visual coding information.

[0093] In some embodiments, combining the above formulas (16) to (18), and Figure 4 In the edge mapping layer, multiple spiking convolutional layers are constructed based on multiple spiking neurons (i.e., Figure 4The system includes a Conv module; multiple feature splicing layers (including a first feature splicing layer and a second feature splicing layer); multiple pulse convolutional layers (including a first pulse convolutional layer, a second pulse convolutional layer, a third pulse convolutional layer, and a fourth pulse convolutional layer); and multiple interpolation sampling layers (including a first interpolation sampling layer, a second interpolation sampling layer, and a third interpolation sampling layer).

[0094] The first spiking convolutional layer extracts features and performs spiking neuron operations on the first simulated visual encoding information of each pixel, then outputs the first processing result to the first feature concatenation layer. Combined with... Figure 4 And according to formula (16), the expression for the first pulse convolutional layer is: ; This represents the first analog visual encoding information.

[0095] In this application, for each target pixel, the corresponding first processing result can be obtained according to its first analog visual coding information by formula (16), that is, each pixel has a corresponding first processing result.

[0096] The first feature concatenation layer concatenates the first processing result and the original feature information of each pixel, and outputs the second processing result to the second pulse convolutional layer among multiple pulse convolutional layers. Combining formula (17), the expression for the first feature concatenation layer is: ; This refers to the original feature information of the target pixel.

[0097] In this application, for each target pixel, the corresponding first processing result and the original feature information are concatenated through the first feature concatenation layer to obtain the corresponding second processing result.

[0098] After performing feature extraction and spiking neuron operations on the second processing result, the second spiking convolutional layer outputs the third processing result to the first interpolation sampling layer.

[0099] The first interpolation sampling layer performs interpolation sampling on the third processing result based on nearest neighbor interpolation, and outputs the first interpolation sampling result to the second feature splicing layer.

[0100] Combination Figure 4 Combined with formula (18), the processing procedures of the second pulse convolutional layer and the first interpolation sampling layer are merged, and the expression is: ;in, This represents nearest neighbor interpolation. The reason for using nearest neighbor interpolation is to avoid introducing non-binary values, as non-binary values ​​pose a risk of reintroducing multiply-accumulate (MAC) operations. Avoiding non-binary values ​​effectively eliminates this risk. Each target pixel has a corresponding first interpolation sample result.

[0101] The third pulse convolutional layer extracts features and performs spiking neuron operations on the second simulated visual encoding information of each pixel, and then outputs the fourth processing result to the second feature splicing layer.

[0102] Combination Figure 4 And according to formula (16), the expression for the third pulse convolutional layer is: ;in, The second analog visual encoding information represents the pixel.

[0103] In this application, for each target pixel, a corresponding fourth processing result can be obtained through a third pulse convolutional layer based on its second analog visual coding information, that is, each pixel has a corresponding fourth processing result.

[0104] The second feature splicing layer splices the fourth processing result and the first interpolation sampling result, outputting the fifth processing result to the fourth pulse convolutional layer. Combined with... Figure 4 And according to formula (17), the expression for the second feature splicing layer is: .

[0105] In this application, for each target pixel, its corresponding fourth processing result and first difference sampling result are concatenated to obtain the corresponding fifth processing result.

[0106] After the fourth spiking convolutional layer performs feature extraction and spiking neuron operations on the fifth processing result, it outputs the sixth processing result to the third interpolation sampling layer.

[0107] The third interpolation sampling layer interpolates and samples the results of the sixth processing step, then outputs the attention edge image.

[0108] The processing procedures of the fourth pulse convolutional layer and the third interpolation sampling layer are combined, and the expression is: .

[0109] In some embodiments, such as Figure 5 As shown, a further convolutional layer can be placed between the fourth pulse convolutional layer and the third interpolation sampling layer to further enhance the extracted features and output... 'As an attention edge image.'

[0110] In some embodiments, before executing S110, an initial salient object detection model based on SNN, including the spiking neuron coding layer, spiking neuron feature processing layer, skip coding layer, self-attention mechanism layer, multilayer perceptron layer and edge mapping layer described above, can be pre-constructed. The initial salient object detection model is trained to obtain a salient object detection model. The target image is input into the salient object detection model, and the above-described S110 to S150 can be executed based on the salient object detection model to finally obtain the detection result of the salient object detection of the target image.

[0111] When training the initial salient object detection model, the training parameters can be set as follows: Use the AdamW optimizer to train the model for 150 epochs (training rounds). Set the initial learning rate to 4E-3 and the weight decay coefficient to 2E-5. Set the batch size to 8 and train the model in both single-step and multi-step settings.

[0112] Single-step and multi-step training modes: In multi-step training mode, the same image is input in each step, and the model iterates for 5 steps. In actual experiments, this training method yields better performance. In single-step training mode, the image to be detected is input into the model only once.

[0113] Training Dataset: The DUTS-TR dataset can be used as the training set. This dataset is currently the largest and most widely used training dataset in the field of salient object detection, containing 10,553 images. These images cover a rich variety of scenes and objects, providing sufficient sample diversity for model training. Before training, all images are resized to 224×224 resolution.

[0114] Hardware environment for model training: All experiments were conducted in an environment equipped with a single RTX 4090 GPU, and the entire training process was expected to take about 40 hours.

[0115] The evaluation metrics used in model training are determined in the following manner in this embodiment.

[0116] Evaluation datasets: To comprehensively evaluate the performance of the framework in this application, several widely used datasets were selected, including DUTS-TE, ECSSD, HKU-IS, PASCAL-S, and HRSOD.

[0117] Evaluation metrics: Following the evaluation standards of traditional SOD tasks, mean absolute error (MAE), E-measure, weighted F-measure, and structural metric Sm are used as evaluation metrics. These metrics can accurately measure the quality difference between the predicted saliency map and the true value, thereby effectively evaluating the performance of the embodiments of this application on different datasets.

[0118] In this embodiment, the SNN architecture is directly applied to the general RGB image SOD task. This embodiment draws inspiration from the primary visual pathways of primates for biomimetic design, significantly improving the model's biological rationality and performance. Furthermore, this embodiment employs a full-spike-stream design pattern: introducing a full-spike-stream design pattern suitable for SNN architectures and applying it to SpikeSOD. This pattern avoids the multiply-accumulate (MAC) operations within the SNN, effectively evaluating whether the SNN algorithm is compatible with dedicated SNN chips, thereby fully leveraging the energy efficiency advantages of SNNs.

[0119] In a specific implementation, the salient target detection method provided in the embodiments of this application was evaluated, and the evaluation data is as follows.

[0120] (1) Multidimensional comparative experimental design.

[0121] This application comprehensively evaluates SpikeSOD (the target saliency detection model trained in this application) against ANN and SNN methods in terms of performance, computational efficiency, and power consumption. Given the lack of a prior SNN method for RGB image SOD tasks, this application compares SpikeSOD with state-of-the-art SNN architectures (SEWResNet, MA-SNN, MS-ResNet, SPT-v2) and image segmentation methods, adapting these methods to the RGB image SOD task. Additionally, RST, an SNN method specifically designed for event-based saliency detection, is included in the comparison. Furthermore, an ANN method is selected for performance evaluation, resulting in the following results: Figure 6 The comparison results are shown.

[0122] See Figure 6 As shown, the input image is a natural image, and GT represents the ground truth. Each column under each method displays the predicted saliency map results. The first row shows ordinary sample images, while the remaining four rows, from top to bottom, depict challenging samples, including overlapping, multi-region, fine detail, and occlusion cases. Figure 6The comparison results shown clearly demonstrate that SpikeSOD exhibits excellent performance even when faced with these challenging samples, with detection results close to the true values ​​and performance approaching that of ANN.

[0123] (2) Performance evaluation indicators and results: The embodiments of this application use four indicators to comprehensively evaluate the model performance from different perspectives. The specific results are as follows: Figure 7 As shown in the figure. Experimental results show that SpikeSOD outperforms the current state-of-the-art SNN-based methods in terms of performance, with an advantage ranging from 15% to 50%. Methods based on semantic segmentation frameworks are difficult to directly adapt to the SOD task. Methods built on mainstream advanced SNN architectures perform poorly, proving that independent backbone networks cannot directly meet the requirements of the SOD task. In contrast, compared to ANN methods (… Figure 7 Compared to existing backbone technologies (such as ANN methods), SpikeSOD achieved comparable results, reaching approximately 90%-95% of the performance of ANN methods across various metrics. This fully demonstrates SpikeSOD's superior performance in the RGB SOD task, highlighting the innovation and superiority of the embodiments in this application in the field of computer vision. Whether compared with similar SNN methods or with ANN methods, SpikeSOD exhibits unique advantages. Figure 7 In the diagram, ↓ indicates that a lower value is better, and M represents the Mean Absolute Error (MAE). This refers to the Enhanced-alignment Measure (E-measure), S m This represents the structural similarity measure (S-measure). This indicates the weighted F-measure. Figure 7In this comparison, Method 1 is from "Basnet: Boundary-Aware Salient Target Detection" in the 2019 IEEE / CVF Conference on Computer Vision and Pattern Recognition; Method 2 is from "Beyond Classification: Directly Training Spike Neural Networks for Semantic Segmentation"; Method 3 is from "Deep Residual Learning in Spike Neural Networks" in "Advances in Neural Information Processing Systems"; Method 4 is from "Attention Spike Neural Networks" in "IEEE Transactions on Pattern Analysis and Machine Intelligence"; Method 5 is from "Driving Spike Neural Networks Towards Deep Residual Learning" in "IEEE Transactions on Neural Networks and Learning Systems" 2024; Method 6 is from "Spike-Driven Transformer V2: Meta-Spike Neural Network Architectures Inspiring Next-Generation Neuromorphic Chip Design"; and Method 7 is from "Finding Visual Salientity in Continuous Pulse Streams" in the 2024 AAAI Conference on Artificial Intelligence. Volume 38, Issue 7.

[0124] exist Figure 7 In this context, DUTS-TE represents the DUTS test set (DUTS-TE is the test portion of the DUTS dataset, full name: DUT-OMRON-Saliency). ECSSD represents the ECSSD dataset (full name: Extended Complex Scene SaliencyDataset). HKU-IS represents the HKU-IS dataset (full name: Hong Kong University - IS). PASCAL-S represents the PASCAL-S dataset (a subset of PASCAL VOC specifically designed for saliency detection tasks). HRSOD represents the HRSOD dataset (full name: High-Resolution Salient Object Detection Dataset).

[0125] In terms of energy consumption indicators, SpikeSOD also demonstrates a significant advantage, with specific data as follows: Figure 8 As shown. Based on Figure 8 As can be seen, the energy consumption performance of the embodiments of this application is as follows.

[0126] (1) Performance under the same time step: Under the same time step (T=5 / 6), SpikeSOD not only achieves better performance, but also effectively controls power consumption. This shows that in practical applications, SpikeSOD can reduce energy consumption and improve system operating efficiency while ensuring the quality of task completion.

[0127] (2) Performance and complexity at minimum time step: Further analysis of the minimum time step (T = 1) shows that SpikeSOD not only maintains its performance but also achieves a reduction of approximately 10 times in computational complexity. This characteristic is due to the good fit between the binary output and pulse output of the salient target detection task, which allows SpikeSOD to reduce the amount of computation while still ensuring the accuracy of the detection results.

[0128] (3) Impact of model miniaturization on computational cost: To meet the requirements of edge hardware platforms for lower computational demands, we evaluated the effect of miniaturized models by reducing the number of channels in SEM. Experimental results show that at larger time steps, the performance loss of miniaturized models is negligible, while the computational cost is reduced by 3 times. This means that on resource-constrained edge devices, the miniaturized SpikeSOD model can effectively reduce the computational burden and improve the applicability of the model without affecting performance.

[0129] (4) Power consumption advantage compared to ANN: Compared to ANN, SpikeSOD reduces power consumption by 100-1000 times while maintaining comparable performance. This significant advantage makes SpikeSOD highly attractive for edge hardware applications, effectively addressing the power consumption limitations of edge devices and providing strong technical support for the development of the edge computing field.

[0130] exist Figure 8 In this context, GSCOPs (Giga Synaptic Operations) represents giga-level synaptic operations (or billion-times synaptic operations). mJ (millijoule) represents millijoules.

[0131] The salient target detection method provided in this application involves inputting the color intensity values ​​of each pixel in the RGB three channels of the target image into a spiking neuron coding layer to obtain the first encoding result output by the spiking neuron coding layer, which performs pulse coding on the color intensity values ​​of each pixel in each of the RGB three channels. The first encoding result of the color intensity values ​​of each pixel in the RGB three channels is then input into a spiking neuron processing layer constructed based on multiple spiking neurons and a channel concatenation layer to obtain the first simulated visual encoding information of each pixel output by the spiking neuron feature processing layer. Finally, the first simulated visual encoding information of each pixel is input into a skip coding layer constructed using a residual structure. The process involves obtaining the second encoding result of each pixel output by the skip coding layer; inputting the second encoding result of each pixel into a self-attention mechanism layer and a multilayer perceptron layer constructed based on spiking neurons with residual connections, and obtaining the second simulated visual encoding information of each pixel after the second encoding result information of each pixel is enhanced by the self-attention mechanism layer and the multilayer perceptron layer; inputting the second simulated visual encoding information of each pixel into an edge mapping layer constructed based on multiple spiking neurons, multiple interpolation sampling layers, and multiple feature concatenation layers, and obtaining the attention edge image of the target image output by the edge mapping layer, and using the attention edge image as the detection result. Therefore, this application can adopt a biomimetic design to apply the SNN architecture to general RGB salient object detection, effectively leveraging its energy efficiency advantages to solve the shortcomings of existing technologies where the computational speed of salient object detection is too slow and the computational efficiency is too low due to limited device computing resources, thereby achieving the goal of improving the computational efficiency of salient object detection.

[0132] The salient target detection device provided in this application is described below. The salient target detection device described below can be referred to in correspondence with the salient target detection method described above.

[0133] Figure 9 This is a schematic diagram of the salient target detection device provided in the embodiments of this application. Figure 9 As shown, the salient target detection device 900 includes: an encoding processing module 901, a feature processing module 902, an enhancement processing module 903, and a mapping processing module 904.

[0134] The encoding processing module 901 is used to input the color intensity values ​​of each pixel in the RGB three channels of the target image into the spiking neuron encoding layer, and obtain the first encoding result output by the spiking neuron encoding layer, which performs pulse encoding on the color intensity values ​​of each pixel in each of the RGB three channels.

[0135] The feature processing module 902 is used to input the first encoding result of the color intensity value of each pixel in the RGB three channels into the spiking neuron feature processing layer constructed based on multiple spiking neurons and channel splicing layers, and obtain the first simulated visual encoding information of each pixel output by the spiking neuron feature processing layer.

[0136] The skip coding module 903 is used to input the first analog visual coding information of each pixel into the skip coding layer constructed with a residual structure, and obtain the second coding result of each pixel output by the skip coding layer.

[0137] The enhancement processing module 904 is used to input the second encoding result of each pixel into a self-attention mechanism layer and a multilayer perceptron layer constructed based on spiking neurons with residual connections, and to obtain the second analog visual encoding information of each pixel after enhancing the first analog visual encoding information of each pixel based on the self-attention mechanism layer and the multilayer perceptron layer.

[0138] The mapping processing module 905 is used to input the first and second analog visual coding information of each pixel into the edge mapping layer constructed based on multiple spiking neurons, multiple interpolation sampling layers, and multiple feature splicing layers, to obtain the attention edge image of the target image output by the edge mapping layer, and to use the attention edge image as the detection result.

[0139] The salient target detection device provided in this application inputs the color intensity values ​​of each pixel in the RGB three channels of the target image into a spiking neuron coding layer to obtain the first encoding result output by the spiking neuron coding layer, which performs pulse coding on the color intensity values ​​of each pixel in each of the RGB three channels; inputs the first encoding result of the color intensity values ​​of each pixel in the RGB three channels into a spiking neuron processing layer constructed based on multiple spiking neurons and a channel splicing layer to obtain the first simulated visual coding information of each pixel output by the spiking neuron feature processing layer; and inputs the first simulated visual coding information of each pixel into a skip coding layer constructed using a residual structure. The process involves obtaining the second encoding result of each pixel output by the skip coding layer; inputting the second encoding result of each pixel into a self-attention mechanism layer and a multilayer perceptron layer constructed based on spiking neurons with residual connections, and obtaining the second simulated visual encoding information of each pixel after the second encoding result information of each pixel is enhanced by the self-attention mechanism layer and the multilayer perceptron layer; inputting the second simulated visual encoding information of each pixel into an edge mapping layer constructed based on multiple spiking neurons, multiple interpolation sampling layers, and multiple feature concatenation layers, and obtaining the attention edge image of the target image output by the edge mapping layer, and using the attention edge image as the detection result. Therefore, this application can adopt a biomimetic design to apply the SNN architecture to general RGB salient object detection, effectively leveraging its energy efficiency advantages to solve the shortcomings of existing technologies where the computational speed of salient object detection is too slow and the computational efficiency is too low due to limited device computing resources, thereby achieving the goal of improving the computational efficiency of salient object detection.

[0140] Figure 10 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 10 As shown, the electronic device may include a processor 1010, a communications interface 1020, a memory 1030, and a communication bus 1040, wherein the processor 1010, the communications interface 1020, and the memory 1030 communicate with each other via the communication bus 1040. The processor 1010 can call logical instructions in the memory 1030 to execute the salient target detection method provided in this application. Furthermore, the logical instructions in the aforementioned memory 1030 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0141] On the other hand, this application also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer is able to perform the salient target detection methods provided by the above methods.

[0142] In another aspect, this application also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to perform the salient target detection methods provided by the above methods.

[0143] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0144] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0145] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A salient target detection method, characterized in that, include: The color intensity values ​​of each pixel in the target image in the RGB three channels are input into the spiking neuron coding layer, and the first encoding result of pulse encoding of the color intensity values ​​of each pixel in each of the RGB three channels is obtained from the output of the spiking neuron coding layer. The first encoding result of the color intensity value of each pixel in the RGB three channels is input into the spiking neuron feature processing layer constructed based on multiple spiking neurons and channel splicing layers to obtain the first simulated visual encoding information of each pixel output by the spiking neuron feature processing layer. The first analog visual coding information of each pixel is input into a skip coding layer constructed using a residual structure, and the second coding result of each pixel is obtained from the output of the skip coding layer. The second encoding result of each pixel is input into a self-attention mechanism layer and a multilayer perceptron layer with residual connections constructed based on spiking neurons, and the second simulated visual encoding information of each pixel is obtained after the second encoding result of each pixel is enhanced based on the self-attention mechanism layer and the multilayer perceptron layer. The first and second simulated visual encoding information of each pixel is input into an edge mapping layer constructed based on multiple spiking neurons, multiple interpolation sampling layers, and multiple feature concatenation layers. The attention edge image of the target image output by the edge mapping layer is obtained, and the attention edge image is used as the detection result.

2. The salient target detection method according to claim 1, characterized in that, The spiking neuron feature processing layer includes a first primary color feature processing layer, a second primary color feature processing layer, and a third primary color feature processing layer constructed based on multiple spiking neurons; The first primary color feature processing layer is used to obtain the first primary color encoding result of the color intensity value of each pixel in the first channel of the RGB three channels in the first encoding result, perform a spiking neuron operation on the first primary color encoding result and then perform a max pooling operation to output the first primary color spiking encoding information to the channel splicing layer. The second primary color feature processing layer is used to obtain the second primary color encoding result of the color intensity value of each pixel in the second channel of the RGB three channels in the first encoding result, perform a spiking neuron operation on the second primary color encoding result and then perform a max pooling operation to output the second primary color spiking encoding information to the channel splicing layer. The third primary color feature processing layer is used to obtain the third primary color encoding result of the color intensity value of each pixel in the third channel of the RGB three channels in the first encoding result, perform a spiking neuron operation on the third primary color encoding result and then perform a max pooling operation to output the third primary color spiking encoding information to the channel splicing layer. The channel splicing layer is used to splice the first primary color pulse coding information, the second primary color pulse coding information and the third primary color pulse coding information to output the first analog visual coding information of each pixel.

3. The salient target detection method according to claim 2, characterized in that, The skip coding layer includes: a first feature extraction layer, a second feature extraction layer, and a fusion layer; The first feature extraction layer is used to receive the first simulated visual encoding information of each pixel, extract features from the first simulated visual encoding information of each pixel, and output the first feature map of each pixel to the second feature extraction layer and the fusion layer. The second feature extraction layer is used to perform spiking neuron operations on the first feature map of each pixel and then perform feature extraction, and output the second feature map of each pixel to the fusion layer; The fusion layer is used to sum the first feature map and the second feature map of each pixel to obtain a first summation result, and after performing a spiking neuron operation on the first summation result, output the second encoding result of each pixel.

4. The salient target detection method according to claim 3, characterized in that, The self-attention mechanism layer is used for: Obtain the second feature map of each pixel; The second feature map is bisected along the channel dimension to obtain a first bisected feature map and a second bisected feature map. Based on multiple spiking neurons, residual connections are applied between the first bisecting feature map and the second bisecting feature map to obtain attention values; Feature extraction is performed on the attention values ​​to generate a third feature map; Obtain a second summation result by summing the third feature map and the second encoding result, and determine the second summation result as the self-attention mechanism processing result.

5. The salient target detection method according to claim 4, characterized in that, The multilayer perceptron layer is used for: After performing spiking neuron operations on the self-attention mechanism processing results, the first operation result is output. The first operation result is then normalized and then convolutional to obtain the fourth feature map. After performing a spiking neuron operation on the fourth feature map, a second operation result is output. After performing a convolution operation on the second operation result, a normalization operation is performed to obtain a fifth feature map. The fourth feature map and the fifth feature map are summed to obtain a third summation result. After performing spiking neuron operations on the third summation result, feature extraction is performed to obtain a sixth feature map. The sixth feature map and the self-attention mechanism processing result are summed to obtain a fourth summation result. The fourth summation result is determined as the second simulated visual coding information of each pixel.

6. The salient target detection method according to claim 5, characterized in that, The edge mapping layer includes a first pulse convolutional layer, a second pulse convolutional layer, a third pulse convolutional layer, and a fourth pulse convolutional layer constructed based on multiple spiking neurons; the multiple feature splicing layers include a first feature splicing layer and a second feature splicing layer; the multiple interpolation sampling layers include a first interpolation sampling layer, a second interpolation sampling layer, and a third interpolation sampling layer; The first pulse convolutional layer is used to perform feature extraction and spiking neuron operation on the first simulated visual coding information of each pixel, and output the first processing result to the first feature splicing layer. The first feature splicing layer is used to splice the first processing result and the original feature information of each pixel, and output the second processing result to the second pulse convolution layer among the plurality of pulse convolution layers; The second pulse convolutional layer is used to extract features and perform spiking neuron operations on the second processing result, and output the third processing result to the first interpolation sampling layer; The first interpolation sampling layer is used to perform interpolation sampling on the third processing result based on nearest neighbor interpolation, and output the first interpolation sampling result to the second feature splicing layer; The third pulse convolutional layer is used to perform feature extraction and spiking neuron operations on the second simulated visual coding information of each pixel, and outputs the fourth processing result to the second feature splicing layer. The second feature splicing layer is used to splice the fourth processing result and the first interpolation sampling result, and output the fifth processing result to the fourth pulse convolutional layer; The fourth pulse convolutional layer is used to extract features and perform spiking neuron operations on the fifth processing result, and output the sixth processing result to the third interpolation sampling layer; The third interpolation sampling layer is used to interpolate and sample the sixth processing result and then output the attention edge image.

7. A salient target detection device, characterized in that, include: The encoding processing module is used to input the color intensity values ​​of each pixel in the RGB three-channel of the target image into the spiking neuron encoding layer, and obtain the first encoding result output by the spiking neuron encoding layer, which performs pulse encoding on the color intensity values ​​of each pixel in each of the RGB three-channel; The feature processing module is used to input the first encoding result of the color intensity value of each pixel in the RGB three channels into the spiking neuron feature processing layer constructed based on multiple spiking neurons and channel splicing layers, and obtain the first simulated visual encoding information of each pixel output by the spiking neuron feature processing layer; The skip coding module inputs the first analog visual coding information of each pixel into the skip coding layer constructed using a residual structure, and obtains the second coding result of each pixel output by the skip coding layer. An enhancement processing module is used to input the second encoding result of each pixel into a self-attention mechanism layer and a multilayer perceptron layer with residual connections constructed based on spiking neurons, and to obtain the second simulated visual encoding information of each pixel after the second encoding result of each pixel is enhanced based on the self-attention mechanism layer and the multilayer perceptron layer. The mapping processing module is used to input the first and second simulated visual coding information of each pixel into an edge mapping layer constructed based on multiple spiking neurons, multiple interpolation sampling layers, and multiple feature splicing layers, to obtain the attention edge image of the target image output by the edge mapping layer, and to use the attention edge image as the detection result.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the salient target detection method as described in any one of claims 1 to 6.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the salient target detection method as described in any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the salient target detection method as described in any one of claims 1 to 6.

Images