An image edge detection algorithm, device and medium based on a pulse neural network

Abstract

The present application relates to the technical field of digital image processing, and particularly relates to an image edge detection algorithm and device based on a pulse neural network and a medium. A VGG16 model is used to extract feature information of an image, and an attention information matrix of the feature information is extracted, and a CNSNP operation is performed on the attention information matrix to obtain a first information matrix; matrix point multiplication is performed on the feature information and the first information matrix to obtain a second information matrix, and element-by-element addition is performed on the second information matrix and the first information matrix to obtain a third information matrix, and finally, a CNSNP operation is performed on the third information matrix to obtain a fourth information matrix; and finally, the fourth information matrix is converted into an edge information image for output. The method has higher precision, and an attention mechanism is introduced, so that rich context information between adjacent keys can be fully utilized, and finally, image feature information is fused through multiple CNSNP operations, so that the output image has rich texture information and contour information in the high-level side output.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of digital image processing, and particularly relates to an image edge detection algorithm based on a pulse neural network, a device and a medium. BACKGROUND

[0002] Edge detection is an important research direction in the field of computer vision. At present, in the field of deep learning, there are many processing methods for edge detection problems. The edge detection task has the following research and optimization directions:

[0003] 1. Algorithm optimization: continuously improve the efficiency and accuracy of edge detection algorithms, such as optimizing edge extraction operators, optimizing filter design, and using deep learning methods.

[0004] 2. Parameter adjustment: adjust the parameters of the edge detection algorithm, such as smoothing degree, threshold, etc., to adapt to different scenes and images.

[0005] 3. Multi-scale processing: use multi-scale edge detection methods to effectively identify edges at different scales and improve the robustness of the algorithm.

[0006] 4. Edge connection: optimize edge connection algorithms to solve problems such as broken edges and missed detection, and improve the continuity of overall edge detection.

[0007] 5. Data augmentation: use data augmentation techniques to increase data sample diversity and improve the generalization ability of edge detection algorithms.

[0008] 6. Hardware acceleration: use GPU, TPU and other hardware acceleration technologies to speed up the running speed of edge detection algorithms.

[0009] 7. Introduction of domain knowledge: combine the knowledge of specific application fields to design targeted edge detection methods to improve the effectiveness of the algorithm.

[0010] Among the current popular edge detection algorithms, DeepEdge uses multi-stage CNN to extract target perception clues for contour detection. HED learns rich hierarchical features through supervised side output layers to improve edge detection performance. RCF combines hierarchical features from all convolutional layers into a unified architecture. To obtain effective results, BDCN uses layer-specific supervision inferred from a bidirectional cascading structure to guide the training of each layer. PiDiNet integrates traditional edge detection operators into the CNN model to improve performance. CHCN uses a cascaded and high-resolution network to obtain fine edge detection by maintaining high resolution of edges during the training process.

[0011] In mainstream convolutional neural networks, as the network depth increases, the receptive field gradually expands, and the features gradually tend to global information, resulting in the loss of the most detailed information. In order to solve this problem, some methods aggregate low-level information and high-level information in the CNN network to obtain more accurate edge detection results. However, in these existing detectors, edge blur mixing of the edge output is easy to occur, that is, all pixels are treated as errors Average, resulting in the inability to correctly fuse the rich texture information in the low-level side output and the contour information in the high-level side output. SUMMARY

[0012] In view of the deficiencies in the prior art, the present application provides an image edge detection algorithm based on a spiking neural network, a device and a medium, which solves the problem of edge blur mixing in the current image edge detection algorithm, that is, all pixels are treated as errors Average, resulting in the inability to correctly fuse the rich texture information in the low-level side output and the contour information in the high-level side output.

[0013] The present application discloses an image edge detection algorithm based on a spiking neural network through an embodiment, which comprises the following steps:

[0014] The feature information of the image is extracted, the attention information matrix of the feature information is extracted, the CNSNP operation is performed on the attention information matrix to obtain a first information matrix; the feature information and the first information matrix are subjected to matrix point multiplication to obtain a second information matrix, and then the second information matrix and the first information matrix are subjected to element-by-element addition to obtain a third information matrix, and finally the CNSNP operation is performed on the third information matrix to obtain a fourth information matrix; the fourth information matrix is converted into an edge information image and output.

[0015] The CNSNP operation comprises the following steps: S1, performing convolution operation on the matrix, then performing batch normalization on the convolution result, and finally performing ReLU activation function on the elements in the normalized result matrix; S2, the result of the ReLU activation function is subjected to the S1 step again. It should be noted that when performing the ReLU activation function, the activation function operation is performed on the elements in the normalized matrix.

[0016] Further, the feature information of the image is extracted by using a VGG16 model.

[0017] Further, the feature information comprises an edge detail feature map, a contour information feature map and a context information feature map of the image.

[0018] Further, the attention information matrix of the feature information is extracted by using a context Transformer.

[0019] In order to achieve the above object, the application further provides an image edge detection device based on a pulse neural network, comprising:

[0020] a feature extraction module for extracting feature information of the image;

[0021] an attention extraction module for extracting an attention information matrix of the feature information;

[0022] a feature fusion module for performing a CNSNP operation on the attention information matrix to obtain a first information matrix, performing a matrix point multiplication of the feature information and the first information matrix to obtain a second information matrix, performing an element-by-element addition of the second information matrix and the first information matrix to obtain a third information matrix, and finally performing a CNSNP operation on the third information matrix to obtain a fourth information matrix;

[0023] an output module for converting the fourth information matrix into an edge information image and outputting the same.

[0024] In order to achieve the above object, the application further provides a storage medium storing a plurality of computer instructions for executing the image edge detection algorithm based on the pulse neural network.

[0025] Compared with the prior art, the application has the following beneficial effects: the VGG16 model is used to extract the feature information of the image, and the precision is higher. Meanwhile, the attention mechanism is introduced, the rich context information between adjacent keys can be fully utilized, and finally the image feature information is fused through multiple CNSNP operations, so that the output image has rich texture information and contour information in the high-level side output. BRIEF DESCRIPTION OF DRAWINGS

[0026] In order to more clearly illustrate the technical solutions of the application, and to facilitate further understanding of the technical effects, technical features and purposes of the application, the application will be described in detail below with reference to the drawings, which constitute an integral part of the specification and are used to explain the technical solutions of the application together with the embodiments of the application, but do not constitute a limitation of the application.

[0027] Figure 1 a flowchart of the method described in the application;

[0028] Figure 2 a flowchart of the CNSNP operation in the method described in the application;

[0029] Figure 3 an operation schematic diagram of the method described in the application;

[0030] Figure 4 a schematic diagram for extracting the attention information matrix using the Transformer in the method described in the application. DETAILED DESCRIPTION

[0031] The application will be described in further detail below with reference to the drawings and embodiments. Of course, the specific embodiments described below are only to explain the technical solutions of the application, and are not a limitation on the application. In addition, the parts described in the embodiments or the drawings are only illustrative of the relevant parts of the application, and not the whole of the application. Meanwhile, all other embodiments obtained by those of ordinary skill in the art based on the embodiments of the application shall of course fall within the protection scope of the application.

[0032] As shown in Figures 1-4 The application proposes an image edge detection algorithm based on a pulse neural network, including the following steps:

[0033] A1, extracting feature information of an image, extracting an attention information matrix of the feature information, and performing a CNSNP operation on the attention information matrix to obtain a first information matrix;

[0034] A2, performing matrix point multiplication on the feature information and the first information matrix to obtain a second information matrix, performing element-wise addition on the second information matrix and the first information matrix to obtain a third information matrix, and finally performing a CNSNP operation on the third information matrix to obtain a fourth information matrix;

[0035] A3, converting the fourth information matrix into an edge information image for output.

[0036] The CNSNP operation includes the following steps: S1, performing a convolution operation on a matrix, then performing batch normalization on the convolution result, and finally performing a ReLU activation function on the normalized result; S2, performing the S1 step again on the result of the ReLU activation function.

[0037] In this embodiment, the feature information of the image is extracted by using a VGG16 model.

[0038] Meanwhile, it is worth noting that, compared with other feature extraction models, the VGG16 model is used to extract the feature information of the image. Since the structure of VGG16 is relatively simple, the convolution layers are divided into 5 stages, and a pooling layer is connected after the end of each stage. With the increase of the receptive field size, the feature information obtained by each convolution layer gradually approaches the contour information. By constructing a side output, each convolution layer can obtain more rich hierarchical information. Therefore, VGG16 is adopted as a feature extraction network to complete the feature extraction of the image. Since an image usually contains low-level edge details, high-level texture details and context information, VGG16 has 5 side outputs, which contain different edge-related information from top to bottom. Then, the 5 side outputs are mapped and connected together by a connection operation as the feature information.

[0039] In the embodiment, the feature information includes an edge detail feature map, a contour information feature map and a context information feature map of the image.

[0040] In the embodiment, the attention information matrix of the feature information includes using a context Transformer to extract the attention information matrix.

[0041] It is worth noting that, as Figure 4 shown, for using the Transformer to extract the attention information matrix. Assuming the input feature map information E cot ∈R H×W×C , where H, W and C correspond to the height, width and channel number of the feature map E respectively. The key, query and value are defined as K=E, Q=E and V=EW v .

[0042] Assuming S is the static context information obtained after the feature map E is input into the Transformer module, which can be expressed as:

[0043] S=[K S Q]W θ W δ

[0044] For input, the Transformer module first uses a 3*3 convolution to spatially encode the input to obtain the key K S containing static context information. Then the key K S is connected with the query Q, encoded by W θ (1*1 convolution containing ReLU activation function) and W δ (1*1 convolution without activation function), to obtain the key S with static context information.

[0045] Corresponding to the dynamic context information, assuming that the feature map E obtained after inputting the Transformer module can be expressed as:

[0046]

[0047] By fusing the value V with the extracted static context information, D with dynamic context information is obtained.

[0048] For the input feature map E, the output Y cot ∈R H×W×5 obtained after processing by the Transformer module is:

[0049]

[0050] In order to achieve the above object, the application further provides an image edge detection device based on a pulse neural network, comprising:

[0051] a feature extraction module for extracting feature information of the image;

[0052] an attention extraction module for extracting an attention information matrix of the feature information;

[0053] a feature fusion module for performing a CNSNP operation on the attention information matrix to obtain a first information matrix, performing a matrix point multiplication of the feature information and the first information matrix to obtain a second information matrix, performing an element-by-element addition of the second information matrix and the first information matrix to obtain a third information matrix, and finally performing a CNSNP operation on the third information matrix to obtain a fourth information matrix;

[0054] an output module for converting the fourth information matrix into an edge information image and outputting the same.

[0055] In order to achieve the above object, the application further provides a storage medium storing a plurality of computer instructions for executing the image edge detection algorithm based on the pulse neural network.

[0056] It should be noted that the above embodiments are only for more clearly illustrating the technical solutions of the application, and those skilled in the art can understand that the embodiments of the application are not limited to the above content, and the obvious changes, replacements or substitutions based on the above content do not exceed the scope of the technical solutions of the application; other embodiments will certainly fall within the scope of the application without departing from the concept of the application.

Claims

1. An image edge detection algorithm based on a spiking neural network, characterized in that, The method comprises the following steps: extracting feature information of an image, extracting an attention information matrix of the feature information, performing CNSNP operation on the attention information matrix to obtain a first information matrix; performing matrix point multiplication on the feature information and the first information matrix to obtain a second information matrix, performing element-wise addition on the second information matrix and the first information matrix to obtain a third information matrix, and finally performing CNSNP operation on the third information matrix to obtain a fourth information matrix; converting the fourth information matrix into an edge information image for output; The CNSNP operation comprises the following steps: S1, performing convolution operation on a matrix, then performing batch normalization on the convolution result, and finally performing ReLU activation function on the normalized result; S2, performing S1 step again on the result of the ReLU activation function.

2. The pulse-based neural network based image edge detection algorithm as claimed in claim 1, wherein, The feature information of the image is extracted by using a VGG16 model.

3. The pulse-based neural network based image edge detection algorithm as claimed in claim 1 or 2, wherein, The feature information comprises edge detail feature maps, contour information feature maps and context information feature maps of the image.

4. The pulse-based neural network based image edge detection algorithm as claimed in claim 1, wherein, The attention information matrix of the feature information is extracted by using a context Transformer.

5. An image edge detection apparatus based on a spiking neural network, characterized by, An image edge detection algorithm based on a pulse neural network is executed, comprising: a feature extraction module for extracting feature information of an image; an attention extraction module for extracting an attention information matrix of the feature information; a feature fusion module for performing CNSNP operation on the attention information matrix to obtain a first information matrix, performing matrix point multiplication on the feature information and the first information matrix to obtain a second information matrix, performing element-wise addition on the second information matrix and the first information matrix to obtain a third information matrix, and finally performing CNSNP operation on the third information matrix to obtain a fourth information matrix; an output module for converting the fourth information matrix into an edge information image for output.

6. A storage medium characterized by: A plurality of computer instructions are stored for executing the image edge detection algorithm based on the pulse neural network. A plurality of computer instructions are stored for executing the image edge detection algorithm based on the pulse neural network.

Images