A contour detection method based on a multi-level interaction mechanism of biological vision

Abstract

The application aims to provide a contour detection method based on a biological vision multi-level interaction mechanism, comprising the following steps: constructing a deep neural network structure, the deep neural network structure being specifically as follows: an encoding network, a decoding network; wherein the encoding network comprises a VGG16 network, an LFE module and a multi-level interaction network; the VGG16 network is divided into five stages with a pooling layer as a boundary, and the five stages are respectively a Retina stage, an LGN stage, a V1 stage, a V2 stage and a V4 stage; an original image sequentially passes through the encoding network and the decoding network to obtain a final output contour; the application guarantees the integrity of a target contour and can effectively solve the problem of discontinuous contour; and the application designs a multi-level interaction network in the encoding network, so that visual information at different stages is exchanged and optimized, and the contour detection performance can be improved.

Description

Technical Field

[0001] This invention relates to the field of image processing, and more specifically to a contour detection method based on a multi-level interaction mechanism of biological vision. Background Technology

[0002] Contour detection aims to extract the boundary between the background and the target in an image. It is typically a key step in the front-end processing of various intermediate and advanced computer vision tasks and is one of the fundamental tasks in computer vision research. Currently, contour detection employs two main methods:

[0003] Deep learning methods typically employ publicly available models such as VGG-Net and Res-Net for transfer learning, using them as encoding networks to represent features. Then, a matching decoding network is researched and designed to analyze these features and ultimately obtain the target contour. While convolutional neural networks themselves are inspired by biological mechanisms, their integration with these mechanisms has not been well-developed in later stages.

[0004] Biological methods: Simply simulating a part of the physiological characteristics of the visual system, in the process of contour extraction, some dynamic characteristics of vision cannot be simulated well, resulting in problems such as the loss of contour information and the enhancement of texture information, thus failing to ensure the integrity of the target contour. Summary of the Invention

[0005] This invention aims to provide a contour detection method based on a multi-level interaction mechanism of biological vision. This method proposes a visual information feedback pathway and a weak and diffuse interconnection with cross-level transmission. A multi-level interactive network structure is designed in the coding network to enable visual information at different stages to communicate and be optimized, ultimately improving the contour detection performance.

[0006] The technical solution of the present invention is as follows:

[0007] The contour detection method based on the multi-level interaction mechanism of biological vision includes the following steps:

[0008] A. Construct a deep neural network structure, the specific structure of which is as follows:

[0009] Encoding networks, decoding networks;

[0010] The coding network includes a VGG16 network, an LFE module, and a multi-level interactive network.

[0011] The VGG16 network is divided into five stages based on the pooling layer: Retina stage, LGN stage, V1 stage, V2 stage, and V4 stage.

[0012] B. The original image is input into the VGG16 network and undergoes five stages of convolution processing to obtain five output images, which are then input into the LFE module and the decoding network, respectively.

[0013] C. In the LFE module, the five output images input to the VGG16 network are divided into nine square regions of the same size using a nine-grid method, resulting in nine two-dimensional images of the regions. The nine two-dimensional images of the regions at each stage are then transformed into nine one-dimensional data points. For each of the nine one-dimensional data points at each stage, the one-dimensional data points are numbered according to their positions in the input images of the respective regions, and then input into the multi-level interactive network.

[0014] D. In multi-level interactive networks:

[0015] One-dimensional data with the same number in each stage are grouped together to obtain nine one-dimensional data groups. Each one-dimensional data group is processed by a self-attention mechanism to obtain nine information interaction data groups after information interaction. The one-dimensional data information in the nine information interaction data groups is regrouped according to the original stage, converted back into two-dimensional images and stitched together to obtain five information interaction images that correspond one-to-one with the five output images, which are then input into the decoding network.

[0016] E. The decoding network integrates the features of the five output images and the five information interaction images to obtain the final output contour.

[0017] In step C, the formula for performing nine-grid segmentation on the five output images input to the VGG16 network is as follows:

[0018] Set the output image size to X*Y, the convolution kernel size to L*L, the stride to S, and the number of pixels filled in the output image to P;

[0019] O X =(X-L+2P) / S+1

[0020] O Y =(Y-L+2P) / S+1 (1)

[0021] Among them, O N O represents the length of the segmented image. M This represents the width of the segmented image.

[0022] In step C, the linearization module of PyTorch is used to transform the two-dimensional images of the nine regions in each stage into nine one-dimensional information data.

[0023] In step D, the calculation formula for the self-attention mechanism is as follows:

[0024] (2)

[0025] Where Q, K, and V represent vectors, K T d represents the transpose of vector K. k The dimension of vector K is represented by Q, K, and V, which all have the same dimension.

[0026] The decoding process of the decoding network is as follows:

[0027] Each information interaction image and its corresponding output image serve as two input results for the FIM module, which processes them to obtain five FIM processing results: the FIM result corresponding to the Retina stage, the FIM result corresponding to the LGN stage, the FIM result corresponding to the V1 stage, the FIM result corresponding to the V2 stage, and the FIM result corresponding to the V4 stage.

[0028] The FIM results corresponding to stage V2 and stage V4 are input into a single FIM module for processing. The output results are input into a single FIM module along with the FIM results corresponding to stage V1, stage LGN, and stage Retina. The output results are then input into a single FIM module to obtain the final output contour.

[0029] The processing procedure in the FIM module is as follows:

[0030] The low-resolution input result is sequentially processed by N*N convolution, instance normalization algorithm, and ReLU function, and then deconvolutioned to make its resolution the same as the high-resolution input result, resulting in processing result I; the high-resolution input result is sequentially processed by N*N convolution, instance normalization algorithm, and ReLU function, resulting in processing result II; where N=3, 5, or 7;

[0031] The output result is obtained by adding and fusing the processing result I and processing result II.

[0032] Preferably, N is 3.

[0033] The method of this invention generates modulation features at different stages of the coding network using a multi-level interactive network, optimizes each stage using these modulation features, and then integrates information from different visual levels. This approach enables the optimization of lower-level visual levels using higher-level visual levels, thereby improving contour detection performance. Attached Figure Description

[0034] Figure 1 This is a schematic diagram of the overall structure of the neural network in Embodiment 1 of the present invention;

[0035] Figure 2 This is a schematic diagram of the LFE module and multi-level interactive network in Embodiment 1 of the present invention;

[0036] Figure 3 This is a schematic diagram of the FIM module in Embodiment 1 of the present invention;

[0037] Figure 4 This is a comparison diagram of the contour detection effects of the scheme in Embodiment 1 of the present invention and the scheme in Document 1. Detailed Implementation

[0038] The present invention will now be described in detail with reference to the accompanying drawings and embodiments. Example

[0039] A contour detection method based on a multi-level interaction mechanism of biological vision includes the following steps:

[0040] A. Construct a deep neural network structure, such as Figure 1-3 As shown, the specific structure of a deep neural network is as follows:

[0041] Encoding networks, decoding networks;

[0042] The coding network includes a VGG16 network, an LFE module, and a multi-level interactive network.

[0043] The VGG16 network is divided into five stages based on the pooling layer: Retina stage, LGN stage, V1 stage, V2 stage, and V4 stage.

[0044] B. The original image is input into the VGG16 network and undergoes five stages of convolution processing to obtain five output images, which are then input into the LFE module and the decoding network, respectively.

[0045] C. In the LFE module, the five output images input to the VGG16 network are divided into nine square regions of the same size using a nine-grid segmentation, resulting in nine two-dimensional images of the regions. The linearization module of PyTorch is used to transform the nine two-dimensional images of the regions at each stage into nine one-dimensional data points. For the nine one-dimensional data points at each stage, the one-dimensional data points are numbered according to their positions in the input images of each region, and then input into the multi-level interactive network respectively.

[0046] In step C, the formula for performing nine-grid segmentation on the five output images input to the VGG16 network is as follows:

[0047] Set the output image size to X*Y, the convolution kernel size to L*L, the stride to S, and the number of pixels filled in the output image to P;

[0048] O X=(X-L+2P) / S+1

[0049] O Y =(Y-L+2P) / S+1 (1)

[0050] Among them, O N O represents the length of the segmented image. M Represents the width of the segmented image;

[0051] D. In multi-level interactive networks:

[0052] One-dimensional data with the same number in each stage are grouped together to obtain nine one-dimensional data groups. Each one-dimensional data group is processed by a self-attention mechanism to obtain nine information interaction data groups after information interaction. The one-dimensional data information in the nine information interaction data groups is regrouped according to the original stage, converted back into two-dimensional images and stitched together to obtain five information interaction images that correspond one-to-one with the five output images, which are then input into the decoding network.

[0053] In step D, the calculation formula for the self-attention mechanism is as follows:

[0054] (2)

[0055] Where Q, K, and V represent vectors, K T d represents the transpose of vector K. k The dimension of vector K is represented by the dimension of vectors Q, K, and V, which are all equal.

[0056] E. The decoding network integrates the features of the five output images and the five information interaction images to obtain the final output contour.

[0057] The decoding process of the decoding network is as follows:

[0058] Each information interaction image and its corresponding output image serve as two input results for the FIM module, which processes them to obtain five FIM processing results: the FIM result corresponding to the Retina stage, the FIM result corresponding to the LGN stage, the FIM result corresponding to the V1 stage, the FIM result corresponding to the V2 stage, and the FIM result corresponding to the V4 stage.

[0059] The FIM results corresponding to stage V2 and stage V4 are input into a single FIM module for processing. The output results are input into a single FIM module along with the FIM results corresponding to stage V1, stage LGN, and stage Retina. The output results are then input into a single FIM module to obtain the final output contour.

[0060] The processing procedure in the FIM module is as follows:

[0061] The low-resolution input result is sequentially processed by N*N convolution, instance normalization algorithm, and ReLU function, and then deconvolutioned to make its resolution the same as the high-resolution input result, resulting in processing result I; the high-resolution input result is sequentially processed by N*N convolution, instance normalization algorithm, and ReLU function, resulting in processing result II; where N=3;

[0062] The output result is obtained by adding and fusing the processing result I and processing result II. Example

[0063] For the quantitative performance evaluation of the final edge graph, we adopted the same performance measurement standard as in Reference 1, and the specific evaluation is shown in Formula (3).

[0064] (3)

[0065] in, Indicates accuracy. This indicates the recall rate. The larger the value, the better the performance.

[0066] Document 1: Xie S, Tu Z. Holistically-Nested Edge Detection[J].International Journal of Computer Vision, 2017, 125(1-3): 3.

[0067] The parameters used in Reference 1 are the same as those in the original text, and they are guaranteed to be the optimal parameters for the model.

[0068] Figure 4 The image shows, from left to right, three randomly selected natural images from the Berkeley Segmentation Dataset (BSDS500), their corresponding ground truth contour maps, the optimal contour map detected by the method in Reference 1, and the optimal contour map detected by the method in this paper; Table 1 compares the performance of the edge detection method provided in Example 1 with that in Reference 1.

[0069] ODS OIS AP Reference 1 Method 0.790 0.808 0.811 Example 1 0.822 0.843 0.875

[0070] Based on the experimental results, the detection method in Example 1 is superior to the detection method in Reference 1.

Claims

1. A contour detection method based on a multi-level interaction mechanism of biological vision, characterized in that It comprises the following steps: A. Constructing a deep neural network structure, which is specifically as follows: An encoding network, a decoding network; The encoding network comprises a VGG16 network, an LFE module, and a multi-level interaction network; The VGG16 network is divided into five stages by pooling layers, namely, a Retina stage, an LGN stage, a V1 stage, a V2 stage, and a V4 stage; B. The original image is input into the VGG16 network and sequentially subjected to convolution processing in the five stages to obtain five output images, which are input into the LFE module and the decoding network; C. In the LFE module, the five output images input by the VGG16 network are divided into nine square regions of the same size by nine-square segmentation to obtain nine two-dimensional images of the regions; the nine two-dimensional images of the regions of each stage are converted into nine one-dimensional information data; for the nine one-dimensional data information of each stage, the one-dimensional data information is numbered according to the position of each region image in the input image, and then input into the multi-level interaction network; D. In the multi-level interaction network: The one-dimensional data of the same number in each stage is taken as a group to obtain nine one-dimensional data groups, each one-dimensional data group is subjected to self-attention mechanism processing to obtain nine information interaction data groups after information interaction; the one-dimensional data information in the nine information interaction data groups is re-grouped according to the original stage, re-converted into two-dimensional images, and spliced to obtain five information interaction images corresponding to the five output images, which are input into the decoding network; E. The decoding network obtains the final output contour after feature integration of the five output images and the five information interaction images.

2. The contour detection method based on the multi-level interaction mechanism of biological vision according to claim 1, wherein: In step C, the formula for dividing the five output images input by the VGG16 network into nine-square segments is: The size of the output image is X*Y, the size of the convolution kernel is L*L, the step is S, and the number of pixels filled in the output image is P. O X = (X - L + 2P) / S + 1 O Y = (Y - L + 2P) / S + 1 (1) where O N represents the length of the partitioned picture, O M represents the width of the partitioned picture.

3. The contour detection method based on the multi-level interaction mechanism of biological vision according to claim 1, wherein: In step C, the pytorch linearization module is used to convert the nine two-dimensional images of the regions of each stage into nine one-dimensional information data.

4. The contour detection method based on the multi-level interaction mechanism of biological vision according to claim 1, wherein: In step D, the calculation formula of the self-attention mechanism is: （2） where Q, K, V represent vectors, respectively, K T represents the transpose of K vector, d k represents the dimension of vector K, the dimensions of Q, K, V vectors are equal.

5. The contour detection method based on the multi-level interaction mechanism of biological vision according to claim 1, wherein: The decoding process of the decoding network is as follows: Each information interaction image and its corresponding output image are taken as two input results of the FIM module, processed by the FIM module to obtain five FIM processing results, namely, the FIM result corresponding to the Retina stage, the FIM result corresponding to the LGN stage, the FIM result corresponding to the V1 stage, the FIM result corresponding to the V2 stage, and the FIM result corresponding to the V4 stage. The FIM module processes the FIM result corresponding to the V2 stage and the FIM result corresponding to the V4 stage, and outputs a result; the FIM module processes the result corresponding to the V1 stage and the result, and outputs a result; the FIM module processes the result corresponding to the LGN stage and the result, and outputs a result; the FIM module processes the result corresponding to the Retina stage and the result, and outputs a final output contour.

6. The contour detection method based on the multi-level interaction mechanism of biological vision, according to claim 5, wherein: The processing procedure of the FIM module is as follows: The low-resolution input result is sequentially subjected to N*N convolution, Instance normalization algorithm calculation, Relu function processing, and then deconvolution to make the resolution the same as that of the high-resolution input result, to obtain processing result I; the high-resolution input result is sequentially subjected to N*N convolution, Instance normalization algorithm calculation, and Relu function processing, to obtain processing result II; wherein N = 3, 5, or 7; The output result is obtained by adding and fusing processing result I and processing result II.

7. The method of claim 6, wherein the method further comprises: determining a plurality of contours of the object based on the plurality of images; and determining a plurality of contours of the object based on the plurality of images. The N is 3.

Images