A target detection method under an occlusion condition

Abstract

The application discloses a target detection method under a shielding condition and relates to the technical field of target detection. The application takes improving the recognition accuracy of an AI vision algorithm under a shielding condition as a starting point and proposes a target detection method under a shielding condition. After a certain picture or video is input into a target detection framework after pre-processing, the shielding mode of the input image is learned, and feature information is differentially processed from spatial dimensions and channel dimensions according to the shielding condition. The method considers the correlation between spatial attention and channel attention, matches the shielding mode through spatial attention, and guides the channel attention to decompose the feature information. Ideal results are obtained through testing on a shielding dataset. The application has wide coverage, strong expansibility and can be conveniently integrated into an existing target detection algorithm framework.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of target detection, and particularly relates to a target detection method under a shielding condition. BACKGROUND

[0002] With the help of the superiority of deep learning in feature extraction, target detection technology has been widely applied in the fields of security, automatic driving, medical treatment and military, etc. However, deep learning belongs to the category of statistical learning and highly depends on data with high quality and large quantity. In real scenes, due to the existence of shielding, target detection is still a challenging task. The human visual system enables humans to continue and infer the attributes of objects through the existing outlines in the scene even if part of the information of the objects is shielded or lost. However, it is still difficult for computer vision systems based on deep learning to effectively detect shielded objects. The neural network model obtained by training cannot fully learn the complex shielding mode and has weak generalization ability and poor processing ability for scenes not appearing in the training set.

[0003] A general shielding target detection method can be simply divided into an input module, a feature extraction module, a detection network module, a post-processing module and an output module. The input module sends the photos taken by image acquisition devices such as cameras after data preprocessing to the feature extraction module; the feature extraction module extracts features of the input pictures through a backbone network, and the extracted features are sent to the detection network module for target prediction; the detection network module judges the target category of each prediction box according to the input features; the post-processing module aims to keep one detection result for each target and remove other redundant detection results; and the output module outputs the final detection result after labeling. The detection process is as shown in Figure 1 .

[0004] When the detected target is shielded, it will affect the analysis tasks such as feature extraction and recognition detection, and thus reduce the performance of the computer vision system. Therefore, how to improve the recognition accuracy of the target detection algorithm under the shielding condition in the actual scene is a very important research direction, and has important theoretical and guiding significance for visual landing application.

[0005] At present, the detection algorithm based on the attention mechanism can significantly improve the feature expression ability of the target. However, the existing attention methods do not fully consider the relevance of spatial attention and channel attention, resulting in limited feature enhancement effect of the proposed method and poor processing of the shielding condition.

[0006] For the case that the target is occluded, researchers have proposed various attention algorithms to improve the accuracy of target detection. In general, the related research can be divided into three strategy directions: (1) using spatial attention, focusing on where the features of the input image are visible, which compresses the channel information to produce a two-dimensional spatial attention mask; (2) using channel attention, focusing on what kind of features of the input image are visible, which compresses the spatial information to produce a one-dimensional channel attention weight; (3) using the combination of spatial attention and channel attention, which combines the advantages of both and considers where the features of the input image are visible and what kind of features are visible.

[0007] The first strategy has less current research, and the acquisition of spatial attention mostly adopts global maximum pooling and global average pooling operations. This method is simple, but the feature expression ability is weak, and the single spatial attention ignores the influence of channel information; the second strategy has more current research, different channels represent different parts of the target feature information, but the way of acquiring channel attention through global average pooling cannot capture rich input information well, and lacks feature diversity when processing different inputs; the third strategy combines the advantages of spatial attention and channel attention, but in the existing method, the acquisition of the two attentions is independent of each other, without fully considering the relevance between the two. SUMMARY

[0008] In view of the fact that the existing AI visual algorithm will have a relatively obvious decrease in recognition accuracy in the actual occlusion environment, and the research work on this problem is less, no unified improvement framework has been formed. The present application takes improving the recognition accuracy of AI visual algorithm under occlusion condition as the starting point, and proposes a target detection scheme under occlusion condition, which significantly reduces the model parameter quantity while ensuring the model accuracy by using the residual mechanism.

[0009] The first aspect of the present application discloses a target detection method under occlusion condition. The method comprises:

[0010] Step S1, calling an input module to collect image data, and pre-processing the image data including cropping operation and magnifying operation, taking the image data after the pre-processing as a target detection image, wherein the input module is an optical instrument device;

[0011] Step S2, calling a feature extraction module based on a convolutional neural network structure to extract image feature information from the target detection image, the feature extraction module comprising a backbone network and a visible region enhancement network;

[0012] Step S3: Call the detection network module to determine the prediction information based on the image feature information. The network layer of the detection network module determines the target confidence in the prediction box according to the learned parameters. The target with a confidence higher than the threshold is the prediction information.

[0013] Step S4: Call the post-processing module to select the optimal prediction result for each target from the prediction information and delete other redundant prediction results;

[0014] Step S5: Call the output module to label the optimal prediction result for each target and output it.

[0015] According to the method of the first aspect, the size of the target detection image is C×H×W, where H×W is the image size and C is the number of image channels; when the ResNet-based backbone network extracts features from the target detection image, a visible region enhancement network is added after each convolutional block. The visible region enhancement network consists of a spatial attention network and a channel attention network; the feature information map F (of size C×H×W) extracted by the convolutional block is used as the input of the visible region enhancement network, and the visible region enhancement network sequentially calls the spatial attention network to calculate the spatial attention A based on the feature information map. s (size is 1×H×W), and the channel attention network is invoked to calculate the channel attention A. c (Size C×1×1); The recombined feature information map F″ is composed of the feature information map F and the spatial attention A. s and the channel attention A c After calculation, the following was obtained:

[0016] F′=F×A s

[0017]

[0018] F″=F′×A c

[0019] Wherein, F′ represents the feature information after passing through the spatial attention network.

[0020] According to the method of the first aspect, the spatial attention network consists of a C1 convolutional layer, a C2 convolutional layer, and a C3 convolutional layer; wherein:

[0021] The C1 convolutional layer receives the feature information map F obtained by the ResNet convolutional block of the backbone network as input, and performs downsampling through convolution operation and SiLU nonlinear activation function activation operation. The convolution kernel size is 1×1, the convolution stride is 1, and the output channel is 1 / 2 of the input channel.

[0022] The C2 convolutional layer receives the feature information processed by the C1 convolutional layer, performs convolution operations and SiLU nonlinear activation function activation operations for downsampling, wherein the convolution kernel size is 3×3, the convolution stride is 1, and the output channel size is 1.

[0023] The C3 convolutional layer receives the feature information processed by the C2 convolutional layer, performs convolution operations and SiLU nonlinear activation function activation operations for downsampling, wherein the convolution kernel size is 3×3, the convolution stride is 1, and the output channel size is 1;

[0024] The feature information map F obtained by the ResNet convolutional block of the backbone network is then passed through the spatial attention network composed of three convolutional layers C1-C3 to obtain the spatial attention A. s The feature information F′ (of size C×H×W) after passing through the spatial attention network is represented as:

[0025]

[0026] In this context, the feature information of each channel c of the feature information map F is pixel-wise (i, j) and spatial attention A is used. s Multiply the corresponding pixels (i, j).

[0027] According to the method of the first aspect, the channel attention network consists of a convolutional summation layer CS and four fully connected layers F1-F4; wherein:

[0028] The CS convolutional summation layer uses the feature information map F and the spatial attention A. s Using the input as input, perform multiplication and summation of corresponding pixel points (i, j) to obtain A. cs (of size C×1×1), represented as:

[0029]

[0030] The fully connected layers F1 and F2 will A cs The number of channels gradually decreases in the proportions of 1 / 4 and 1 / 2;

[0031] The fully connected layers F3 and F4 gradually increase the number of channels based on the output of the fully connected layer F2 to obtain channel attention A. c ;

[0032] The feature information F″ (of size C×H×W) after passing through the channel attention network is:

[0033]

[0034] Wherein, the feature information of each channel of the feature information F' after the spatial attention network is multiplied by the attention weight of the corresponding channel to obtain F''. c The attention weight of the corresponding channel is multiplied to obtain F''.

[0035] The second aspect of the present application discloses a target detection system under occlusion conditions; the processing unit of the system is configured to sequentially execute:

[0036] Step S1, calling an input module to collect image data, and pre-processing the image data including cropping operation and magnification operation, taking the image data after the pre-processing as a target detection image, wherein the input module is an optical instrument device;

[0037] Step S2, calling a feature extraction module based on a convolutional neural network structure to extract image feature information from the target detection image, the feature extraction module including a backbone network and a visible region enhancement network;

[0038] Step S3, calling a detection network module to determine prediction information based on the image feature information, wherein the network layer of the detection network module determines the target confidence in the prediction box according to the learned parameters, and the target whose target confidence is higher than the threshold is the prediction information;

[0039] Step S4, calling the post-processing module to filter out an optimal prediction result of each target from the prediction information, and deleting other redundant prediction results;

[0040] Step S5, calling an output module to label and output the optimal prediction result of each target.

[0041] According to the system of the second aspect, the size of the target detection image is CxHxW, wherein HxW is the image size, and C is the image channel number; the backbone network based on Resnet increases a visible region enhancement network after each convolution block when extracting features from the target detection image, the visible region enhancement network is composed of a spatial attention network and a channel attention network; the feature information graph F (size CxHxW) extracted by the convolution block is input into the visible region enhancement network, the visible region enhancement network sequentially calls the spatial attention network to calculate the spatial attention A s (size 1xHxW), and calls the channel attention network to calculate the channel attention A c (size Cx1x1); the recombined feature information graph F'' is obtained by calculating the feature information graph F, the spatial attention A s and the channel attention A c ​

[0042] F' = F x A s

[0043]

[0044] F" = F' x A c

[0045] wherein F' represents the feature information after the spatial attention network.

[0046] According to the system of the second aspect, the spatial attention network is composed of a C1 convolution layer, a C2 convolution layer and a C3 convolution layer; wherein:

[0047] The C1 convolution layer receives the feature information map F obtained by the Resnet convolution block of the backbone network as input, and performs downsampling through convolution operation and SiLU nonlinear activation function activation operation, wherein the convolution kernel size is 1x1, the convolution step is 1, and the output channel is 1 / 2 of the input channel.

[0048] The C2 convolution layer receives the processed feature information of the C1 convolution layer, and performs convolution operation and SiLU nonlinear activation function activation operation for downsampling, wherein the convolution kernel size is 3x3, the convolution step is 1, and the output channel size is 1.

[0049] The C3 convolution layer receives the processed feature information of the C2 convolution layer, and performs convolution operation and SiLU nonlinear activation function activation operation for downsampling, wherein the convolution kernel size is 3x3, the convolution step is 1, and the output channel size is 1.

[0050] After the feature information map F obtained by the Resnet convolution block of the backbone network passes through the spatial attention network composed of three convolution layers C1-C3, the spatial attention A is obtained. s The feature information F' (size CxHxW) after the spatial attention network is represented as:

[0051]

[0052] wherein the feature information of each channel c of the feature information map F is multiplied pixel by pixel (i, j) with the corresponding pixel (i, j) of the spatial attention A. s

[0053] According to the system of the second aspect, the channel attention network is composed of a convolution sum layer CS and four fully connected layers F1-F4; wherein:

[0054] The CS convolution sum layer takes the feature information map F and the spatial attention A s ​For input, multiplication and summation between corresponding pixel points (i, j) are performed to obtain A cs (size Cx1x1), which is expressed as:

[0055]

[0056] The full connection layer F1 and the full connection layer F2 gradually reduce the number of channels of A cs by 1 / 4 and 1 / 2;

[0057] The full connection layer F3 and the full connection layer F4 gradually increase the number of channels based on the output of the full connection layer F2 to obtain channel attention A c .

[0058] The feature information F'' (size CxHxW) after the channel attention network is:

[0059]

[0060] The feature information of each channel of the feature information F' after the spatial attention network is multiplied by the attention weight of the corresponding channel of the channel attention A c to obtain F''.

[0061] The third aspect of the present application discloses an electronic device. The electronic device comprises a memory and a processor, the memory stores a computer program, and the processor implements the steps of the target detection method under the occlusion condition in any one of the first aspect of the present application when executing the computer program.

[0062] The fourth aspect of the present application discloses a computer readable storage medium. The computer readable storage medium stores a computer program, and the computer program implements the steps of the target detection method under the occlusion condition in any one of the first aspect of the present application when executed by a processor.

[0063] In summary, the technical scheme of the present application has wide coverage and strong expandability, and can be easily integrated into an existing target detection algorithm framework. After inputting a certain picture or video and pre-processing, the target detection framework is inputted, the method learns the occlusion mode of the input image, and the feature information is differentially processed from the spatial dimension and the channel dimension according to the occlusion condition. It is worth noting that the method considers the correlation between spatial attention and channel attention, and guides the channel attention to decompose the feature information after matching the occlusion mode through spatial attention. Through testing on an occlusion dataset, the present application achieves ideal results. BRIEF DESCRIPTION OF DRAWINGS

[0064] In order to more clearly illustrate the specific embodiments of the present application or the technical solutions in the prior art, the accompanying drawings required by the specific embodiments or prior art description will be briefly introduced below. Obviously, the accompanying drawings in the following description are some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor on the basis of these drawings.

[0065] Figure 1 It is a schematic diagram of the whole process of target detection in the prior art.

[0066] Figure 2 It is a schematic diagram of the process of a target detection method under occlusion condition according to an embodiment of the present application.

[0067] Figure 3 It is a schematic diagram of the principle of a feature extraction module according to an embodiment of the present application.

[0068] Figure 4 It is a schematic diagram of the principle of a visible region enhancement network according to an embodiment of the present application.

[0069] Figure 5 It is a schematic diagram of the principle of a spatial attention network according to an embodiment of the present application.

[0070] Figure 6 It is a schematic diagram of the principle of a channel attention network according to an embodiment of the present application.

[0071] Figure 7 It is a schematic diagram of the structure of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0072] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0073] The first aspect of the present application discloses a target detection method under occlusion condition. The method comprises:

[0074] Step S1, calling an input module to collect image data, and performing preprocessing including cropping operation and magnifying operation on the image data, taking the image data after the preprocessing as a target detection image, wherein the input module is an optical instrument device;

[0075] Step S2, calling a feature extraction module based on a convolutional neural network structure to extract image feature information from the target detection image, the feature extraction module comprising a backbone network and a visible region enhancement network;

[0076] Step S3, calling a detection network module to determine prediction information based on the image feature information, wherein the network layer of the detection network module determines the target confidence in the prediction box according to the learned parameters, and the target with a target confidence higher than a threshold is the prediction information;

[0077] Step S4, calling the post-processing module to filter out one optimal prediction result of each target from the prediction information, and deleting other redundant prediction results;

[0078] Step S5, calling the output module to label and output one optimal prediction result of each target.

[0079] Specifically, the application improves the feature extraction method on the basis of the traditional detection algorithm, and the improved method is called "visible region enhancement network" (VREN), as shown in Figure 2 The network can learn the occlusion mode of the target, and can improve the recognition accuracy in the occlusion environment.

[0080] Compared with the general occlusion target detection process, the method proposed by the application mainly acts on the feature extraction module. In the feature extraction module, the most commonly used residual network is used as the backbone network for feature extraction, and the proposed method is inserted after the residual block to perform feature reorganization, so that the finally extracted features can cope with the information interference caused by occlusion, as shown in Figure 3 .

[0081] Input module

[0082] The input module uses optical instrument equipment to collect image data, and performs preprocessing such as cropping and magnifying on the collected image to serve as the input of the target detection algorithm.

[0083] Feature extraction module

[0084] The image data is input into the feature extraction module based on the convolutional neural network structure for feature extraction, and the feature information of the input image is obtained through the backbone network and the visible region enhancement network.

[0085] Detection network module

[0086] The feature information is input into the detection network module, the network layer in the detection network determines the confidence of the target in the prediction box according to the learned parameters, and the target with high confidence is output as the prediction information.

[0087] post-processing module

[0088] The prediction information is input into the post-processing module, so that only one prediction result is retained for each target, and other redundant results are removed.

[0089] output module

[0090] The post-processed prediction result is input into the output module, and the output module outputs the final detection result after labeling.

[0091] According to the method of the first aspect, the size of the target detection image is CxHxW, wherein HxW is the image size, and C is the number of image channels; when the backbone network based on Resnet performs feature extraction on the target detection image, a visible region enhancement network is added after each convolution block, and the visible region enhancement network is composed of a spatial attention network and a channel attention network; the feature information map F extracted by the convolution block is input into the visible region enhancement network, the spatial attention network is called based on the feature information map F, the spatial attention A s is calculated, the channel attention network is called, and the channel attention A c is calculated; and the recomposed feature information map F" is obtained by calculating the feature information map F, the spatial attention A s , and the channel attention A c .

[0092] F' = F x A s

[0093]

[0094] F" = F' x A c

[0095] wherein F' represents the feature information after the spatial attention network.

[0096] Specifically, in some embodiments, the input is a certain occlusion image, and the output is a labeled target image. Image data is obtained, and the input size after preprocessing is 3xH0xW0, wherein H0xW0 is the size of the input image, and 3 is the number of channels of the input image.

[0097] The image data is input into a detection module based on a convolutional neural network structure for feature extraction. When a backbone network based on Resnet performs feature extraction, a visible region enhancement network is added after each convolution block, and the network is composed of spatial attention and channel attention.

[0098] The convolutional block extracts a feature information map F as input, and the visible region enhancement network sequentially infers spatial attention A s and channel attention A c . The recomposed feature information map F" is obtained by calculating the original feature information and the spatial attention and the channel attention, as shown in Figure 4 The overall calculation process is as follows:

[0099] F'=FxA s

[0100]

[0101] F"=F'xA c

[0102] The obtained recomposed feature map F" will be sent to the subsequent network layer for further processing.

[0103] According to the method of the first aspect, the spatial attention network is composed of a C1 convolutional layer, a C2 convolutional layer and a C3 convolutional layer; wherein:

[0104] The C1 convolutional layer receives the feature information map F obtained by the Resnet convolutional block of the backbone network as input, and performs downsampling through convolution operation and SiLU nonlinear activation function activation operation, wherein the convolution kernel size is 1x1, the convolution step is 1, and the output channel is 1 / 2 of the input channel;

[0105] The C2 convolutional layer receives the processed feature information of the C1 convolutional layer, and performs convolution operation and SiLU nonlinear activation function activation operation for downsampling, wherein the convolution kernel size is 3x3, the convolution step is 1, and the output channel size is 1;

[0106] The C3 convolutional layer receives the processed feature information of the C2 convolutional layer, and performs convolution operation and SiLU nonlinear activation function activation operation for downsampling, wherein the convolution kernel size is 3x3, the convolution step is 1, and the output channel size is 1;

[0107] The feature information map F obtained by the Resnet convolutional block of the backbone network is processed through the spatial attention network composed of C1-C3 three convolutional layers, and the spatial attention A s is obtained.

[0108]

[0109] Wherein, the feature information of each channel c of the feature information map F is multiplied pixel by pixel (i, j) with the corresponding pixel (i, j) of the spatial attention A s ​

[0110] Specifically, in order to improve the learning ability of spatial attention and the nonlinear expression ability, the spatial attention network is composed of three convolutional layers, denoted as: C1 convolutional layer, C2 convolutional layer and C3 convolutional layer, as shown in the following formula: Figure 5 .

[0111] The C1 convolutional layer receives the feature information map F obtained by the Resnet convolutional block as input, and performs downsampling through convolution operation and SiLU nonlinear activation function activation operation, wherein the convolution kernel size is 1x1, the convolution step is 1, and the output channel is half of the input feature information map.

[0112] The C2 convolutional layer receives the feature information processed by the C1 convolutional layer for convolution operation and SiLU nonlinear activation function activation operation for downsampling, wherein the convolution kernel size is 3x3, the convolution step is 1, and the output channel size is 1.

[0113] The C3 convolutional layer receives the feature information processed by the C2 convolutional layer for convolution operation and SiLU nonlinear activation function activation operation for downsampling, wherein the convolution kernel size is 3x3, the convolution step is 1, and the output channel size is 1.

[0114] The feature information obtained by the convolutional block is finally obtained after passing through the spatial attention network composed of the above three convolutional layers. s The feature information F' after feature reorganization by spatial attention can be expressed as:

[0115]

[0116] The meaning is that the feature information of each channel c of the feature information map F is multiplied pixel by pixel (i, j) with the spatial attention A s The corresponding pixel (i, j).

[0117] According to the method of the first aspect, the channel attention network is composed of one convolutional sum layer CS and four fully connected layers F1-F4; wherein:

[0118] The CS convolutional sum layer takes the feature information map F and the spatial attention A s as input, and performs multiplication and summation between corresponding pixel points (i, j) to obtain A cs , which is expressed as:

[0119]

[0120] The fully connected layer F1 and the fully connected layer F2 gradually reduce the channel number of A cs by 1 / 4 and 1 / 2;

[0121] The full connection layer F3 and the full connection layer F4 gradually increase the number of channels based on the output of the full connection layer F2 to obtain channel attention A c ;

[0122] The feature information F" after the channel attention network is:

[0123]

[0124] The feature information of each channel of the feature information F' after the spatial attention network is multiplied by the attention weight of the corresponding channel to obtain F". c

[0125] Specifically, in order to improve the learning ability and nonlinear expression ability of channel attention, the channel attention network is composed of one convolution sum layer and four full connection layers, denoted as: CS convolution sum layer, F1 full connection layer, F2 full connection layer, F3 full connection layer and F4 full connection layer, as shown in Figure 6 .

[0126] The CS convolution sum layer takes the feature information F and the spatial attention A s as input, and performs multiplication and summation between corresponding pixel points (i, j) to obtain A cs , which can be expressed as:

[0127]

[0128] The F1 full connection layer and the F2 full connection layer gradually reduce the number of channels of A cs by one fourth and one half, and the design concept is to discard unimportant channels, reduce their influence on the whole, and improve the robustness of the algorithm.

[0129] The F3 full connection layer and the F4 full connection layer gradually increase the number of channels of the output of the F2 full connection layer to obtain channel attention A c , and the parameters represent the importance of each channel learned by the network.

[0130] The final feature information F" after the visible region enhancement network is reorganized, and the calculation process is:

[0131]

[0132] The meaning is that the feature information of each channel of the feature map F' is multiplied by the attention weight of the corresponding channel of the channel attention A c to obtain F".

[0133] ​Subsequently, the feature information F" is input to a detection network module, and a network layer in the detection network determines the confidence of the target in the prediction frame according to the learned parameters, and outputs the target with high confidence as prediction information. The prediction information is input to a post-processing module, so that only one prediction result is retained for each target, and other redundant results are removed. The prediction result after post-processing is input to an output module, and the output module outputs the final detection result after labeling.

[0134] The second aspect of the present application discloses a target detection system under occlusion conditions; the processing unit of the system is configured to sequentially perform:

[0135] Step S1, calling an input module to collect image data, and pre-processing the image data including cropping operation and magnification operation, taking the image data after the pre-processing as a target detection image, wherein the input module is an optical instrument device;

[0136] Step S2, calling a feature extraction module based on a convolutional neural network structure to extract image feature information from the target detection image, the feature extraction module including a backbone network and a visible region enhancement network;

[0137] Step S3, calling a detection network module to determine prediction information based on the image feature information, wherein the network layer of the detection network module determines the target confidence in the prediction frame according to the learned parameters, and the target with a confidence higher than a threshold is the prediction information;

[0138] Step S4, calling the post-processing module to filter out one optimal prediction result for each target from the prediction information, and deleting other redundant prediction results;

[0139] Step S5, calling an output module to label and output the one optimal prediction result for each target.

[0140] According to the system of the second aspect, the size of the target detection image is CxHxW, wherein HxW is the image size, and C is the number of image channels; the backbone network based on Resnet increases a visible region enhancement network after each convolution block when extracting features from the target detection image, the visible region enhancement network is composed of a spatial attention network and a channel attention network; the feature information map F extracted by the convolution block is input to the visible region enhancement network, the visible region enhancement network sequentially calls the spatial attention network to calculate the spatial attention A s and calls the channel attention network to calculate the channel attention A c based on the feature information map; the reorganized feature information map F" is composed of the feature information map F, the spatial attention As and the channel attention A c After the calculation, we obtain:

[0141] F' = F x A s

[0142]

[0143] F" = F' x A c

[0144] wherein F' represents the feature information after the spatial attention network.

[0145] According to the system of the second aspect, the spatial attention network is composed of a C1 convolution layer, a C2 convolution layer and a C3 convolution layer; wherein:

[0146] The C1 convolution layer receives the feature information map F obtained by the Resnet convolution block of the backbone network as input, and performs downsampling through convolution operation and SiLU nonlinear activation function activation operation, wherein the convolution kernel size is 1x1, the convolution step is 1, and the output channel is 1 / 2 of the input channel.

[0147] The C2 convolution layer receives the processed feature information of the C1 convolution layer, and performs downsampling through convolution operation and SiLU nonlinear activation function activation operation, wherein the convolution kernel size is 3x3, the convolution step is 1, and the output channel size is 1.

[0148] The C3 convolution layer receives the processed feature information of the C2 convolution layer, and performs downsampling through convolution operation and SiLU nonlinear activation function activation operation, wherein the convolution kernel size is 3x3, the convolution step is 1, and the output channel size is 1.

[0149] After the feature information map F obtained by the Resnet convolution block of the backbone network passes through the spatial attention network composed of three convolution layers C1-C3, the spatial attention A is obtained s The feature information F' after the spatial attention network is represented as:

[0150]

[0151] wherein the feature information of each channel c of the feature information map F is multiplied pixel by pixel (i, j) with the spatial attention A s corresponding pixel (i, j).

[0152] According to the system of the second aspect, the channel attention network is composed of a convolution sum layer CS and four fully connected layers F1-F4; wherein:

[0153] The CS convolutional summation layer uses the feature information map F and the spatial attention A. s Using the input as input, perform multiplication and summation of corresponding pixel points (i, j) to obtain A. cs , represented as:

[0154]

[0155] The fully connected layers F1 and F2 will A cs The number of channels gradually decreases in the proportions of 1 / 4 and 1 / 2;

[0156] The fully connected layers F3 and F4 gradually increase the number of channels based on the output of the fully connected layer F2 to obtain channel attention A. c ;

[0157] The feature information F″ after passing through the channel attention network is:

[0158]

[0159] Among them, the feature information of each channel of the feature information F′ after passing through the spatial attention network is related to the channel attention A. c The attention weights of the corresponding channels are multiplied together to obtain F″.

[0160] A third aspect of the present invention discloses an electronic device. The electronic device includes a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, it implements the steps of the target detection method under occlusion conditions as described in any one of the first aspects of the present disclosure.

[0161] Figure 7 This is a structural diagram of an electronic device according to an embodiment of the present invention, such as... Figure 7 As shown, the electronic device includes a processor, memory, communication interface, display screen, and input device connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, carrier networks, Near Field Communication (NFC), or other technologies. The display screen can be an LCD screen or an e-ink screen. The input device can be a touch layer covering the display screen, buttons, a trackball, or a touchpad mounted on the device's casing, or an external keyboard, touchpad, or mouse.

[0162] Those skilled in the art can understand that, Figure 7 The structure shown in the figure is only a structure diagram of part of the technical solution of the present disclosure, and does not constitute a limitation on the electronic device to which the scheme of the present application is applied.

[0163] The fourth aspect of the present application discloses a computer readable storage medium. The computer readable storage medium stores a computer program, and the computer program is executed by a processor to implement the steps of the target detection method under the occlusion condition in any one of the first aspect of the present application.

[0164] In summary, the technical solution of the present application has wide coverage and strong scalability, and can be easily integrated into existing target detection algorithm framework. After pre-processing, input a picture or video into the target detection framework. This method learns the occlusion mode of the input image, and differentiates the feature information from the spatial dimension and the channel dimension according to the occlusion condition. It is worth noting that this method considers the correlation between spatial attention and channel attention, and guides the channel attention to decompose the feature information after matching the occlusion mode through spatial attention. Through testing on the occlusion dataset, the present application has achieved ideal results.

[0165] The present application designs an algorithm for improving the recognition accuracy of AI vision algorithm under occlusion condition. The key point of the algorithm lies in the designed visible region enhancement network. According to the actual occlusion scene, the visible region enhancement network is used to adaptively optimize the feature extraction process, and the reorganized features can reduce the interference caused by occlusion, which is more conducive to target recognition. The visible region enhancement network proposed by the present application is composed of spatial attention and channel attention. Unlike existing attention mechanisms, the double attention of the present application is redesigned to enhance the correlation between the two, which can better improve the target detection accuracy under occlusion condition.

[0166] Please note that the technical features of the above embodiments can be combined in any way, and in order to make the description simple, not all possible combinations of the technical features in the above embodiments are described, but as long as the combination of the technical features does not exist contradictory, it should be considered as the scope of the present application. The above-described embodiments only express several embodiments of the present application, and the description is more specific and detailed, but it should not be construed as limiting the scope of the patent. It should be noted that for those skilled in the art, without departing from the concept of the present application, some modifications and improvements can be made, which are all within the scope of the present application. Therefore, the protection scope of the present application patent should be subject to the appended claims.

Claims

1. A method for target detection in occlusion conditions, characterized in that, The method comprises: Step S1, calling an input module to collect image data, and performing preprocessing including cropping and magnifying on the image data, taking the image data after the preprocessing as a target detection image, wherein the input module is an optical instrument device; Step S2, calling a feature extraction module based on a convolutional neural network structure to extract image feature information from the target detection image, the feature extraction module comprising a backbone network and a visible region enhancement network; Step S3, calling a detection network module to determine prediction information based on the image feature information, wherein the network layer of the detection network module determines the target confidence in the prediction box according to the learned parameters, and the target whose target confidence is higher than a threshold is the prediction information; Step S4, calling a post-processing module to filter out an optimal prediction result of each target from the prediction information, and delete other redundant prediction results; Step S5, calling an output module to label and output the optimal prediction result of each target; In the method: The size of the target detection image is CxHxW, wherein HxW is the image size, and C is the number of image channels; The backbone network based on Resnet increases a visible region enhancement network after each convolution block when extracting features from the target detection image, and the visible region enhancement network is composed of a spatial attention network and a channel attention network; The feature information graph F extracted by the convolution block is taken as an input of the visible region enhancement network, and the visible region enhancement network sequentially calls the spatial attention network to calculate the spatial attention A based on the feature information graph s , and calls the channel attention network to calculate the channel attention A c ; The recomposed feature information map F'' is obtained by calculating the feature information map F, the spatial attention A s and the channel attention A c After the calculation, the following is obtained: F' = F x A s F" = F' x A c Wherein, F' represents the feature information after the spatial attention network.

2. The target detection method in an occlusion condition according to claim 1, wherein, The spatial attention network is composed of a C1 convolution layer, a C2 convolution layer and a C3 convolution layer; wherein: The C1 convolution layer receives the feature information map F obtained by the Resnet convolution block of the backbone network as input, and performs downsampling through convolution operation and SiLU nonlinear activation function activation operation, wherein the convolution kernel size is 1x1, the convolution step is 1, and the output channel is 1 / 2 of the input channel; The C2 convolution layer receives the processed feature information of the C1 convolution layer, and performs downsampling through convolution operation and SiLU nonlinear activation function activation operation, wherein the convolution kernel size is 3x3, the convolution step is 1, and the output channel size is 1; The C3 convolution layer receives the processed feature information of the C2 convolution layer, and performs downsampling through convolution operation and SiLU nonlinear activation function activation operation, wherein the convolution kernel size is 3x3, the convolution step is 1, and the output channel size is 1; The feature information map F obtained by the Resnet convolution block of the backbone network is input into the spatial attention network composed of C1-C3 convolution layers, and the spatial attention A is obtained s The feature information F' after the spatial attention network is represented as wherein the feature information of each channel c of the feature information map F is multiplied pixel-wise (i,j) with the spatial attention A s corresponding pixel (i,j).

3. The target detection method in an occlusion condition according to claim 2, characterized in that, The channel attention network is composed of a convolution sum layer CS and four fully connected layers F1-F4; wherein: The convolution sum layer CS takes the feature information map F e Rm x n and the spatial attention A e Rm x n as input, and performs multiplication sum between corresponding pixel points (i, j) to obtain A s cs , which is represented as:​ The fully connected layer F1 and the fully connected layer F2 reduce the number of channels of A cs by a ratio of 1 / 4 and 1 / 2, respectively; The full connection layer F3 and the full connection layer F4 gradually increase the number of channels based on the output of the full connection layer F2, to obtain a channel attention A c ; The feature information F'' after the channel attention network is: Wherein, the feature information of each channel of the feature information F' after the spatial attention network is multiplied by the attention weight of the corresponding channel to obtain F''. c The attention weight of the corresponding channel is multiplied to obtain F''.

4. A target detection system under occlusion conditions, characterized in that The processing unit of the system is configured to sequentially perform: Step S1, calling an input module to collect image data, and performing preprocessing including cropping and magnifying on the image data, taking the image data after the preprocessing as a target detection image, wherein the input module is an optical instrument device; Step S2, calling a feature extraction module based on a convolutional neural network structure to extract image feature information from the target detection image, the feature extraction module including a backbone network and a visible region enhancement network; Step S3, calling a detection network module to determine prediction information based on the image feature information, wherein the network layer of the detection network module determines the target confidence in the prediction box according to the learned parameters, and the target whose target confidence is higher than the threshold is the prediction information; Step S4, calling a post-processing module to filter out one optimal prediction result of each target from the prediction information and delete other redundant prediction results; Step S5, calling an output module to label and output one optimal prediction result of each target; Wherein: The size of the target detection image is C×H×W, wherein H×W is the image size and C is the image channel number; The backbone network based on Resnet increases a visible region enhancement network after each convolution block when extracting features from the target detection image, and the visible region enhancement network is composed of a spatial attention network and a channel attention network. The feature information graph F extracted by the convolution block is taken as an input of the visible region enhancement network, and the visible region enhancement network sequentially calls the spatial attention network to calculate the spatial attention A based on the feature information graph s , and calls the channel attention network to calculate the channel attention A c ; The recomposed feature information map F'' is obtained by computing the feature information map F, the spatial attention A s and the channel attention A c after the calculation: F' = F x A s F" = F' x A c Wherein, F' represents the feature information after the spatial attention network.

5. The target detection system in occluded conditions according to claim 4, characterized in that, The spatial attention network is composed of a C1 convolution layer, a C2 convolution layer and a C3 convolution layer; wherein: The C1 convolution layer receives the feature information map F obtained by the Resnet convolution block of the backbone network as input, and performs downsampling through convolution operation and SiLU nonlinear activation function activation operation, wherein the convolution kernel size is 1×1, the convolution step is 1, and the output channel is 1 / 2 of the input channel; The C2 convolution layer receives the processed feature information of the C1 convolution layer, performs convolution operation and SiLU nonlinear activation function activation operation for downsampling, wherein the convolution kernel size is 3×3, the convolution step is 1, and the output channel size is 1; The C3 convolution layer receives the processed feature information of the C2 convolution layer, performs convolution operation and SiLU nonlinear activation function activation operation for downsampling, wherein the convolution kernel size is 3×3, the convolution step is 1, and the output channel size is 1; The feature information map F obtained by the Resnet convolution block of the backbone network is input into the spatial attention network composed of C1-C3 convolution layers, and the spatial attention A is obtained s The feature information F' after the spatial attention network is represented as wherein the feature information of each channel c of the feature information map F is multiplied pixel-wise (i,j) with the spatial attention A s corresponding pixel (i,j).

6. The target detection system in occluded conditions according to claim 5, characterized in that, The channel attention network is composed of a convolution sum layer CS and four fully connected layers F1-F4; wherein: The convolution sum layer CS takes the feature information map F and the spatial attention A as inputs, and performs multiplication and summation between corresponding pixel points (i, j) to obtain A s , which is expressed as: cs , which is expressed as: The fully connected layer F1 and the fully connected layer F2 reduce the number of channels of A cs by a ratio of 1 / 4 and 1 / 2, respectively; The full connection layer F3 and the full connection layer F4 gradually increase the number of channels based on the output of the full connection layer F2, to obtain a channel attention A c ; The feature information F'' after the channel attention network is: Wherein, the feature information of each channel of the feature information F' after the spatial attention network is multiplied by the attention weight of the corresponding channel to obtain F''. c The attention weight of the corresponding channel is multiplied to obtain F''.

7. An electronic device, comprising: The electronic device includes a memory and a processor, the memory stores a computer program, and the processor executes the computer program to realize the steps of the target detection method under the occlusion condition in any one of claims 1-3.

8. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a computer program, and the computer program is executed by the processor to realize the steps of the target detection method under the occlusion condition in any one of claims 1-3.

Images