A shadow detection method and device based on attention to shadow boundary and feature correction

Abstract

The application discloses a shadow detection method and device based on attention shadow boundary and feature correction, and belongs to the technical field of deep learning and image shadow detection. The method comprises the following steps: acquiring a data set, dividing the data set into a training set and a test set; constructing a shadow detection neural network model, inputting the training set to train the shadow detection neural network model, and outputting the trained shadow detection neural network model; inputting the test set into the trained shadow detection neural network model, outputting a shadow prediction result, and perceiving and learning missing detection features and mis-detection features existing in a shadow area through a feature correction module arranged in the shadow detection neural network model, so that the occurrence of shadow mis-detection and missing detection is greatly reduced, and the shadow detection performance is further improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of deep learning and image shadow detection, and particularly relates to a shadow detection method and device based on attention to shadow boundary and feature correction. BACKGROUND

[0002] In the real world shadow situation, due to the complex background texture, there are often rapid intensity and color transitions near the shadow boundary, and the shadow is of great significance to scene understanding, optical flow tracking, light source position estimation, etc. Shadows can also interfere with video detection, target recognition and positioning tasks, such as blurring object edges, reducing target recognition accuracy, and causing some safety hazards. Therefore, shadow detection is an important visual task.

[0003] The state-of-the-art shadow detection algorithms improve the performance of shadow detection in several important aspects. For example, the DSC (Direction-aware Spatial Context Features for Shadow Detection) method aggregates spatial context features for shadow detection by extracting image context from different directions. The BDRAR (Bidirectional Feature Pyramid Network with Recurrent Attention Residual Modules for Shadow Detection) method refines context features by combining global and local context at deep and shallow layers, respectively, and extracts shadow features in a bidirectional convolutional neural network. Meanwhile, the MTMT (A Multi-task MeanTeacher for Semi-supervised Shadow Detection) method proposes a semi-supervised shadow detection algorithm that mainly improves the performance of shadow detection by combining semi-supervised methods and supervised shadow very fine edges. The FSD (Fast Shadow Detection) method uses a lightweight convolutional neural network to greatly improve the speed of shadow detection. Recently, the FDRNet (Mitigating Intensity Bias in Shadow Detection via Feature Decomposition and Reweighting) method reweights some intensity-variant and some intensity-invariant shadow features to redistribute attention maps for shadow detection.

[0004] Although the state-of-the-art shadow detection algorithms have achieved impressive results, their performance on shadow boundaries is still limited. Figure 1 The detection results of three state-of-the-art shadow detection algorithms, FSD, BDRAR, and MTMT, and the error maps of the real shadow regions are qualitatively shown. It can be seen that larger errors are more likely to occur around the shadow boundaries compared to the interior regions of the shadows. Figure 2This paper quantitatively presents the commonly used Binary Error Rate (BER) metric for MTMT and BDRAR on three public baseline shadow detection datasets (lower BER indicates higher detection accuracy and better performance). The horizontal axis represents the pixel width of the shadow boundary extending along the gradient direction of each edge pixel in the shadow region. The vertical axis represents the BER metric obtained by detecting only the shadow boundary of the current pixel width. The sharp drop on the left side of the curve indicates that when detecting shadows, the more shadow pixels we extend into the shadow boundary (i.e., the larger the shadow boundary), the lower the shadow detection error rate. The calculated BER metric tends to stabilize until it extends into the inner shadow region. This shows that the detection accuracy of pixels on the shadow boundary is much lower than that of pixels within the shadow region.

[0005] Furthermore, a significant reason for the low shadow boundary accuracy of existing state-of-the-art shadow detection methods is that they typically use the true mask of the shadow region to equally supervise pixels at the shadow edge and those inside the shadow. However, color and intensity variations near the shadow boundary are more complex than those inside the shadow. Additionally, the number of pixels at the shadow edge is usually significantly less than that inside the shadow, resulting in an unbalanced pixel distribution despite equal supervision, making the supervision of shadow edge and shadow interior regions unfair. To ensure that shadow detection algorithms pay attention to these few pixels at the shadow edge, researchers often increase the penalty for supervising shadow edge pixels. Therefore, inaccurate prediction of complex shadow edges can significantly impact the overall performance of shadow detection algorithms. For example, the semi-supervised shadow detection algorithm MTMT explicitly uses edge supervision to reduce detection errors at shadow edges, but it also suffers from the problem of unbalanced supervision in the shadow region. Therefore, the performance of MTMT, which only focuses on shadow edges, is easily affected by the accuracy of shadow edge prediction. Figure 1 The error plot of the MTMT method in the fourth image of the first row clearly shows that it still has some errors in detecting shadow edges. Note that shadow edges and shadow boundaries are not the same concept. A shadow edge only represents the portion of high-frequency pixels in the image that are in shadow, while a shadow boundary represents the area between the shadow edge and the inward region; it is a semantic boundary of the shadow region.

[0006] Another problem is the common issue of false detections and missed detections of shadows in shadow detection algorithms. Even the most advanced shadow detection algorithms still incorrectly detect darker object surfaces as shadow areas, or miss lighter shadow areas, mistaking them for not being shadow areas. Current advanced DSC methods attempt to indirectly avoid this problem by extracting image context information from different directions, but their BER on baseline shadow datasets remains high, indicating poor detection performance. Similarly, BDRAR also faces the problem of false detections and missed detections of shadows.

[0007] In fact, the two problems are interrelated, when the shadow detection algorithm produces false detection and missed detection, the method for detecting the boundary of the shadow is naturally inaccurate. Therefore, the application designs a new shadow detection deep neural network (BSDN), which improves the shadow detection performance, especially the detection performance on the shadow boundary, by explicitly supervising the boundary area and the internal area of the shadow respectively. And in order to further improve the performance of shadow detection, the application designs a feature correction module (CM) in the network and applies it to the BSDN to remove the interference features in the shadow which easily lead to false detection and missed detection of the shadow. SUMMARY

[0008] The application aims to overcome the false detection and missed detection of the shadow detection algorithm in the prior art, which leads to inaccurate detection of the shadow boundary, and provides a shadow detection method based on attention to the shadow boundary and feature correction.

[0009] In order to achieve the above application purpose, the application provides the following technical scheme:

[0010] A shadow detection method based on attention to the shadow boundary and feature correction, comprising the following steps:

[0011] S1: obtaining a shadow data set, dividing the data set into a training set and a test set;

[0012] S2: constructing a shadow detection neural network model, inputting the training set to train the shadow detection neural network model, and outputting the trained shadow detection neural network model;

[0013] The shadow detection neural network model comprises a hybrid encoder, a shadow boundary decoder, a shadow internal area decoder and a feature interaction decoder, the front part of the hybrid encoder contains a convolution residual neural network, and the rear part contains a Transformer neural network, the convolution residual neural network and the Transformer neural network are used to improve the detection performance of the shadow, the shadow boundary decoder and the shadow internal area decoder learn the features of the shadow internal area and the shadow boundary area respectively, and the feature interaction decoder fuses the output results of the shadow boundary decoder and the shadow internal area decoder to obtain a complete shadow feature map of each scale, and removes the interference features through a feature correction module;

[0014] S3: inputting the test set into the trained shadow detection neural network model, and outputting a shadow detection result.

[0015] By adopting the technical scheme, the shadow boundary and the shadow internal region can be focused on, the features can be learned and detected, the feature correction module is arranged to perceive and learn the missed detection features and the false detection features existing in the shadow region, the occurrence of the shadow false detection and the shadow missed detection is greatly reduced, and the shadow detection performance is further improved.

[0016] As a preferred scheme of the present application, the step S1 comprises a shadow mask decoupling preprocessing of the data set: decoupling an original shadow mask in the image into a shadow internal mask and a shadow boundary mask.

[0017] As a preferred scheme of the present application, the decoupling of the original shadow mask into the shadow internal mask and the shadow boundary mask comprises: converting a value of a pixel in the shadow mask into a distance of the pixel to a nearest background pixel by distance transformation, regarding a shadow region in the shadow mask as foreground and regarding a non-shadow region as background, and a distance transformation calculation formula of each pixel is:

[0018]

[0019] D(p, q) is a Euclidean distance between p and q, foreground, background, distance transformed pixel value, obtained by mapping a minimum value and a maximum value of from the original shadow mask G to a linear normalization in a [0, 1] range, and the shadow boundary region mask is obtained by subtracting the shadow internal region mask G' from the original shadow mask G.

[0020] As a preferred scheme of the present application, the step S1 further comprises a shadow missed detection supervision and shadow false detection supervision mask preprocessing of the data set: first, a plurality of shadow detection methods are adopted to detect the image to obtain a real shadow mask and a prediction result corresponding to the image respectively, and then a difference between the prediction result and the real shadow mask is regarded as an error map.

[0021] An expression of the error map is: E = abs(M-G)

[0022] An expression of the shadow missed detection region is: FN = E⊙G;

[0023] An expression of the shadow false detection region is: FP = E⊙1-G;

[0024] Wherein, G is a real shadow mask, M is a prediction result, E is an error map, and is an element-wise multiplication.

[0025] As a preferred scheme of the present application, in the step S2, a structure of the hybrid encoder is:

[0026] In the first step, the input image is outputted through a convolution residual neural network to obtain four layers of feature maps with different scales;

[0027] In the second step, the four layers of feature maps with different scales are flattened into serialized two-dimensional vector blocks;

[0028] In the third step, the two-dimensional vector blocks are mapped to a D (768 by default) dimensional embedding space through a trainable linear projection;

[0029] In the fourth step, the vectors mapped to the D dimensional embedding space are adjusted into four-dimensional feature maps, and feature fusion and cascaded upsampling are performed with the feature maps outputted by the subsequent feature interaction decoder.

[0030] As a preferred scheme of the present application, in the step S2, the structure of the shadow internal region decoder is two upsampling layers, one convolution block layer, and one cascaded operation, first, the feature map outputted by the left side decoder at the previous scale is upsampled, and then the feature map generated by the interaction feature decoder is upsampled through a convolution block layer, and finally, the feature map on the left side and the corrected feature map on the bottom are cascaded to obtain the feature map outputted by the internal region decoder at the current scale.

[0031] As a preferred scheme of the present application, in the step S2, the structure of the shadow boundary decoder is two upsampling layers, one convolution block layer, one cascaded operation, one pixel multiplication operation, and one operation of calculating the difference map M, first, the feature map outputted by the left side decoder at the previous scale is upsampled, and then the feature map generated by the interaction feature decoder is upsampled through a convolution block layer, and finally, the difference map is calculated by the feature map on the left side and the feature map on the top, and then the pixel dot product of the difference map and the feature map on the left side is calculated, and finally, the feature map on the bottom is cascaded to obtain the feature map outputted by the boundary decoder at the current scale.

[0032] As a preferred scheme of the present application, in the step S2, the structure of the feature interaction decoder is two upsampling layers, two convolution block layers, one feature correction module, and one pixel summation operation, first, the boundary feature map on the bottom and the internal region feature map on the top are summed by pixels to obtain a global feature map at the current scale, and then the global feature map is corrected through the feature correction module, and finally, the corrected feature map is respectively generated into a corrected shadow boundary feature map and a corrected shadow internal region feature map through the upsampling layer and the convolution layer;

[0033] The feature correction module comprises a shadow miss detection feature extractor and a shadow false detection feature extractor, which are used to extract the shadow miss detection feature and the shadow false detection feature respectively, and each of the shadow miss detection extractor and the shadow false detection extractor is composed of a branch for context extraction, each branch comprising a convolution block C for local feature extraction i and a dilated convolution block D for context perception i The calculation formula of each branch is:

[0034]

[0035] Where f i represents the feature extracted from the i-th convolution branch, C(·) is a convolution block composed of a convolution layer containing a 3×3 convolution kernel, a batch normalization layer and an activation function layer, F is an input feature, and for each f i , the kernel size of C i (·) is k i ×k i , and the kernel size and dilation rate of D i (·) are 3×3 and r i respectively.

[0036] The calculation formula of the shadow miss detection feature f fp is:

[0037] f fp =(C(Cat(f1,f2,f3,f4)))

[0038] The calculation formula of the shadow false detection feature f fn is:

[0039] f fn =(C(Cat(f1,f2,f3,f4)))

[0040] In the feature correction module, by adding the shadow miss detection feature and removing the shadow false detection feature, the calculation formula of the corrected feature map F′ is:

[0041] F′=(1+attn(f fn ))*F-ReLU(attn(f fp )*F)

[0042] Where attn(·) is a combination of a 3×3 convolution layer and a Sigmoid(·) function.

[0043] As a preferred scheme of the present application, in the step S2, the loss function of the shadow detection neural network model is:

[0044]

[0045] wherein, λ bound , λ body and λ f are mask-related weighting parameters in the shadow boundary branch, the shadow inner branch and the fusion branch, respectively;

[0046] In the branch of the shadow boundary decoder and the branch of the shadow inner region decoder, the loss function and is composed of and respectively. The BCE loss of the mask predicted by the branch of the shadow boundary decoder and the branch of the shadow inner decoder at 3 scales is represented as

[0047]

[0048]

[0049] The loss function in the feature interaction decoder is:

[0050]

[0051] wherein, α fn and α fp are weighting parameters of the loss function in the feature interaction decoder, is the BCE loss of the fusion mask at the i th th scale, is the BCE loss of the FP prediction of the fusion mask at the i th th scale.

[0052] In another aspect, a shadow detection device based on attention to shadow boundary and feature correction is disclosed, comprising at least one processor, and a memory connected in communication with the at least one processor; the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute any of the above-mentioned methods.

[0053] Compared with the prior art, the beneficial effects of the present application are that the shadow boundary and the shadow inner region can be focused, learned and detected, the feature correction module is set to perceive and learn the missed detection features and the false detection features of the shadow region, the occurrence of shadow false detection and missed detection is greatly reduced, and the shadow detection performance is further improved. BRIEF DESCRIPTION OF DRAWINGS

[0054] Figure 1A flowchart of a shadow detection method based on attention to shadow boundary and feature correction according to Embodiment 1 of the present application;

[0055] Figure 2 A structure diagram of a shadow detection neural network model of a shadow detection method based on attention to shadow boundary and feature correction according to Embodiments 1 and 2 of the present application;

[0056] Figure 3 A structure diagram of a hybrid encoder of a shadow detection method based on attention to shadow boundary and feature correction according to Embodiments 1 and 2 of the present application;

[0057] Figure 4 A single-layer structure diagram of a shadow internal region decoder of a shadow detection method based on attention to shadow boundary and feature correction according to Embodiments 1 and 2 of the present application;

[0058] Figure 5 A single-layer structure diagram of a shadow boundary decoder of a shadow detection method based on attention to shadow boundary and feature correction according to Embodiments 1 and 2 of the present application;

[0059] Figure 6 A feature interaction decoder structure diagram of a shadow detection method based on attention to shadow boundary and feature correction according to Embodiments 1 and 2 of the present application;

[0060] Figure 7 An error diagram of a prediction result and a real shadow region of a shadow detection algorithm of a shadow detection method based on attention to shadow boundary and feature correction according to Embodiment 2 of the present application;

[0061] Figure 8 A distance transformation schematic diagram of a shadow detection method based on attention to shadow boundary and feature correction according to Embodiment 2 of the present application;

[0062] Figure 9 A structure diagram of a feature correction module of a shadow detection method based on attention to shadow boundary and feature correction according to Embodiment 2 of the present application;

[0063] Figure 10 A prediction result of a shadow detection algorithm of a shadow detection method based on attention to shadow boundary and feature correction according to Embodiment 2 of the present application on a shadow boundary with different pixel widths;

[0064] Figure 11 An effective verification diagram of a feature correction module of a shadow detection method based on attention to shadow boundary and feature correction according to Embodiment 2 of the present application;

[0065] Figure 12Performance diagram of different detection methods of a shadow detection method based on attention to shadow boundary and feature correction described in embodiment 2 of the present application on shadow boundary of different widths;

[0066] Figure 13 Comparison diagram of shadow detection visual effects of BSDN designed for a shadow detection method based on attention to shadow boundary and feature correction described in embodiment 2 of the present application and other algorithms;

[0067] Figure 14 Effectiveness verification diagram of Transformer in a hybrid encoder of a shadow detection method based on attention to shadow boundary and feature correction described in embodiment 2 of the present application;

[0068] Figure 15 Performance comparison table of a shadow detection method based on attention to shadow boundary and feature correction described in embodiment 2 of the present application and the most advanced shadow detection algorithm;

[0069] Figure 16 Performance comparison table of a shadow detection method based on attention to shadow boundary and feature correction described in embodiment 2 of the present application and the most advanced related field detection algorithm;

[0070] Figure 17 Ablation study table of network design of a shadow detection method based on attention to shadow boundary and feature correction described in embodiment 2 of the present application;

[0071] Figure 18 Loss function ablation study table of a shadow detection method based on attention to shadow boundary and feature correction described in embodiment 2 of the present application;

[0072] Figure 19 Structural block diagram of a shadow detection device based on attention to shadow boundary and feature correction described in embodiment 3 of the present application. DETAILED DESCRIPTION

[0073] The present application will be further described in conjunction with test examples and specific embodiments. However, this should not be understood as limiting the scope of the above-mentioned subject matter of the present application to the following examples, and any technology realized based on the content of the present application falls within the scope of the present application.

[0074] Embodiment 1

[0075] A shadow detection method based on attention to shadow boundary and feature correction, as shown in Figure 1 , comprises the following steps:

[0076] S1: Obtain a shadow data set, and divide the data set into a training set and a test set;

[0077] S2: constructing a shadow detection neural network model, inputting the training set to train the shadow detection neural network model, and outputting a trained shadow detection neural network model;

[0078] As shown in the formula (1), the shadow detection neural network model comprises a hybrid encoder, a shadow boundary decoder, a shadow internal region decoder, and a feature interaction decoder, the hybrid encoder comprises a convolution residual neural network in a front part and a Transformer neural network in a rear part, the convolution residual neural network and the Transformer neural network are used to improve the detection performance of the shadow, the shadow boundary decoder and the shadow internal region decoder learn the features of the shadow internal region and the shadow boundary region respectively, and the feature interaction decoder fuses the output results of the shadow boundary decoder and the shadow internal region decoder to obtain a complete shadow feature map of each scale, and removes interference features through a feature correction module. Figure 2

[0079] S3: inputting the test set into the trained shadow detection neural network model, and outputting a shadow detection result.

[0080] The step S1 comprises a shadow mask decoupling preprocessing on the data set: decoupling an original shadow mask in the image into a shadow internal mask and a shadow boundary mask.

[0081] The decoupling of the original shadow mask into the shadow internal mask and the shadow boundary mask comprises: converting the value of a pixel in the shadow mask into the distance of the pixel to the nearest background pixel by using distance transformation, taking the shadow region in the shadow mask as foreground and taking the non-shadow region as background, and the distance transformation calculation formula of each pixel is:

[0082]

[0083] D(p, q) is the Euclidean distance between p and q, foreground, background, distance transformed pixel value, the decoupled shadow internal region soft mask is obtained by linear normalization from the minimum value and the maximum value of to the [0, 1] range, and the shadow boundary region soft mask is obtained by subtracting the shadow internal region soft mask G' from the original binary shadow mask G.

[0084] ​The step S1 further comprises pre-processing of the shadow leakage detection supervision and the shadow false detection supervision mask of the data set: first, a plurality of shadow detection methods are used to detect the image to obtain the real shadow mask and the prediction result corresponding to the image respectively, and then the difference between the prediction result and the real shadow mask is regarded as an error map;

[0085] The expression of the error map is: E = abs (M-G)

[0086] The expression of the shadow leakage detection area is: FN = E o G;

[0087] The expression of the shadow false detection area is: FP = E o 1-G;

[0088] Wherein, G is the real shadow mask, M is the prediction result, E is the error map, and o is the element multiplication.

[0089] In the step S1, the data set is divided into a training set and a test set according to a ratio of 8:2.

[0090] In the step S2, the structure of the mixed encoder is:

[0091] First, the input image is outputted through a convolution residual neural network to obtain four layers of feature maps with different scales;

[0092] Second, the four layers of feature maps with different sizes are flattened into a serialized two-dimensional vector block;

[0093] Third, the two-dimensional vector block is mapped to a D (default 768) dimensional embedding space using a trainable linear projection;

[0094] Fourth, the vector mapped to the D-dimensional embedding space is adjusted to a four-dimensional feature map, and participates in feature fusion and cascading upsampling with the feature map output by the subsequent feature interaction decoder.

[0095] In the step S2, as shown in the structure of the shadow internal area decoder: Figure 4 The structure of the shadow internal area decoder is: two upsampling layers, one convolution block layer, and one concatenation operation (concatenation), specifically, first, the feature map output by the left side of the last scale decoder is upsampled, and the corrected feature map generated by the interaction feature decoder below is upsampled, then the feature map is passed through a convolution block layer, and then the left feature map and the corrected feature map below are concatenated to obtain the feature map output by the internal area decoder at the current scale.

[0096] In the step S2, as shown in the structure of the shadow internal area decoder: Figure 5As shown, the structure of the shadow boundary decoder is: two up-sampling layers, a convolution block layer, a concatenation operation, a pixel-wise multiplication operation, and an operation for calculating the difference map M. Specifically, first, the feature map output by the left side of the previous scale decoder is up-sampled, and the corrected feature map generated by the interaction feature decoder above is up-sampled. After the up-sampling, the up-sampled feature map is input into a convolution block layer. Then, the difference map is calculated by subtracting the feature map above from the feature map below. After that, the pixel-wise product of the difference map and the feature map below is calculated. Finally, the feature map above is concatenated with the result to obtain the feature map output by the boundary decoder at the current scale.

[0097] In the step S2, as shown in Figure 6 As shown, the structure of the feature interaction decoder is: two up-sampling layers, two convolution block layers, a feature correction module (CM), and a pixel-wise addition operation. Specifically, first, the boundary feature map below and the internal region feature map above are added pixel by pixel to obtain the global feature map at the current scale. Then, the global feature map is input into the feature correction module (CM) for feature correction. After that, the corrected feature map is input into the up-sampling layer and the convolution layer to generate the corrected shadow boundary feature map (i.e., the feature map below) and the shadow internal region feature map (i.e., the feature map above).

[0098] The feature correction module includes a shadow missing detection feature extractor and a shadow false detection feature extractor, which are used to extract the shadow missing detection feature and the shadow false detection feature, respectively. The shadow missing detection extractor and the shadow false detection extractor are both composed of branches for context extraction. Each branch includes a convolution block C i and a dilated convolution block D i for context perception. The calculation formula of each branch is:

[0099]

[0100] where f i represents the feature extracted from the i-th convolution branch, C(·) is a convolution block composed of a convolution layer containing a 3×3 convolution kernel, a batch normalization layer, and an activation function layer, F is the input feature, and for each f i , the kernel size of C i (·) is k i ×k i , and the kernel size and the dilation rate of D i (·) are 3×3 and r i , respectively.

[0101] The shadow missing detection feature f fpThe calculation formula is:

[0102] f fp = (C(Cat(f1, f2, f3, f4)))

[0103] The calculation formula of the shadow mis-detection feature f fn is:

[0104] f fn = (C(Cat(f1, f2, f3, f4)))

[0105] In the feature correction module, by adding the shadow mis-detection feature and removing the shadow mis-detection feature, the calculation formula of the corrected feature map F' is:

[0106] F' = (1 + attn(f fn ))*F-ReLU(attn(f fp )*F)

[0107] Wherein, attn(·) is a combination of a 3x3 size convolution layer and a Sigmoid(·) function.

[0108] In the step S3, the loss function of the shadow detection neural network model is:

[0109]

[0110] Wherein, λ bound , λ body and λ f are respectively the mask-related weighting parameters in the shadow boundary branch, the shadow internal branch and the fusion branch;

[0111] In the branch of the shadow boundary decoder and the branch of the shadow internal area decoder, the loss functions and are composed of and respectively. The BCE loss of the mask predicted by the branch of the shadow boundary decoder and the branch of the shadow internal decoder at 3 scales is respectively represented as:

[0112]

[0113]

[0114] The loss function in the feature interaction decoder is:

[0115]

[0116] Wherein, α fn and α fprespectively are the weighted parameters of the loss function in the feature interaction decoder, is the BCE loss of the fusion mask at the i-th scale, is the BCE loss of the FP prediction of the fusion mask at the i-th scale.

[0117] With the above technical solution, the shadow boundary and the shadow internal region can be focused on, learned features and detected, the feature correction module is set to perceive and learn the missed detection features and the misdetected features existing in the shadow region, the occurrence of shadow misdetection and missed detection is greatly reduced, and the shadow detection performance is further improved.

[0118] Embodiment 2

[0119] This embodiment is a specific embodiment of embodiment 1:

[0120] Original shadow mask decoupling preprocessing.

[0121] The prediction difficulty of a pixel is closely related to its position. Due to the existence of complex background in the image, the pixels near the shadow edge are more easily mispredicted. In contrast, due to the consistency of the internal region of the shadow region, the shadow center pixels have higher prediction accuracy. It is more reasonable to process these shadow pixels according to their respective characteristics than to treat them equally.

[0122] Therefore, the original shadow mask is decoupled into a shadow internal mask and a shadow boundary mask. To achieve this goal, we introduce a traditional image processing algorithm, distance transform (DT), to decouple the shadow mask, as shown in Figure 8 DT can convert the value of a pixel in a binary image (shadow mask) to the distance from itself to the nearest background pixel. By taking the shadow region in the binary shadow mask as the foreground and the other region as the background, the distance transform calculation of each pixel can be described as:

[0123]

[0124] D(p, q) is the Euclidean distance between p and q, is the foreground, is the background, is the pixel value after distance transform, the decoupled shadow internal region soft mask is obtained by linear normalization from the minimum and maximum values of to the [0, 1] range, and the shadow boundary region soft mask is obtained by subtracting the shadow internal region soft mask G' from the original binary shadow mask G.

[0125] Generate mask preprocessing for shadow missed detection and misdetection supervision.

[0126] In order to make the shadow miss detection and false detection features predicted by the feature correction module (CM) designed in the application accurate, it is necessary to generate a shadow mask in advance for supervising the interference features. The miss detection features can be regarded as shadow false negative (FN) features, the false detection features can be regarded as shadow false positive (FP) features, and the difference between the shadow prediction result M generated by other best shadow detection algorithm and the real shadow mask G can be regarded as an error map (E), as shown in Figure 7 E = abs (M - G), so the generated shadow FP feature mask can be represented as: FP = E 1-G, and the generated shadow FN feature mask can be represented as: FN = E 1-G, wherein represents element-wise multiplication, and the shadow FN and FP shadow areas predicted by the subsequent feature correction module (CM) will be supervised by the generated shadow FN and FP feature masks.

[0127] The application designs a neural network module for shadow feature correction to solve the shadow miss detection and false detection problems often occurring in complex scenes in other existing best shadow detection algorithms. As shown in Figure 10 The effectiveness of the feature correction module designed in the application in shadow detection is qualitatively analyzed

[0128] The training set, validation set and test set of the divided benchmark shadow dataset are divided.

[0129] In fact, the three published benchmark shadow datasets (SBU, ISTD, and CUHK) provide training sets and validation sets, and the smallest published benchmark shadow dataset UCF only contains 110 images, but only the CUHK shadow dataset provides a test set, so the training of the shadow detection neural network method designed in the application is consistent with other best shadow detection algorithms: the validation set of the SBU and ISTD datasets is also used as the test set, and after training on the SBU dataset, the UCF dataset is tested to evaluate the shadow detection performance of the application on the UCF dataset.

[0130] The overall structure of the shadow detection neural network model designed in the application is constructed.

[0131] As shown in Figure 2As shown, the overall structure of the shadow detection neural network model (BSDN) designed by the present application is shown, which follows the multi-scale decoder style and includes a hybrid encoder and three decoders, namely a shadow boundary decoder, a shadow body decoder and an interaction decoder. The hybrid encoder uses the local feature extraction capability of the convolution residual neural network (ResNet) and the self-attention mechanism of the Transformer to improve the detection performance, the first two decoders learn the features of the shadow internal region and the shadow boundary region respectively, and the interaction decoder fuses the complete shadow feature map of each scale obtained by the first two decoders and removes the interference features through a feature correction module (CM).

[0132] The hybrid encoder in the network model is constructed.

[0133] As Figure 3 shown, the front part of the hybrid encoder (Hybrid Encoder) contains a convolution residual neural network (ResNet), and the rear part contains a Transformer neural network, which is the first shadow detection algorithm to introduce the Transformer, and from Figure 3 it can be seen that the output of the ResNet is mainly divided into four layers of feature maps of different scales. Let the size of the current input image be (H, W and C correspond to the length, width and channel number of the image, C is 3, that is, R, G and B three channels), then the feature map size of the first layer output is The feature map size of the second layer output is The feature map size of the third and fourth feature layers of the ResNet is 1 / 2 of the previous layer, and the channel number is twice that of the previous layer. Therefore, the feature map size of the fourth layer of the ResNet output is

[0134] Since the input of the Transformer requires a sequence rather than a feature map, the fourth layer of the feature map needs to be reshaped before being sent to the Transformer, including:

[0135] Image feature serialization. As shown, first, the input feature map is flattened into a serialized two-dimensional vector block x p , and where the size of each block is P x P, is the number of blocks, that is, the total length of the input sequence.

[0136] patch embeddings. Each vectorized patch is mapped to a latent dimensional embedding space using a trainable linear projection. To encode the spatial information of the patch, specific positional embeddings are learned that are added to the patch embeddings to preserve positional information, as shown in (1):

[0137]

[0138] where is the embedding projection of the patch, denotes the positional embedding. The Transformer encoder consists of layers of multi-head self-attention (MSA) and multi-layer perceptron (MLP). Thus the output of the i-th layer can be written as follows:

[0139]

[0140] where LN(·) denotes the layer normalization operator, is the encoded image representation. Thus, the output of the hybrid encoder includes both the feature maps extracted at multiple scales by ResNet and the sequence of features output by the Transformer.

[0141] The shadow interior region decoder in the network model is constructed.

[0142] The shadow interior region decoder (Body Decoder) is the inverse process of the hybrid encoder (Hybrid Encoder), and the single-layer decoding structure is shown in Figure 4 Each layer of the decoder fuses feature maps of different scales to perceive small-scale semantic information and large-scale fine-grained information, and the specific process is shown in equation (4):

[0143] F i = C(Cat(F i , Up(F i+1 ), Up(F′ i+1 ))) (4)

[0144] where C(·) denotes a convolution block consisting of a convolution layer (Convolution Layer), LN(·), and a ReLU(·) activation function, Up() denotes that the height and width of the feature map are upsampled by one time through the bilinear interpolation method, F i denotes the i-th layer feature map, F′ i+1 denotes the i+1-th layer feature map after feature correction, and F i+1 denotes the i+1-th layer feature map.

[0145] Constructing a shadow boundary decoder in the network model.

[0146] The overall structure of the shadow boundary decoder (Boundary Decoder) in the network is shown in Figure 5 It can be seen that it is basically consistent with the shadow internal region decoder (Body Decoder), but there are some differences in details. In order to focus on the boundary region, the shadow boundary decoder (Boundary Decoder) calculates the difference between the shadow boundary feature map and the boundary correction feature map at each scale Where γ represents a learnable weight parameter, and Up() represents an up-sampling operation. By finding the weight parameter γ that minimizes The corrected shadow boundary feature F′ i of the shadow boundary decoder extracts the shadow boundary feature F i is as consistent as possible, that is, the difference between the feature maps is reduced. The shadow boundary region feature predicted by each layer of the shadow boundary decoder is more accurate, thereby optimizing the shadow boundary feature extraction capability of the decoder. And this step is performed at each level in the decoder, achieving cross-scale optimization.

[0147] Designing a feature correction module in the network model.

[0148] The method of decoupling the shadow mask into the shadow boundary region and the shadow internal region and focusing on the features of the respective regions still has the problems of shadow false detection and missed detection, and the missed detection and false detection of the shadow will affect the feature extraction of the shadow boundary and the shadow internal region by the neural network. Therefore, the present application designs a feature correction module (CM), which learns and removes the false positive (False Positive) and false negative (False Negative) features existing in the image by analyzing the image space context, to remove the shadow false detection and shadow missed detection phenomenon respectively.

[0149] The detailed information of the feature correction module (CM) designed by the present application is shown in Figure 9 The feature correction module has two flows (FN flow and FP flow) for extracting FN features and FP features of the shadow. Both flows include a feature extractor (Feature Extractor). The feature extractor consists of four branches for context extraction. Each branch includes a convolution block C i for local feature extraction, and a dilated convolution block D i for context awareness. The calculation of each branch can be described as:

[0150]

[0151] where f i represents the feature extracted from the i-th convolution branch, C(·) and step 106 are consistent, and F is the input feature. For each f i , C i (·) has a kernel size of k i × k i , and D i (·) has a kernel size and dilation rate of 3×3 and r i , respectively. In the FP stream, the FP feature f fp is calculated as:

[0152] f fp = (C(Cat(f1, f2, f3, f4))) (6)

[0153] where, in step 106, has the same definition, and in the FN stream, the FN feature f fn is calculated in the same way.

[0154] In the feature correction module, by adding the FN feature and removing the FP feature, the corrected feature map F' is calculated as follows:

[0155] F' = (1 + attn(f fn ))*F - ReLU(attn(f fp )*F) (7)

[0156] where attn(·) is a 3×3 convolution layer combined with a Sigmoid(·) function.

[0157] To further verify the effectiveness of the feature correction module (CM), Figure 11 as shown, the visual comparison is shown, Figure 11 (c) clearly shows that the mountain in the right frame is incorrectly detected as a shadow, while the white road in the left frame is missed, in contrast, in Figure 11 (d), the shadow areas in the image are successfully detected.

[0158] Constructing a shadow feature interaction decoder in the network model.

[0159] The structure of the feature interaction decoder (Interaction Decoder) is shown in Figure 6 , combined with the shadow internal area decoder (Body Decoder) and the shadow boundary decoder (Boundary Decoder), the feature interaction decoder can be divided into three stages.

[0160] Feature fusion and correction stage: At each scale i, the feature maps extracted by the Body Decoder and Boundary Decoder are summed to obtain the complete feature map. Then, x is fed into the feature correction module (CM) designed in this invention. Through the extraction of shadow FN and FP features by CM, shadow interference features are further removed to obtain the corrected feature map F′. i .

[0161] Feature interaction and facilitation phase: The corrected feature map F′ obtained in the previous phase i Passing through F′ respectively i-1 =Up(C(F′) i Generate the feature map F′ of the previous scale. i-1 Furthermore, the feature-corrected shadow boundary region features and shadow interior features are introduced into the shadow boundary decoder and shadow interior decoder at each scale, respectively, thereby promoting the two decoders to extract accurate shadow features at each stage.

[0162] Multi-scale cascade stage. To combine high-level semantics and low-level fine-grained information, the feature sequence output by the Transformer in the Hybrid Encoder is... The data is reconstructed into feature maps, and then cascaded with the corrected feature maps at each scale through a series of upsamplers, as shown below. Figure 5 The purple arrows in the overall network structure are shown. Finally, the largest-scale shadow feature map is fed into the sigmoid activation function to generate the shadow detection results.

[0163] Design the loss function corresponding to this neural network model.

[0164] The overall loss function of the shadow detection neural network method (BSDN) designed in this invention It consists of three loss constraints, each corresponding to the shadow boundary decoder. Shadow Inside Decoder and interactive decoder To optimize the intermediate decoder results, each decoder is supervised in a multi-scale manner with binary cross-entropy (BCE) loss, as follows:

[0165]

[0166] Where, λ bound , λ body and λ f These are the mask-related weighted parameters in the shadow boundary branch, shadow interior branch, and blending branch, respectively.

[0167] In the branch of the shadow boundary decoder and the branch of the shadow interior region decoder, the loss function and Each by and The composition represents the BCE loss function of the mask predicted by the branch of the shadow boundary decoder and the branch of the shadow interior decoder at three scales, as shown below:

[0168]

[0169]

[0170] The loss function in the feature interaction decoder is:

[0171]

[0172] Where, α fn and α fp These are the weighting parameters of the loss function in the feature interaction decoder. The BCE loss of the fusion mask at the i-th scale, For the FP prediction of the fusion mask at the i-th scale, the BCE loss is... The representation is similar to

[0173] Configure the neural network implementation details.

[0174] Considering that the distance transformation calculation of shadow pixels during shadow decoupling is, in principle, a costly global operation, this invention follows the traditional approach, considering only the small neighborhood of shadow pixels, but still providing a reasonable approximation of the Euclidean distance. In the implementation of shadow decoupling, this invention uses the distance transformation function from the OpenCV 4.0 library, with the kernel size involved in the function set to 5.

[0175] In the Feature Extractor within the Feature Correction Module (CM), when i = {1, 2, 3, 4}, D i The dilation rate r of the dilated convolution in (·) i Set them to {1, 2, 3, 4} respectively, C i The kernel size k of the convolution in (·) i Set to k respectively i ={1, 3, 5, 7}. To achieve good performance, [the following will be used]. The weight parameter λ in bound , λ body and λ f Set them to 4, 1, and 1 respectively, and set α. fn and αfp are both 2.

[0176] A neural network training strategy is set.

[0177] The application uses ResNet-50 and Transformer to compose our Hybrid Encoder, and references R50-ViT-B_16, a backbone network model parameter pre-trained on ImageNet, for initialization. In our work, other newly introduced trainable parameters are based on random initialization.

[0178] The resolution of the input image is adjusted to 512*512, and the mini-batch size is 4. Data augmentation is performed by random horizontal flipping. Secondly, the application uses a stochastic gradient descent (SGD) optimizer with a momentum of 0.9 and a weight decay of 0.00005 to optimize the loss function. The initial learning rate is set to 0.0005, and the learning rate decay is 0.9.

[0179] The application also optimizes the entire network during encoding implementation. After about 150k iterations on a single RTX3090 graphics card, the shadow detection result is sent to a conditional random field (CRF) for post-processing optimization to obtain the final shadow detection result, following other best shadow detection algorithms.

[0180] The effectiveness of the deep network model structure and loss function designed by the application is verified.

[0181] The proposed shadow detection neural network method (BSDN) is tested and verified on four shadow detection datasets, SBU, UCF, ISTD, and CUHK. In terms of evaluation indicators, the application uses the balanced error rate (BER) widely used in the field of shadow detection to quantitatively evaluate the shadow detection result, and the calculation method of BER is as follows:

[0182]

[0183] where N tp , N tn , N p , and N n represent the number of true positives, true negatives, shadow pixels, and non-shadow pixels, respectively. It should be noted that the lower the BER value, the better the performance of shadow detection.

[0184] As Figure 17As shown, the ablation study results of network design are shown. The BER comparison between the first and second rows clearly shows the effectiveness of adding the Transformer. And in the third row, we compare our Hybrid Encoder with the backbone network ResNeXt commonly used in shadow detection algorithms, and it can be seen that the performance of the Hybrid Encoder is better than ResNeXt. From Figure 17 It can be seen that, thanks to the self-attention mechanism of the Transformer, the Hybrid Encoder can better extract shadow features from a global perspective, making the semantics of the shadow area consistent, such as Figure 14 As shown, the introduction of the Transformer in the Hybrid Encoder can make it easier for the neural network to pay attention to the location of the shadow area. Figure 17 The fourth to sixth rows in verify the importance of feature interaction, where the first two rows show that the interaction features between the internal decoder and the boundary decoder can improve the network performance whether they are large-scale or small-scale, and multiple interactions do not surprise, the scale features of the two decoders further improve the detection accuracy. The effect of the feature correction module (CM) is shown in the last row. Numerically, it greatly reduces the overall error of the detection by adding the missed detection area and removing the false detection area.

[0185] As shown in Figure 18 The baseline loss shown in the first row is commonly used by the most advanced shadow detection algorithms. It equivalently supervises the pixels inside the shadow and around the shadow boundary, and is affected by the number imbalance between the internal pixels and the edge pixels. The second and third rows show that whether simply supervising the shadow interior area or supervising the shadow boundary can effectively improve the performance of the network. Please note that explicitly supervising shadow boundary pixels can alleviate the supervision imbalance and lead to fine-grained prediction of the shadow boundary. Therefore, it achieves more BER improvement than only supervising shadow interior area pixels. Therefore, the supervision of shadow body prediction and shadow boundary prediction enables the network to learn more comprehensive shadow features, thereby achieving significant shadow detection performance, as shown in the fourth row of Figure 18 The last row verifies the effectiveness of considering these two disturbances in the CM module.

[0186] Obtain the shadow detection results, and compare and verify the shadow detection performance of the present application.

[0187] The BSDN shadow detection algorithm designed in the application is compared with 13 state-of-the-art shadow detection algorithms, including FDRNet, ECA (Robust Shadow Detection by Exploring Effective Shadow Contexts), MTMT, FSD, DSDNet (Distraction-aware Shadow Detection), DC-DSPF (Densely Cascaded Shadow Detection Network via Deeply Supervised Parallel Fusion), BDRAR, AD-Net (A+D Net: Training a Shadow Detector with Adversarial Shadow Attenuation), DSC, ST-CGAN (Stacked Conditional Generative Adversarial Networks for Jointly Learning Shadow Detection and Shadow Removal.), patched-CNN (Fast Shadow Detection From a Single Image Using a Patched Convolutional Neural Network), scGAN (Shadow Detection with Conditional Generative Adversarial Networks) and stacked-CNN (Large-scale Training of Shadow Detectors with Noisily-Annotated Shadow Examples). As shown in Table 1, the shadow detection visual effect comparison of the BSDN shadow detection algorithm and other state-of-the-art detection methods is listed. Figure 13 Figure 15 ​The table shows that the BSDN proposed in the present application has achieved a BER index reduction of 9.9%, 3.3%, and 5.2% respectively on the SBU, ISTD, and UCF data sets compared with the FDRNet, which is the best shadow detection algorithm at present, indicating the strong performance of the BSDN network designed in the present application. On the challenging CUHK data set, the BSDN method proposed in the present application achieves a BER index reduction of 3.7% compared with the most advanced method MTMT, and the BER index is reduced by 13.0%, 5.8%, and 14.5% on the SBU, UCF, and ISTD respectively.

[0188] In order to specially measure the detection accuracy of different methods on the shadow boundary, the present application tests the BER index of multiple detection algorithms corresponding to different shadow boundary widths on the SBU, ISTD, and CUHK baseline shadow data sets. As shown in Figure 12 The bottommost curve (BSDN) shows the highest shadow boundary detection accuracy among all the comparison methods, proving the effectiveness of the perceived shadow boundary proposed in the present application in detecting shadows.

[0189] In addition, the designed BSDN is also compared with the shadow removal method DeshadowNet (DeshadowNet: A multi-context embedding deep network for shadow removal), four state-of-the-art salient object detection methods (i.e. MINet (Multi-scale interactive network for salient object detection), EGNet (EGNet: Edge Guidance Network for Salient Object Detection.), LDF (Label Decoupling Framework for Salient Object Detection), SRM (Stagewise Refinement Model for Detecting Salient Objects in Images.)) and three semantic segmentation methods (i.e. Segmenter (Segmenter: Transformer for semantic segmentation), MiB (Modeling the background for incremental learning in semantic segmentation) and PSPNet (Pyramid Scene Parsing Network)) on four benchmark shadow datasets. The released codes are retrained and the best shadow detection results are selected after fine-tuning the training parameters. As shown in Figure 16 Figure 3, the BER index of the performance of these algorithms on different datasets is shown. It can be seen that the BSDN achieves the best shadow detection performance on the four benchmark datasets.

[0190] Embodiment 3

[0191] As shown in Figure 19 Figure 4, an electronic device includes at least one processor, and a memory connected in communication with the at least one processor; the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the method described in the foregoing embodiments. The input and output interface can include a display, a keyboard, a mouse, and a USB interface for inputting and outputting data; a power supply for providing power to the electronic device.

[0192] Those skilled in the art can understand that all or part of the steps of the above-mentioned method embodiments can be completed by program instruction related hardware, and the foregoing program can be stored in a computer readable storage medium, and the program executes the steps of the above-mentioned method embodiments when executed; and the foregoing storage medium includes a mobile storage device, a read only memory (ROM), a magnetic disc or an optical disc and various storage medium capable of storing program codes.

[0193] When the integrated unit of the present application is realized in the form of a software function unit and sold or used as an independent product, it can also be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the embodiments of the present application can be embodied in the form of a software product, and the computer software product is stored in a storage medium, includes a plurality of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute all or part of the method described in the embodiments of the present application. The foregoing storage medium includes a mobile storage device, a ROM, a magnetic disc or an optical disc and various storage medium capable of storing program codes.

[0194] The above only describes the preferred embodiments of the present application and is not intended to limit the present application. Any modification, equivalent replacement and improvement made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A shadow detection method based on attention to shadow boundary and feature correction, characterized in that, The method comprises the following steps: S1: obtaining a shadow data set, dividing the data set into a training set and a test set; S2: constructing a shadow detection neural network model, inputting the training set to train the shadow detection neural network model, and outputting a trained shadow detection neural network model; The shadow detection neural network model comprises a hybrid encoder, a shadow boundary decoder, a shadow internal region decoder and a feature interaction decoder, the front part of the hybrid encoder comprises a convolution residual neural network, and the rear part comprises a Transformer neural network, the convolution residual neural network and the Transformer neural network are used to improve the detection performance of the shadow, the shadow boundary decoder and the shadow internal region decoder learn the features of the shadow internal region and the shadow boundary region respectively, and the feature interaction decoder fuses the output results of the shadow boundary decoder and the shadow internal region decoder to obtain a complete shadow feature map of each scale, and removes interference features through a feature correction module; S3: inputting the test set into the trained shadow detection neural network model to output a shadow detection result; In the step S1, the data set is preprocessed by shadow mask decoupling: the original shadow mask in the data set is decoupled into a shadow internal mask and a shadow boundary mask; The original shadow mask is decoupled into a shadow internal mask and a shadow boundary mask, which comprises: using distance transformation to convert the value of each pixel in the shadow mask into the distance of the pixel to the nearest background pixel, regarding the shadow region in the shadow mask as the foreground and the non-shadow region as the background, and the distance transformation calculation formula of each pixel is: is and the Euclidean distance between is foreground, is background, is the pixel value after distance transform, decoupled shadow inner region soft mask is obtained by linear normalization from the minimum and maximum value of to the range of [0, 1], the shadow boundary region mask is obtained by subtracting the shadow inner region mask from the original shadow mask.

2. The shadow detection method based on attention to shadow boundary and feature correction according to claim 1, characterized in that, In the step S1, the data set is preprocessed by shadow leakage detection supervision and shadow mis-detection supervision mask: first, a plurality of shadow detection methods are used to detect the image to obtain the corresponding real shadow mask and prediction result, and then the difference between the prediction result and the real shadow mask is regarded as an error graph; The expression of the error map is: ; The expression of the shadow leak detection region is: ; The expression of the shadow false detection region is: ; where G is the ground truth shadow mask, M is the prediction result, and E is the error map, is the element-wise multiplication. 3.The shadow detection method based on attention shadow boundary and feature correction of claim 1, wherein, In the step S2, the structure of the hybrid encoder is: Firstly, the input image is outputted through the convolution residual neural network to obtain four layers of feature maps of different scales; Secondly, the four layers of feature maps of different sizes are flattened into a serialized two-dimensional vector block; Thirdly, the two-dimensional vector block is mapped to a D-dimensional embedding space through a trainable linear projection; Fourthly, the vector mapped to the D-dimensional embedding space is adjusted into a four-dimensional feature map, and participates in feature fusion and cascaded upsampling with the feature map outputted by the subsequent feature interaction decoder.

4. The shadow detection method based on attention to shadow boundary and feature correction of claim 1, wherein, In the step S2, the structure of the shadow internal region decoder is: two upsampling layers, one convolution block layer and one cascading operation, firstly, the feature map outputted by the left side of the previous scale decoder is upsampled, and the corrected feature map generated by the interaction feature decoder is upsampled, then the feature map is outputted through a convolution block layer, and then the left feature map and the corrected feature map are cascaded to obtain the feature map outputted by the internal region decoder at the current scale.

5. The shadow detection method based on attention to shadow boundary and feature correction according to claim 1, characterized in that, In the step S2, the structure of the shadow boundary decoder is: two up-sampling layers, one convolution block layer, one cascading operation, one pixel multiplication operation, and one operation of calculating the difference map M. First, the feature map output by the left side of the previous scale decoder is up-sampled, and the corrected feature map generated by the interactive feature decoder above is up-sampled. Then, the up-sampled feature map is input into the convolution block layer. Then, the difference map is calculated by the feature map on the left side and the feature map above. Then, the pixel point product of the difference map and the feature map on the left side is calculated, and then the feature map below is cascaded to obtain the feature map output by the boundary decoder at the current scale.

6. The shadow detection method based on attention to shadow boundary and feature correction of claim 1, wherein, In the step S2, the structure of the feature interaction decoder is: two up-sampling layers, two convolution block layers, one feature correction module, and one pixel summation operation. First, the boundary feature map below and the internal region feature map above are summed by pixels to obtain the global feature map at the current scale. Then, the global feature map is input into the feature correction module for feature correction. Then, the corrected feature map is input into the up-sampling layer and the convolution layer to generate the corrected shadow boundary feature map and the shadow internal region feature map. The feature correction module comprises a shadow miss detection feature extractor and a shadow false detection feature extractor, which are respectively used for extracting a shadow miss detection feature and a shadow false detection feature, and the shadow miss detection extractor and the shadow false detection extractor are both composed of branches for context extraction, and each branch comprises a convolution block for local feature extraction and a dilated convolution block for context perception , and the calculation formula of each branch is: wherein represents the features extracted from the first convolution branch, is a convolution block composed of a convolution layer with a 3x3 kernel, a batch normalization layer, and an activation function layer, is an input feature, and for each , the kernel size of , the kernel size and the expansion rate of are 3x3 and 3x3 and 3, respectively; The shadow leak detection feature The calculation formula is: The shadow mis-detection feature The calculation formula is: In the feature correction module, the shadow leakage detection feature is added and the shadow error detection feature, corrected feature map The calculation formula is: wherein is a combination of a 3x3 size convolutional layer with the function.

7. The shadow detection method based on attention to shadow boundary and feature correction of claim 1, wherein, In the step S2, the loss function of the shadow detection neural network model is: wherein, , and are mask-dependent weighting parameters in the shadow boundary branch, the shadow interior branch, and the fusion branch, respectively. In the branch of the shadow boundary decoder and the branch of the shadow interior region decoder, the loss function and consists of and respectively, representing the BCE loss function of the masks predicted by the branch of the shadow boundary decoder and the branch of the shadow interior decoder at 3 scales, respectively as follows: In the feature interaction decoder, the loss function is: wherein, and are weighting parameters for the loss function in the feature interaction decoder, respectively, is the BCE loss of the fusion mask at the i-th dimension, is the BCE loss of the FP prediction of the fusion mask at the i-th dimension.

8. A shadow detection device based on attention to shadow boundary and feature correction, characterized by, The device comprises at least one processor and a memory connected in communication with the at least one processor; the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the method in any one of claims 1 to 7.

Images