Camera occlusion detection method and device, storage medium and electronic device

Abstract

The application discloses a camera shielding detection method and device, a storage medium and electronic equipment. The method comprises the following steps: obtaining an initial camera shielding detection model, wherein the initial camera shielding detection model at least comprises a feature extraction network, a mixed spatial pyramid network and a mixed up-sampling network, the output end of the feature extraction network is connected with the input end of the mixed spatial pyramid network and the input end of the mixed up-sampling network; inputting target image data into the initial camera shielding detection model for training to obtain a trained camera shielding detection model; and under the condition that a target loss function value meets a loss function threshold, taking the trained camera shielding detection model as a target camera shielding detection model. The application solves the technical problem that the camera shielding detection method in the related art has low recognition efficiency, poor accuracy and single output result, and cannot meet the real-time video analysis requirement.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence and intelligent video analysis, and more specifically, to a method, apparatus, storage medium, and electronic device for detecting camera occlusion. Background Technology

[0002] In the field of video surveillance, data relies on video streams transmitted by cameras. If a camera is obstructed, the collected data becomes meaningless for monitoring. Traditionally, edge detection and feature extraction methods using machine learning are used to identify camera occlusion. However, this approach is only effective in scenarios where the camera is completely or extensively obstructed, and is ineffective against partial obstruction or darkness rendering the camera "invisible." While neural networks have been used to identify camera occlusion in industry, relying solely on neural networks for occlusion detection suffers from low accuracy and limited capabilities, failing to meet the demands of real-time video analysis. Deep neural networks contain rich positional and semantic information in their high-level features, but current technologies often fail to effectively utilize these discriminative features, resulting in low model prediction accuracy.

[0003] There is currently no effective solution to the above problems. Summary of the Invention

[0004] This invention provides a camera occlusion detection method, apparatus, storage medium, and electronic device to at least solve the technical problems of low recognition efficiency, poor accuracy, and single output results in related camera occlusion detection methods, which cannot meet the needs of real-time video analysis.

[0005] According to one aspect of the present invention, a camera occlusion detection method is provided, comprising: acquiring a pre-constructed initial camera occlusion detection model, wherein the initial camera occlusion detection model includes at least a feature extraction network, a hybrid spatial pyramid network, and a hybrid upsampling network, wherein the output of the feature extraction network is connected to the input of the hybrid spatial pyramid network, and the output of the feature extraction network is connected to the input of the hybrid upsampling network; inputting target image data into the initial camera occlusion detection model for training to obtain a trained camera occlusion detection model; and, if the target loss function value corresponding to the trained camera occlusion detection model satisfies a loss function threshold, using the trained camera occlusion detection model as a target camera occlusion detection model, wherein the target loss function value is determined based on a first loss function value corresponding to the hybrid spatial pyramid network and a second loss function value corresponding to the hybrid upsampling network.

[0006] According to another aspect of the present invention, a camera occlusion detection device is also provided, comprising: an acquisition module, configured to acquire a pre-constructed initial camera occlusion detection model, wherein the initial camera occlusion detection model includes at least a feature extraction network, a hybrid spatial pyramid network, and a hybrid upsampling network, wherein the output end of the feature extraction network is connected to the input end of the hybrid spatial pyramid network, and the output end of the feature extraction network is connected to the input end of the hybrid upsampling network; a training module, configured to input target image data into the initial camera occlusion detection model for training, thereby obtaining a trained camera occlusion detection model; and a determination module, configured to, when the target loss function value corresponding to the trained camera occlusion detection model satisfies a loss function threshold, use the trained camera occlusion detection model as a target camera occlusion detection model, wherein the target loss function value is determined based on a first loss function value corresponding to the hybrid spatial pyramid network and a second loss function value corresponding to the hybrid upsampling network.

[0007] According to another aspect of the present invention, a non-volatile storage medium is also provided, wherein the non-volatile storage medium stores a plurality of instructions, the instructions being adapted to be loaded by a processor and executed any one of the above-described camera occlusion detection methods.

[0008] According to another aspect of the present invention, an electronic device is also provided, including one or more processors and a memory, wherein the memory is used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement any one of the above-described camera occlusion detection methods.

[0009] In this embodiment of the invention, a pre-constructed initial camera occlusion detection model is obtained. This initial model includes at least a feature extraction network, a hybrid spatial pyramid network, and a hybrid upsampling network. The output of the feature extraction network is connected to the input of the hybrid spatial pyramid network, and the output of the feature extraction network is connected to the input of the hybrid upsampling network. Target image data is input into the initial camera occlusion detection model for training, resulting in a trained camera occlusion detection model. If the target loss function value corresponding to the trained camera occlusion detection model satisfies the loss function threshold, the trained camera occlusion detection model is used as the target camera occlusion detection model. The target loss function value is determined based on the first loss function value corresponding to the hybrid spatial pyramid network and the second loss function value corresponding to the hybrid upsampling network. This achieves the goal of constructing a camera occlusion detection model with higher accuracy and better performance, thereby improving the accuracy and efficiency of camera occlusion detection. This solves the technical problems of low recognition efficiency, poor accuracy, and single output results in related camera occlusion detection methods, which cannot meet the needs of real-time video analysis. Attached Figure Description

[0010] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, illustrate exemplary embodiments of the invention and, together with their description, serve to explain the invention and do not constitute an undue limitation thereof. In the drawings:

[0011] Figure 1 This is a schematic diagram of a camera occlusion detection method according to an embodiment of the present invention;

[0012] Figure 2 This is an optional flowchart for generating a hybrid spatial pyramid network according to an embodiment of the present invention;

[0013] Figure 3 This is a flowchart of an optional spatial hidden layer feature network generation method according to an embodiment of the present invention;

[0014] Figure 4 This is a schematic diagram of the model structure of an optional initial camera occlusion detection model according to an embodiment of the present invention;

[0015] Figure 5 This is a schematic diagram of the occlusion state and occlusion ratio output of an optional image to be identified according to an embodiment of the present invention;

[0016] Figure 6 This is a schematic diagram of a camera occlusion detection device according to an embodiment of the present invention. Detailed Implementation

[0017] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0018] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0019] In the field of video surveillance, data relies on video streams transmitted by cameras. If a camera is obstructed, the collected data becomes meaningless for monitoring. Traditionally, edge detection and feature extraction methods using machine learning are used to identify camera occlusion. However, this approach is only effective in scenarios where the camera is completely or extensively obstructed, and is ineffective against partially obstructed or darkened cameras. While neural networks have been used to identify camera occlusion in industry, relying solely on neural networks for occlusion detection suffers from low accuracy and limited capabilities, failing to meet the demands of real-time video analysis. Deep neural networks contain rich positional and semantic information in their high-level features, but current technologies often fail to effectively utilize these discriminative features, resulting in low model prediction accuracy. The advantages and disadvantages of existing camera occlusion detection systems include:

[0020] (1) Advantages of using traditional methods to identify camera occlusion: By extracting image texture features through a manually designed feature extractor and combining it with an edge detection algorithm, large-area or extremely obvious camera occlusion can be detected quickly. Disadvantages of using traditional methods to identify camera occlusion: The model can only handle large-area or extremely obvious occlusion scenes, and is powerless for partially occluded or dark scenes, and the output results are singular; it relies on other auxiliary devices or manual judgment.

[0021] (2) Advantages of using neural networks to identify camera occlusion: The powerful feature extraction capabilities of neural networks can easily handle various occlusion scenarios, detecting camera occlusion in a fully intelligent manner. Disadvantages of using neural networks to identify camera occlusion: The model relies on a large amount of camera occlusion training data, resulting in low accuracy and limited capabilities. Existing neural networks cannot fully utilize mid-to-high-level semantic features and location information, failing to meet the needs of real-time video analysis.

[0022] To address the aforementioned issues, this invention provides a method embodiment for camera occlusion detection. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0023] Figure 1 This is a flowchart of a camera occlusion detection method according to an embodiment of the present invention, such as... Figure 1 As shown, the method includes the following steps:

[0024] Step S102: Obtain a pre-constructed initial camera occlusion detection model, wherein the initial camera occlusion detection model includes at least a feature extraction network, a hybrid spatial pyramid network, and a hybrid upsampling network. The output of the feature extraction network is connected to the input of the hybrid spatial pyramid network, and the output of the feature extraction network is connected to the input of the hybrid upsampling network.

[0025] Optionally, the aforementioned feature extraction network includes: a third number of inverse residual layers, a fourth number of channel attention layers, and a fifth number of spatial attention layers. The inverse residual layers include two convolutional layers, two batch normalization layers, two activation layers, a data warehouse layer, and a residual layer. The channel attention layers include two convolutional layers, an average pooling layer, a max pooling layer, and two fully connected layers. The spatial attention layer includes a convolutional layer and a sigmoid activation function. The aforementioned hybrid spatial pyramid network includes a spatial pyramid sampling network and a spatial hidden layer feature network. The spatial pyramid network includes a first sampling network and a second sampling network. The first sampling network includes two different convolutional layers and a pooling layer. The second sampling network includes an average pooling layer and two pooling layers. The aforementioned hybrid upsampling network includes a deconvolutional layer and two bilinear interpolation layers.

[0026] It can be understood that the initial camera occlusion detection model mentioned above, which includes a feature extraction network, a hybrid spatial pyramid network, and a hybrid upsampling network, can be understood as an end-to-end perceptual network model (denoted as the HybNet model). The HybNet model consists of three parts: a feature extraction module, a classification module, and a semantic segmentation module. The feature extraction module is composed of a feature extraction network, consisting of 15 inverse residual layers, 15 channel attention layers, and 15 spatial attention layers stacked together. Among them, the inverse residual block consists of two 1×1 convolutional layers, two batch normalization (BN) layers, two activation layers, one data warehouse (DW) convolutional layer, and one residual connection, with the connection method being 1×1 convolutional layer -> BN layer -> activation layer -> DW convolutional layer -> 1×1 convolutional layer -> BN layer -> activation layer -> residual connection; the channel attention layer consists of two convolutional layers, one average pooling layer, one max pooling layer, and two fully connected layers; the spatial attention layer consists of one convolutional layer and one sigmoid activation function. The classification module consists of a hybrid spatial pyramid network (HSP operator), which outputs the camera occlusion status (including occluded and unoccluded). The semantic segmentation module consists of a hybrid upsampling network (HUF operator), which learns the camera occlusion region and calculates the occlusion ratio according to the occlusion ratio formula. Both the HSP and HUF operators use SHF features (i.e., spatial weight parameters) to assign weights in the spatial dimension, balancing the output.

[0027] Step S104: Input the target image data into the initial camera occlusion detection model above for training to obtain the trained camera occlusion detection model.

[0028] Optionally, before model training, data preprocessing is required to obtain the target image data, including image smoothing, data augmentation, and data normalization. The specific steps are as follows: a) Image smoothing: Gaussian blur the original image data to obtain the first image data. b) Data augmentation: Augment the first image data to obtain the second image data. This includes rotating the first image data at 15° intervals between -90° and 90°, horizontally and vertically flipping the image to expand it, and using the expanded image data as the second image data. c) Data normalization: Standardize the second image data to obtain the target image data. This includes using the mean μ and standard deviation σ of the second image data, standardizing the image data using Z-Score, and using the standardized image data as the target image data.

[0029] In an optional embodiment, the above-mentioned inputting the target image dataset into the above-mentioned initial camera occlusion detection model for training to obtain a trained camera occlusion detection model includes:

[0030] Step S1041: Input the target image dataset into the feature extraction network to obtain the feature extraction result output by the feature extraction network and the trained feature extraction network.

[0031] Step S1042: Input the first number of feature maps from the above feature extraction results into the above hybrid spatial pyramid network to obtain the first occlusion state output by the above hybrid spatial pyramid network, and the trained hybrid spatial pyramid network.

[0032] Optionally, the first number of feature maps includes one or more of the second feature map output by the last network layer of the feature extraction network and the feature maps output by the intermediate network layers of the feature extraction network, wherein the first number of feature maps have different sizes. For example, the first number of feature maps includes feature maps of three sizes: [40*40, 20*20, 10*10], wherein the feature map with a size of 10*10 is the second feature map output by the last network layer of the feature extraction network.

[0033] In an optional embodiment, when the hybrid spatial pyramid network includes a spatial pyramid sampling network and a spatial hidden layer feature network, the step of inputting a first number of feature maps from the feature extraction results into the hybrid spatial pyramid network to obtain a first occlusion state output by the hybrid spatial pyramid network includes: inputting the first number of feature maps output by the feature extraction network into the spatial pyramid sampling network to obtain a fused feature map corresponding to the first number of feature maps; inputting the second feature map into the spatial hidden layer feature network to obtain spatial weight parameters corresponding to the pixel positions included in the second feature map; and determining the first occlusion state output by the hybrid spatial pyramid network based on the fused feature map and the spatial weight parameters.

[0034] By employing the above method, the feature maps of various sizes output by the feature extraction network (i.e., the first number of feature maps) are first fused to obtain a fused feature map that incorporates semantic information of different depths and breadths from the feature maps, achieving the effect of feature-semantic aggregation. Furthermore, spatial weight parameters obtained through spatial mapping are acquired via a spatial hidden layer feature network to dynamically adjust the values ​​at corresponding positions in each channel feature map. Based on this, the occlusion state of the image determined using the fused feature map and spatial weight parameters has higher accuracy.

[0035] In an optional embodiment, when the spatial pyramid network includes a first sampling network and a second sampling network, the step of inputting the first number of feature maps output by the feature extraction network into the spatial pyramid sampling network to obtain the fused feature map corresponding to the first number of feature maps includes: inputting the first number of feature maps into the first sampling network to obtain first output feature maps corresponding to the first number of feature maps, wherein the first sampling network includes two different convolutional layers and a pooling layer; inputting the first number of feature maps into the second sampling network to obtain second output feature maps corresponding to the first number of feature maps, wherein the second sampling network includes an average pooling layer and two pooling layers; concatenating the first output feature maps corresponding to the first number of feature maps and the second output feature maps corresponding to the first number of feature maps to obtain third output feature maps corresponding to the first number of feature maps; and fusing the third output feature maps corresponding to the first number of feature maps to obtain the fused feature map.

[0036] Optionally, but not limited to, a self-attention mechanism may be used to fuse the third output feature maps corresponding to the first number of feature maps to obtain the fused feature maps.

[0037] Optionally, taking the aforementioned first number of feature maps including three sizes of feature maps [40*40, 20*20, 10*10], with the 10*10 feature map being the second feature map output by the last network layer of the feature extraction network as an example, the spatial pyramid sampling network consists of two sampling combinations: a first sampling network [3*3 convolution, 5*5 convolution, max pooling], and a second sampling network [average pooling, max pooling, max pooling]. The three sizes of feature maps [40*40, 20*20, 10*10] are input into the first and second sampling networks of the spatial pyramid sampling network, respectively. Then, the output feature maps of the two sampling combinations are concatenated along the channel dimension (i.e., the first feature map output by the first sampling network and the second feature map output by the second sampling network are concatenated at the same size), resulting in three feature maps V1, V2, and V3 (i.e., the third feature map). For the three obtained feature maps V1, V2, and V3, a self-attention mechanism is used for fusion, and the fusion formula is as follows:

[0038]

[0039] Among them, F out This represents the output of the fused feature map, and softmax() represents the softmax activation function.

[0040] The above methods fully integrate semantic information from feature maps of different sizes, taking into account both local and global information. It should be noted that the two sampling combinations (i.e., the first and second sampling combinations) use different time lengths *s*, padding, and channel numbers. This approach effectively extracts semantic information from different depths and breadths in the feature maps, and then aggregates this information by concatenating the channels, achieving the effect of feature semantic aggregation.

[0041] Optionally, the spatial hidden layer features (SHF) are weight parameters learned in the spatial dimension for different pixel positions. The second feature map is input into the spatial hidden layer feature network to obtain the spatial weight parameters corresponding to the pixel positions included in the second feature map. The specific calculation formula within the spatial hidden layer feature network is as follows:

[0042] ① Local maximization:

[0043] ② Local meanization:

[0044] ③Lying-layer feature learning:

[0045] ④ Normalization:

[0046] Where F1 represents the maximum value of each pixel in the feature map output by the last layer of the feature extraction network (i.e., the second feature map), F2 represents the average value of each pixel in the feature map output by the last layer of the feature extraction network, F3 represents the concatenation result of F1 and F2 corresponding to the feature map output by the last layer of the feature extraction network, and cat(F1,F2) indicates concatenation of F1 and F2. in The feature map is the output of the last network layer of the feature extraction network (i.e., the second feature map, such as a feature map with a size of 10*10). max means taking the maximum value of the channel, avg means taking the average value of the channel, conv means 1×1 convolution, SHF is the calculated hidden layer feature (i.e., spatial weight parameter), C represents the channel dimension in the feature extraction network, H represents the length of the feature map in each channel, and W represents the width of the feature map in each channel.

[0047] It is understandable that the aforementioned spatial hidden layer feature network uses weight parameters obtained through spatial mapping, which can dynamically adjust the values ​​at corresponding positions in each channel feature map.

[0048] Optionally, based on the above-mentioned fused feature map and spatial weight parameters, the first occlusion state output by the above-mentioned hybrid spatial pyramid network is determined in the following manner:

[0049] F′ out =softmax(avg(F) out *SHF))

[0050] Among them, F ′ out It is the output of the HSP operator (i.e., the first occlusion state of the hybrid spatial pyramid network output), F out It is the output of the spatial pyramid network sampling (i.e., the fused feature map), and SHF is the calculated hidden layer feature (i.e., the spatial weight parameter).

[0051] Step S1043: Input the second number of feature maps from the above feature extraction results into the above hybrid upsampling network to obtain the first occlusion ratio output by the above hybrid upsampling network and the trained hybrid upsampling network.

[0052] Optionally, the second number of feature maps mentioned above are multiple feature maps output from the intermediate network layers of the feature extraction network, and the size of the second number of feature maps is different. Taking the second number of feature maps including the third feature map F1: 20*20, the fourth feature map F2: 40*40, and the fifth feature map F3: 80*80 as an example, the sizes of the third feature map, the fourth feature map, and the fifth feature map are in a predetermined multiple relationship (such as 2 times).

[0053] In an optional embodiment, where the hybrid upsampling network includes a deconvolution layer and two bilinear interpolation layers, and the second number of feature maps includes a third feature map, a fourth feature map, and a fifth feature map, and the sizes of the third feature map, the fourth feature map, and the fifth feature map are in a predetermined multiple relationship, the step of inputting the second number of feature maps from the feature extraction result into the hybrid upsampling network to obtain the first occlusion ratio output by the hybrid upsampling network includes: performing a first upsampling process on the third feature map using the two bilinear interpolation layers to obtain a third feature map after the first upsampling process; concatenating the third feature map after the first upsampling process with the fourth feature map to obtain a first concatenated feature map; and applying the spatial weight parameters to the upper... The pixel positions included in the first stitched feature map are weighted to obtain a second stitched feature map; the second stitched feature map is then upsampled using the two bilinear interpolation layers to obtain a second stitched feature map after the second upsampling; the second stitched feature map after the second upsampling is then stitched with the fifth feature map to obtain a third stitched feature map; the pixel positions included in the third stitched feature map are weighted based on the spatial weight parameters to obtain a fourth stitched feature map; the fourth stitched feature map is then input into a deconvolution layer for a third upsampling to obtain a sixth feature map, wherein the sixth feature map has the same size as the target image data; the sixth feature map is then semantically segmented to obtain the first occlusion ratio.

[0054] Optionally, taking the second number of feature maps as an example, including the third feature map F1: 20*20, the fourth feature map F2: 40*40, and the fifth feature map F3: 80*80, and assuming that the sizes of the third, fourth, and fifth feature maps are in a predetermined multiple relationship (e.g., 2 times), the hybrid upsampling network is applied to the segmentation task. It consists of one deconvolution layer and two bilinear interpolation layers, which upsample the second number of feature maps (e.g., the three sizes of feature maps [F1: 20*20, F2: 40*40, F3: 80*80] respectively). The specific process is as follows: The third feature map F1 is upsampled by 2x using a bilinear interpolation layer and then concatenated with the fourth feature map F2 along the channel dimension. Weights are then assigned using SHF features (i.e., spatial weight parameters). Next, it is upsampled by 2x using a bilinear interpolation layer and concatenated with the fifth feature map F3 along the channel dimension. Weights are then assigned again using SHF features. Finally, deconvolution is used to upsample the image by 8x to the original image size. Semantic segmentation is then performed on the image, and the proportion of the occluded region to the original image is calculated, yielding the first occlusion ratio output by the hybrid upsampling network. Verification shows that bilinear interpolation performs better than deconvolution at low upsampling rates, while deconvolution performs better at high upsampling rates. The formula for calculating the first occlusion ratio output by the hybrid upsampling network (i.e., the HUF operator) is as follows:

[0055] Bilear 8× {Deconv 2× [Deconv 2× [F1),conv(F2)],conv(F3)}

[0056] Wherein, conv refers to 1×1 convolution, Deconv refers to deconvolution, 2× or 8× refers to upsampling by 2 or 8 times, and Bilear refers to bilinear interpolation.

[0057] Formula for calculating occlusion ratio: Among them O ij The occluded pixel is denoted as 1, N is the width of the original image (corresponding to the target graphic data), i represents the column pixel position, and j represents the row pixel position.

[0058] Step S1044: Based on the above-trained feature extraction network, the above-trained hybrid spatial pyramid network, and the above-trained hybrid upsampling network, the above-trained camera occlusion detection model is obtained.

[0059] Optionally, after the model training is completed, the trained camera occlusion detection model can be obtained by combining the acquired trained feature extraction network, trained hybrid spatial pyramid network, and trained hybrid upsampling network.

[0060] Step S106: If the target loss function value corresponding to the trained camera occlusion detection model satisfies the loss function threshold, the trained camera occlusion detection model is used as the target camera occlusion detection model. The target loss function value is determined based on the first loss function value corresponding to the hybrid spatial pyramid network and the second loss function value corresponding to the hybrid upsampling network.

[0061] Optionally, during the training of the initial camera occlusion detection model, the stochastic gradient descent (SGD) optimization algorithm is used, with a minimum batch size of 64, an iteration cycle of 200 rounds, and a loss function of [missing information]. The learning rate decreased by a factor of 10 at each of the [40, 80, 120] rounds. The model outputs the camera occlusion status and the camera occlusion ratio. The output results are compared with the actual results, and the parameters of the model are iteratively updated using the SGD optimization algorithm to obtain the optimal trained camera occlusion detection model as the target camera occlusion detection model.

[0062] In an optional embodiment, before using the trained camera occlusion detection model as the target camera occlusion detection model when the target loss function value corresponding to the trained camera occlusion detection model satisfies the loss function threshold, the method further includes: determining the first loss function value based on the first occlusion state and the actual occlusion state; determining the second loss function value based on the first occlusion ratio and the actual occlusion ratio; and obtaining the target loss function value based on the first loss function value and the second loss function value.

[0063] It should be noted that this embodiment of the invention is a heterogeneous task learning method. The outputs of the classification task and the semantic segmentation task are two different dimensions and different semantic information. In order to train these two tasks simultaneously and make them complementary and balanced, a first loss function value is determined based on the first occlusion state output by the hybrid spatial pyramid network and the actual occlusion state as the loss of the classification task, and a second loss function value is determined based on the second occlusion state output by the hybrid upsampling network and the actual occlusion state as the loss of the segmentation task. During the training process, the loss of the classification task (corresponding to the hybrid spatial pyramid network) and the loss of the segmentation task (corresponding to the hybrid upsampling network) can be dynamically balanced.

[0064] In an optional embodiment, obtaining the target loss function value based on the first loss function value and the second loss function value includes:

[0065] Based on the first loss function value and the second loss function value mentioned above, the target loss function value is obtained as follows:

[0066]

[0067] Where Loss represents the target loss function value, λ represents the preset learnable parameters, L1 represents the first loss function value, L2 represents the second loss function value, and d k The above represents the output dimension of the hybrid upsampling network. The first loss function value is determined based on the cross-entropy loss function, and the second loss function value is determined based on the binary cross-entropy loss function.

[0068] It should be noted that, compared to the direct addition method of Loss = L1 + L2, the loss function calculation method proposed in this embodiment of the invention can dynamically adjust the loss of heterogeneous tasks, making it more flexible. Furthermore, taking log and d... k Parameters can narrow the loss gap between two different tasks, mutually enhancing each other on a unified dimension and interacting with each other.

[0069] In an optional embodiment, after the target loss function value corresponding to the trained camera occlusion detection model satisfies the loss function threshold, and the trained camera occlusion detection model is used as the target camera occlusion detection model, the method further includes: acquiring an image to be identified; inputting the image to be identified into the target camera occlusion detection model to obtain a second occlusion state and a second occlusion ratio corresponding to the image to be identified.

[0070] Optionally, the image to be identified may be, but is not limited to, one captured in real time by a camera.

[0071] After obtaining the target camera occlusion detection model using the above method, the image to be identified, captured in real-time by the camera, is input into the target camera occlusion detection model for testing. The test results include the occlusion state and occlusion ratio of the image to be identified. In other words, the target camera occlusion detection model can not only determine whether occlusion exists in the image to be identified, but also further identify the occlusion ratio, better meeting the needs of real-time video analysis.

[0072] Through the above steps S102 to S106, the goal of constructing a camera occlusion detection model with higher accuracy and better performance can be achieved, thereby improving the technical effect of camera occlusion recognition accuracy and efficiency. This solves the technical problems of low recognition efficiency, poor accuracy and single output results in related camera occlusion detection methods, which cannot meet the needs of real-time video analysis.

[0073] Based on the above embodiments and optional embodiments, the present invention proposes an optional implementation method, which includes the following steps:

[0074] Step S1: Construct an end-to-end perceptual network model (denoted as HybNet model) as the initial camera occlusion detection model. The specific steps are as follows:

[0075] Step S11, as follows Figure 2 As shown, the hybrid spatial pyramid network (denoted as the HSP crossover operator) is generated through the following steps:

[0076] Step S111: Construct a spatial pyramid sampling network. The spatial pyramid sampling network consists of two sampling combinations: a first sampling network [3*3 convolution, 5*5 convolution, max pooling] and a second sampling network [average pooling, max pooling, max pooling]. Feature maps of three sizes [40*40, 20*20, 10*10] are input into the first and second sampling networks of the spatial pyramid sampling network, respectively. Then, the output feature maps of the two sampling combinations are concatenated along the channel dimension (i.e., the first feature map output by the first sampling network and the second feature map output by the second sampling network are concatenated at the same size), resulting in three feature maps V1, V2, and V3 (i.e., the third feature map). The three feature maps V1, V2, and V3 are then fused using a self-attention mechanism to obtain a fused feature map.

[0077] The above methods fully integrate semantic information from feature maps of different sizes, taking into account both local and global information. It should be noted that the two sampling combinations (i.e., the first and second sampling combinations) use different time lengths *s*, padding, and channel numbers. This approach effectively extracts semantic information from different depths and breadths in the feature maps, and then aggregates this information by concatenating the channels, achieving the effect of feature semantic aggregation.

[0078] Step S112, as follows Figure 3 As shown, a spatial hidden layer feature network (SHF) is constructed. The spatial hidden layer features are weight parameters for different pixel positions learned in the spatial dimension. The second feature map is input into the spatial hidden layer feature network to obtain the spatial weight parameters corresponding to the pixel positions included in the second feature map. The spatial hidden layer feature network is a weight parameter obtained through spatial mapping, which can dynamically adjust the values ​​of the corresponding positions in each channel feature map.

[0079] Step S113: Calculation of the output results of the hybrid spatial pyramid network.

[0080] The formula for calculating the output of the Hybrid Spatial Pyramid Network (HSP operator) is as follows:

[0081] F ′ out =softmax(avg(F) out*SHF))

[0082] Among them, F ′ out It is the output of the HSP operator (i.e., the first occlusion state of the hybrid spatial pyramid network output), F out It is the output of the spatial pyramid network sampling (i.e., the fused feature map), and SHF is the calculated hidden layer feature (i.e., the spatial weight parameter).

[0083] Step S12: Generate a hybrid upsampling network (denoted as the HUF operator). The specific steps are as follows: The hybrid upsampling network operates on the segmentation task and consists of one deconvolution layer and two bilinear interpolation layers. It upsamples the second number of feature maps (such as feature maps of three sizes: [F1: 20*20, F2: 40*40, F3: 80*80]). Specifically, the third feature map F1 is upsampled by 2 times using the bilinear interpolation layer and then concatenated with the fourth feature map F2 in the channel dimension. Then, weights are assigned using SHF features (i.e., spatial weight parameters). The third feature map F1 is then upsampled by 2 times using the bilinear interpolation layer and concatenated with the fifth feature map F3 in the channel dimension. Weights are assigned again using SHF features. Finally, deconvolution is used to upsample the image by 8 times to the original image size. Semantic segmentation is then performed on the image, and the proportion of the occluded region to the original image is calculated, which is the first occlusion ratio output by the hybrid upsampling network. Bilinear interpolation can be more effective than deconvolution at low upsampling rates, while deconvolution is better at high upsampling rates.

[0084] Step S13: Build an end-to-end perception network model as the initial camera occlusion detection model. The model structure of the initial camera occlusion detection model is as follows: Figure 5As shown, the initial camera occlusion detection model consists of three parts: a feature extraction module, a classification module, and a semantic segmentation module. The feature extraction module is composed of a feature extraction network, which includes 15 inverse residual layers, 15 channel attention layers, and 15 spatial attention layers stacked together. Specifically, the inverse residual block consists of two 1×1 convolutional layers, two batch normalization (BN) layers, two activation layers, one data warehouse (DW) convolutional layer, and one residual connection, connected in the following order: 1×1 convolutional layer -> BN layer -> activation layer -> DW convolutional layer -> 1×1 convolutional layer -> BN layer -> activation layer -> residual connection. The channel attention layer consists of two convolutional layers, one average pooling layer, one max pooling layer, and two fully connected layers. The spatial attention layer consists of one convolutional layer and one sigmoid activation function. The classification module consists of a hybrid spatial pyramid network (HSP operator), which outputs the camera occlusion status (including occluded and unoccluded). The semantic segmentation module consists of a hybrid upsampling network (HUF operator), which learns the camera occlusion region and calculates the occlusion ratio according to the occlusion ratio formula. Both the HSP and HUF operators use SHF features (i.e., spatial weight parameters) to assign weights in the spatial dimension, balancing the output.

[0085] Step S2, model training, the specific steps are as follows:

[0086] Step S21: Data preprocessing, including image smoothing, data augmentation, and data normalization, specifically as follows: a) Image smoothing: Gaussian blurring is applied to the original image data to obtain the first image data. b) Data augmentation: Data augmentation is performed on the first image data to obtain the second image data. Specifically, this includes rotating the first image data at 15° intervals between -90° and 90°, horizontally flipping and vertically flipping the image to expand the image data, and using the expanded image data as the second image data. c) Data normalization: Standardization is performed on the second image data to obtain the target image data. Specifically, the mean μ and standard deviation σ corresponding to the second image data are used to standardize the image data using Z-Score, and the standardized image data is used as the target image data.

[0087] Step S22: Define the loss function. This embodiment of the invention is a heterogeneous task learning approach. The outputs of the classification task and the semantic segmentation task are information with two different dimensions and semantics. In order to train these two tasks simultaneously and to make them complementary and balanced, the loss of the classification task (corresponding to the hybrid spatial pyramid network) and the loss of the segmentation task (corresponding to the hybrid upsampling network) can be dynamically weighed during training. Therefore, the loss function is designed as follows:

[0088]

[0089] Where λ is a learnable parameter that can be dynamically adjusted during training; L1 refers to the first loss function value of the classification module (i.e., the hybrid spatial pyramid network), which is determined based on the cross-entropy loss function; L2 refers to the second loss function of the segmentation module (i.e., the hybrid upsampling network), which is determined based on the binary cross-entropy (BCE) loss function; and d k This refers to the dimension of the output of the hybrid upsampling network, which is used to reasonably scale L2.

[0090] Step S23: Training the model. During the training of the initial camera occlusion detection model, the SGD optimization algorithm is used, with a minimum batch size of 64, an iteration cycle of 200 rounds, and a loss function of... The learning rate decreased by a factor of 10 at each of the [40, 80, 120] rounds. The model outputs the camera occlusion status and the camera occlusion ratio. The output results are compared with the actual results, and the parameters of the model are iteratively updated using the SGD optimization algorithm to obtain the optimal trained camera occlusion detection model as the target camera occlusion detection model.

[0091] Step S3: After obtaining the target camera occlusion detection model, the image to be recognized, captured in real-time by the camera, is input into the target camera occlusion detection model for testing. The test results include the occlusion state and occlusion ratio of the image to be recognized. Figure 5 As shown, this illustrates the occlusion status and occlusion ratio information of the image to be identified, predicted by the target camera occlusion detection model. In other words, the target camera occlusion detection model can not only determine whether occlusion exists in the image to be identified, but also further identify the occlusion ratio within the image.

[0092] The embodiments of the present invention can achieve at least the following technical effects: (1) By constructing a hybrid feature fusion heterogeneous task learning deep neural network, the problem of camera occlusion detection is solved. The hybrid feature fusion in the model is achieved by a self-designed hybrid spatial pyramid network (HSP) and hybrid upsampling network (HUF). Heterogeneous task learning refers to the model's ability to simultaneously learn and predict camera occlusion and occlusion ratio. Among them, the HSP and HUF operators are used to fuse mid-to-high-level feature semantic information and location information in camera occlusion prediction and occlusion ratio prediction, respectively. The present invention designs a self-learning loss function for heterogeneous task learning. This function can be iteratively updated during backpropagation to balance the conflict between heterogeneous learning tasks. The model adopts a hybrid feature fusion and heterogeneous task learning architecture, making full use of mid-to-high-level feature semantic information and location information, balancing multi-task learning loss, and outputting more accurate and reliable results with faster inference speed, adapting to the needs of real-time video detection and analysis. (2) By employing deep learning, edge detection and other technologies, the following capabilities were achieved: real-time and accurate prediction of camera occlusion and occlusion ratio; full utilization of mid-to-high-level semantic and location information; mutual complementarity and balance of heterogeneous task learning; and significant reduction in system resource consumption and cost investment. (3) Innovative design of HSP and HUF operators: The HSP operator is applied to classification tasks, fully integrating the semantic and location information of mid-to-high-level features in the network, and using spatial hidden layer features (SHF) to reasonably allocate weights in the spatial dimension; The HUF operator is applied to semantic segmentation tasks, sampling mid-to-high-level feature maps at multiple scales, solving the problem of distortion of high-level semantic features during upsampling, and avoiding the problem of feature weight imbalance. (4) Innovative proposal of hybrid feature fusion and heterogeneous task learning network (HybNet): The end-to-end trainable neural network model HybNet is constructed, which uses HSP, HUF and SHF features to accurately analyze camera occlusion and camera occlusion ratio, adapting to various camera occlusion scenarios. (5) A self-learning loss function is designed to complement the classification and segmentation tasks, balance the losses of heterogeneous tasks, and improve the convergence speed and prediction accuracy of the model. (6) The inverse residual module and DW convolution are introduced to improve the network inference speed: the model uses the inverse residual module and DW convolution to reduce the number of parameters, improve the inference speed, and significantly reduce resource consumption and cost. (7) In the context of full-service competition in the video surveillance market, it can be applied to camera monitoring scenarios such as safe campuses, smart cities, and smart security. It can accurately and intelligently predict camera occlusion and the proportion of camera occlusion, and provide alarm information, which greatly reduces the cost of human monitoring and drives the economic development of the corresponding scenarios.

[0093] This embodiment also provides a camera occlusion detection device, which is used to implement the above embodiments and preferred embodiments; details already described will not be repeated. As used below, the terms "module" and "device" can refer to a combination of software and / or hardware that performs a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, hardware implementations, or a combination of software and hardware, are also possible and contemplated.

[0094] According to an embodiment of the present invention, an apparatus embodiment for implementing the above-described camera occlusion detection method is also provided. Figure 6 This is a schematic diagram of a camera occlusion detection device according to an embodiment of the present invention, as shown below. Figure 6 As shown, the aforementioned camera occlusion detection device includes: an acquisition module 600, a training module 602, and a determination module 604, wherein:

[0095] The aforementioned acquisition module 600 is used to acquire a pre-constructed initial camera occlusion detection model, wherein the initial camera occlusion detection model includes at least a feature extraction network, a hybrid spatial pyramid network, and a hybrid upsampling network, wherein the output of the feature extraction network is connected to the input of the hybrid spatial pyramid network, and the output of the feature extraction network is connected to the input of the hybrid upsampling network.

[0096] The training module 602 is connected to the acquisition module 600 and is used to input the target image data into the initial camera occlusion detection model for training, so as to obtain the trained camera occlusion detection model.

[0097] The aforementioned determining module 604, connected to the aforementioned training module 602, is used to take the aforementioned trained camera occlusion detection model as the target camera occlusion detection model when the target loss function value corresponding to the trained camera occlusion detection model satisfies the loss function threshold. The aforementioned target loss function value is determined based on the first loss function value corresponding to the aforementioned hybrid spatial pyramid network and the second loss function value corresponding to the aforementioned hybrid upsampling network.

[0098] In this embodiment of the invention, the acquisition module 600 is configured to acquire a pre-constructed initial camera occlusion detection model. This initial camera occlusion detection model includes at least a feature extraction network, a hybrid spatial pyramid network, and a hybrid upsampling network. The output of the feature extraction network is connected to the input of the hybrid spatial pyramid network, and the output of the feature extraction network is connected to the input of the hybrid upsampling network. The training module 602, connected to the acquisition module 600, is used to input target image data into the initial camera occlusion detection model for training, thereby obtaining a trained camera occlusion detection model. The determination module 604, connected to the training module 602, is used to determine the determined model after training. If the target loss function value of the camera occlusion detection model satisfies the loss function threshold, the trained camera occlusion detection model is used as the target camera occlusion detection model. The target loss function value is determined based on the first loss function value of the hybrid spatial pyramid network and the second loss function value of the hybrid upsampling network. This achieves the goal of constructing a camera occlusion detection model with higher accuracy and better performance, thereby improving the accuracy and efficiency of camera occlusion detection. It also solves the technical problems of low recognition efficiency, poor accuracy, and single output results in related camera occlusion detection methods, which cannot meet the needs of real-time video analysis.

[0099] It should be noted that the above modules can be implemented by software or hardware. For example, for the latter, it can be implemented in the following ways: the above modules can be located in the same processor; or the above modules can be located in different processors in any combination.

[0100] It should be noted that the acquisition module 600, training module 602, and determination module 604 mentioned above correspond to steps S102 to S106 in the embodiments. The instances and application scenarios implemented by the above modules and corresponding steps are the same, but are not limited to the content disclosed in the above embodiments. It should also be noted that the above modules, as part of the device, can run in a computer terminal.

[0101] It should be noted that the optional or preferred implementation methods of this embodiment can be found in the relevant descriptions in the embodiments, and will not be repeated here.

[0102] The aforementioned camera occlusion detection device may also include a processor and a memory. The aforementioned acquisition module 600, training module 602, determination module 604, etc., are all stored in the memory as program modules, and the processor executes the aforementioned program modules stored in the memory to realize the corresponding functions.

[0103] The processor contains a core that retrieves the corresponding program modules from memory. One or more cores may be configured. Memory may include non-persistent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory includes at least one memory chip.

[0104] According to an embodiment of this application, an embodiment of a non-volatile storage medium is also provided. Optionally, in this embodiment, the non-volatile storage medium includes a stored program, wherein, when the program runs, it controls the device containing the non-volatile storage medium to execute any of the aforementioned camera occlusion detection methods.

[0105] Optionally, in this embodiment, the non-volatile storage medium may be located in any computer terminal in a group of computer terminals in a computer network, or in any mobile terminal in a group of mobile terminals, and the non-volatile storage medium includes stored programs.

[0106] Optionally, during program execution, the device containing the non-volatile storage medium is controlled to perform the following functions: acquiring a pre-built initial camera occlusion detection model, wherein the initial camera occlusion detection model includes at least a feature extraction network, a hybrid spatial pyramid network, and a hybrid upsampling network, with the output of the feature extraction network connected to the input of the hybrid spatial pyramid network, and the output of the feature extraction network connected to the input of the hybrid upsampling network; inputting target image data into the initial camera occlusion detection model for training, obtaining a trained camera occlusion detection model; and using the trained camera occlusion detection model as the target camera occlusion detection model if the target loss function value corresponding to the trained camera occlusion detection model satisfies the loss function threshold, wherein the target loss function value is determined based on the first loss function value corresponding to the hybrid spatial pyramid network and the second loss function value corresponding to the hybrid upsampling network.

[0107] According to an embodiment of this application, an embodiment of a processor is also provided. Optionally, in this embodiment, the processor is used to run a program, wherein the program executes any of the above-described camera occlusion detection methods.

[0108] According to an embodiment of this application, an embodiment of a computer program product is also provided, which, when executed on a data processing device, is adapted to execute a program that initializes the camera occlusion detection method steps described above.

[0109] Optionally, when the above-mentioned computer program product is executed on a data processing device, it is suitable to execute an initialization program with the following method steps: obtaining a pre-constructed initial camera occlusion detection model, wherein the initial camera occlusion detection model includes at least a feature extraction network, a hybrid spatial pyramid network, and a hybrid upsampling network, wherein the output of the feature extraction network is connected to the input of the hybrid spatial pyramid network, and the output of the feature extraction network is connected to the input of the hybrid upsampling network; inputting target image data into the initial camera occlusion detection model for training to obtain a trained camera occlusion detection model; and, if the target loss function value corresponding to the trained camera occlusion detection model satisfies the loss function threshold, using the trained camera occlusion detection model as the target camera occlusion detection model, wherein the target loss function value is determined based on the first loss function value corresponding to the hybrid spatial pyramid network and the second loss function value corresponding to the hybrid upsampling network.

[0110] This invention provides an electronic device, which includes a processor, a memory, and a program stored in the memory and executable on the processor. When the processor executes the program, it performs the following steps: obtaining a pre-constructed initial camera occlusion detection model, wherein the initial camera occlusion detection model includes at least a feature extraction network, a hybrid spatial pyramid network, and a hybrid upsampling network; the output of the feature extraction network is connected to the input of the hybrid spatial pyramid network, and the output of the feature extraction network is connected to the input of the hybrid upsampling network; inputting target image data into the initial camera occlusion detection model for training, thereby obtaining a trained camera occlusion detection model; and, if the target loss function value corresponding to the trained camera occlusion detection model satisfies a loss function threshold, using the trained camera occlusion detection model as the target camera occlusion detection model, wherein the target loss function value is determined based on a first loss function value corresponding to the hybrid spatial pyramid network and a second loss function value corresponding to the hybrid upsampling network.

[0111] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0112] In the above embodiments of the present invention, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0113] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of modules described above can be a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, or indirect coupling or communication connection between modules, and may be electrical or other forms.

[0114] The modules described above as separate components may or may not be physically separate. Similarly, the components shown as modules may or may not be physical modules; they may be located in one place or distributed across multiple modules. Some or all of the modules can be selected to achieve the purpose of this embodiment, depending on actual needs.

[0115] Furthermore, the functional modules in the various embodiments of the present invention can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The integrated modules described above can be implemented in hardware or as software functional modules.

[0116] If the aforementioned integrated modules are implemented as software functional modules and sold or used as independent products, they can be stored in a computer-readable non-volatile storage medium. Based on this understanding, the technical solution of this invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a non-volatile storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this invention. The aforementioned non-volatile storage medium includes various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0117] The above are merely preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for detecting camera occlusion, characterized in that, include: Obtain a pre-constructed initial camera occlusion detection model, wherein the initial camera occlusion detection model includes at least a feature extraction network, a hybrid spatial pyramid network, and a hybrid upsampling network, wherein the output of the feature extraction network is connected to the input of the hybrid spatial pyramid network, and the output of the feature extraction network is connected to the input of the hybrid upsampling network; The target image data is input into the initial camera occlusion detection model for training to obtain the trained camera occlusion detection model; If the target loss function value of the trained camera occlusion detection model meets the loss function threshold, the trained camera occlusion detection model is used as the target camera occlusion detection model. The target loss function value is determined based on the first loss function value corresponding to the hybrid spatial pyramid network and the second loss function value corresponding to the hybrid upsampling network. The step of inputting the target image dataset into the initial camera occlusion detection model for training to obtain the trained camera occlusion detection model includes: inputting the target image dataset into the feature extraction network to obtain the feature extraction result output by the feature extraction network and the trained feature extraction network; inputting a first number of feature maps from the feature extraction result into the hybrid spatial pyramid network to obtain a first occlusion state output by the hybrid spatial pyramid network and the trained hybrid spatial pyramid network, wherein the first number of feature maps includes a second feature map output by the last network layer of the feature extraction network and one or more feature maps output by the intermediate network layers of the feature extraction network, and the first number of feature maps have different sizes; inputting a second number of feature maps from the feature extraction result into the hybrid upsampling network to obtain a first occlusion ratio output by the hybrid upsampling network and the trained hybrid upsampling network, wherein the second number of feature maps is multiple feature maps output by the intermediate network layers of the feature extraction network, and the second number of feature maps have different sizes; and obtaining the trained camera occlusion detection model based on the trained feature extraction network, the trained hybrid spatial pyramid network, and the trained hybrid upsampling network.

2. The method according to claim 1, characterized in that, Before using the trained camera occlusion detection model as the target camera occlusion detection model, provided that the target loss function value of the trained camera occlusion detection model meets the loss function threshold, the method further includes: The first loss function value is determined based on the first occlusion state and the actual occlusion state; The second loss function value is determined based on the first occlusion ratio and the actual occlusion ratio; The target loss function value is obtained based on the first loss function value and the second loss function value.

3. The method according to claim 2, characterized in that, The process of obtaining the target loss function value based on the first loss function value and the second loss function value includes: Based on the first loss function value and the second loss function value, the target loss function value is obtained in the following manner: Wherein, represents the target loss function value, represents the preset learnable parameter, represents the first loss function value, represents the second loss function value, represents the output dimension corresponding to the hybrid upsampling network, the first loss function value is determined based on the cross-entropy loss function, and the second loss function value is determined based on the binary cross-entropy loss function.

4. The method according to claim 1, characterized in that, In the case where the hybrid spatial pyramid network includes a spatial pyramid sampling network and a spatial hidden layer feature network, the step of inputting a first number of feature maps from the feature extraction results into the hybrid spatial pyramid network to obtain the first occlusion state output by the hybrid spatial pyramid network includes: The first number of feature maps output by the feature extraction network are input into the spatial pyramid sampling network to obtain the fused feature map corresponding to the first number of feature maps; The second feature map is input into the spatial hidden layer feature network to obtain the spatial weight parameters corresponding to the pixel positions included in the second feature map. Based on the fused feature map and the spatial weight parameters, the first occlusion state output by the hybrid spatial pyramid network is determined.

5. The method according to claim 4, characterized in that, In the case where the spatial pyramid network includes a first sampling network and a second sampling network, the step of inputting the first number of feature maps output by the feature extraction network into the spatial pyramid sampling network to obtain the fused feature map corresponding to the first number of feature maps includes: The first number of feature maps are input into the first sampling network to obtain the first output feature maps corresponding to the first number of feature maps respectively. The first sampling network includes two different convolutional layers and one pooling layer. The first number of feature maps are input into the second sampling network to obtain the second output feature maps corresponding to the first number of feature maps respectively. The second sampling network includes an average pooling layer and two pooling layers. The first output feature maps corresponding to the first number of feature maps and the second output feature maps corresponding to the first number of feature maps are concatenated to obtain the third output feature maps corresponding to the first number of feature maps. The third output feature maps corresponding to the first number of feature maps are fused to obtain the fused feature map.

6. The method according to claim 4, characterized in that, In the hybrid upsampling network, which includes a deconvolution layer and two bilinear interpolation layers, and where the second number of feature maps includes a third, fourth, and fifth feature map, and the sizes of the third, fourth, and fifth feature maps are in a predetermined multiple relationship, the step of inputting the second number of feature maps from the feature extraction result into the hybrid upsampling network to obtain the first occlusion ratio output by the hybrid upsampling network includes: The third feature map is subjected to a first upsampling process using the two bilinear interpolation layers to obtain the third feature map after the first upsampling process. The third feature map after the first upsampling process is concatenated with the fourth feature map to obtain the first concatenated feature map; The second stitched feature map is obtained by weighting the pixel positions included in the first stitched feature map based on the spatial weight parameters. The two bilinear interpolation layers are used to perform a second upsampling process on the second stitched feature map to obtain the second stitched feature map after the second upsampling process; The second spliced ​​feature map after the second upsampling process is spliced ​​with the fifth feature map to obtain the third spliced ​​feature map; The fourth stitched feature map is obtained by weighting the pixel positions included in the third stitched feature map based on the spatial weight parameters. The fourth stitched feature map is input into the first deconvolution layer for third upsampling processing to obtain the sixth feature map, wherein the sixth feature map has the same size as the target image data; The sixth feature map is subjected to semantic segmentation to obtain the first occlusion ratio.

7. The method according to claim 1, characterized in that, The feature extraction network includes: a third number of inverted residual layers, a fourth number of channel attention layers, and a fifth number of spatial attention layers. The inverted residual layers include two convolutional layers, two batch normalization layers, two activation layers, a data warehouse layer, and a residual layer. The channel attention layers include two convolutional layers, an average pooling layer, a max pooling layer, and two fully connected layers. The spatial attention layers include a convolutional layer and a sigmoid activation function.

8. The method according to any one of claims 1 to 7, characterized in that, After the trained camera occlusion detection model is used as the target camera occlusion detection model when the target loss function value of the trained camera occlusion detection model meets the loss function threshold, the method further includes: Acquire the image to be recognized; The image to be identified is input into the target camera occlusion detection model to obtain the second occlusion state and the second occlusion ratio corresponding to the image to be identified.

9. A camera occlusion detection device, characterized in that, include: An acquisition module is used to acquire a pre-built initial camera occlusion detection model, wherein the initial camera occlusion detection model includes at least a feature extraction network, a hybrid spatial pyramid network, and a hybrid upsampling network, wherein the output of the feature extraction network is connected to the input of the hybrid spatial pyramid network, and the output of the feature extraction network is connected to the input of the hybrid upsampling network. The training module is used to input the target image data into the initial camera occlusion detection model for training, so as to obtain the trained camera occlusion detection model. The determination module is used to determine the trained camera occlusion detection model as the target camera occlusion detection model when the target loss function value corresponding to the trained camera occlusion detection model meets the loss function threshold. The target loss function value is determined based on the first loss function value corresponding to the hybrid spatial pyramid network and the second loss function value corresponding to the hybrid upsampling network. The training module is further configured to: input the target image dataset into the feature extraction network to obtain the feature extraction result output by the feature extraction network and the trained feature extraction network; input a first number of feature maps from the feature extraction result into the hybrid spatial pyramid network to obtain a first occlusion state output by the hybrid spatial pyramid network and the trained hybrid spatial pyramid network, wherein the first number of feature maps includes a second feature map output by the last network layer of the feature extraction network and one or more feature maps output by the intermediate network layers of the feature extraction network, and the first number of feature maps have different sizes; input a second number of feature maps from the feature extraction result into the hybrid upsampling network to obtain a first occlusion ratio output by the hybrid upsampling network and the trained hybrid upsampling network, wherein the second number of feature maps is multiple feature maps output by the intermediate network layers of the feature extraction network, and the second number of feature maps have different sizes; and obtain the trained camera occlusion detection model based on the trained feature extraction network, the trained hybrid spatial pyramid network, and the trained hybrid upsampling network.

10. A non-volatile storage medium, characterized in that, The non-volatile storage medium stores multiple instructions, which are adapted to be loaded by a processor and executed by the camera occlusion detection method according to any one of claims 1 to 8.

11. An electronic device, characterized in that, The device includes one or more processors and a memory, the memory being used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors cause the one or more processors to implement the camera occlusion detection method according to any one of claims 1 to 8.

Images