Video surveillance method, device, equipment and storage medium for smart cultural and museums

Abstract

The present invention discloses a video surveillance method, apparatus, device, and storage medium for a smart cultural and museum venue. The present invention obtains images of a cultural and museum venue captured by surveillance cameras installed at different locations in the smart cultural and museum venue; inputs the images of the cultural and museum venue into an improved S-NanoDet model for target detection, and obtains target detection results; and monitors and warns the smart cultural and museum venue based on the target detection results. This embodiment uses the improved S-NanoDet model, which has high robustness and low computational complexity, to detect targets using image information captured by surveillance cameras in cultural and museum venues, thereby resolving the technical problem that security incidents may occur due to the poor robustness of the deployment model and complex computation of cameras in cultural and museum venues.

Description

Technical Field

[0001] The present invention relates to the field of intelligent video surveillance technology, and in particular to a video surveillance method, device, equipment and storage medium for smart cultural and museum use. Background Art

[0002] my country currently has approximately 200 important or first-level (provincial) cultural and museum venues, with this number projected to grow by 6% this year. Security at these venues largely relies on video surveillance. Current intelligent video surveillance technology can replace human effort in processing massive amounts of video data, and is therefore widely used in security and other fields. However, in real-world applications, cameras may generate video data of varying styles depending on their deployment environment. For example, in low-light environments, cameras may capture infrared video data, while in brightly lit settings, they may capture normal-color video data. Consequently, deep learning network models deployed on these cameras may need to process video data of varying styles. If the image styles being processed are relatively rare in the model's training set, this poses significant challenges to the model's robustness. Furthermore, as edge computing devices, cameras have limited computing power, which can lead to blurry surveillance footage in certain scenarios, potentially causing security issues at cultural and museum venues.

[0003] The above content is only used to assist in understanding the technical solution of the present invention and does not constitute an admission that the above content is prior art. Summary of the Invention

[0004] The main purpose of the present invention is to provide a video surveillance method, device, equipment and storage medium for smart cultural and museums, aiming to solve the technical problem in the existing technology that the cameras in cultural and museum venues may cause security accidents due to their poor robustness of deployment models and complex calculations.

[0005] To achieve the above object, the present invention provides a video surveillance method for a smart cultural museum, the method comprising the following steps:

[0006] Obtain images of the smart cultural and museum venues collected by surveillance cameras installed at different locations in the venues;

[0007] The image of the cultural and museum venue is input into the improved S-NanoDet model for target detection, and the target detection result is obtained. The improved S-NanoDet model includes an S-GhostNet backbone network, a path aggregation network, and a classification regression network connected in sequence. The improved S-NanoDet model also includes a CNSN structure inserted at the bottleneck layer. The S-GhostNet backbone network is used to extract image feature information and transmit the image feature information to the subsequent network. The CNSN structure is used to exchange the mean and variance of the feature map channel and adjust the mean and variance of the feature map channel through the attention mechanism. The loss function of the improved S-NanoDet model is the GFLV2 loss function;

[0008] The smart cultural and museum venue is monitored and warned based on the target detection results.

[0009] Optionally, the classification regression network includes a regression branch and a classification branch, and a link is set between the regression branch and the classification branch, and the link is a lightweight network including 2 fully connected layers and corresponding ReLU and Sigmoid activation functions.

[0010] Optionally, before the step of inputting the image of the cultural and museum venue into the improved S-NanoDet model for target detection, the method further includes:

[0011] Obtain the smart museum image dataset;

[0012] The smart museum image dataset is used to iteratively train the S-NanoDet model to be trained to obtain an improved S-NanoDet model.

[0013] Optionally, before the step of iteratively training the S-NanoDet model to be trained using the smart cultural museum image dataset to obtain an improved S-NanoDet model, the method further includes:

[0014] Replace the backbone network ShuffleNetV2 of the original NanoDet model with the S-GhostNet backbone network to obtain the initial S-NanoDe model;

[0015] The GFL loss function in the initial S-NanoDe model is replaced by the GFLV2 loss function to obtain the target S-NanoDet model;

[0016] The CNSN structure is inserted at the bottleneck layer preset position of each bottleneck layer of the target S-NanoDet model to replace the BN structure of the target S-NanoDet model to obtain the S-NanoDet model to be trained.

[0017] Optionally, the CNSN structure includes a CrossNorm structure and a SelfNorm structure, the CrossNorm structure is used to exchange the mean and variance of the feature map channel, and the SelfNorm structure is used to adjust the mean and variance of the feature map channel using an attention mechanism.

[0018] Optionally, the CNSN structure is set at the header pre position of each bottleneck layer in the S-GhostNet backbone network.

[0019] Optionally, the smart museum image dataset is a PASCAL VOC2012 dataset, and the images in the PASCAL VOC2012 dataset include four major categories: people, common animals, transportation vehicles, and indoor furniture and supplies.

[0020] In addition, to achieve the above-mentioned purpose, the present invention further proposes a video surveillance device for smart cultural relics, wherein the video surveillance device for smart cultural relics includes:

[0021] An image acquisition module is used to acquire images of the smart cultural and museum venues collected by surveillance cameras installed at different locations in the venues;

[0022] An image detection module is used to input the image of the cultural and museum venue into the improved S-NanoDet model for target detection and obtain target detection results. The improved S-NanoDet model includes an S-GhostNet backbone network, a path aggregation network, and a classification regression network connected in sequence. The improved S-NanoDet model also includes a CNSN structure inserted at the bottleneck layer. The S-GhostNet backbone network is used to extract image feature information and transmit the image feature information to the subsequent network. The CNSN structure is used to exchange the mean and variance of the feature map channel and adjust the mean and variance of the feature map channel through the attention mechanism. The loss function of the improved S-NanoDet model is the GFLV2 loss function.

[0023] The venue early warning module is used to monitor and warn the smart cultural and museum venue based on the target detection results.

[0024] In addition, to achieve the above-mentioned purpose, the present invention also proposes a video surveillance device under smart cultural relics, which includes: a memory, a processor, and a video surveillance program under smart cultural relics stored on the memory and runnable on the processor. The video surveillance program under smart cultural relics is configured to implement the steps of the video surveillance method under smart cultural relics as described above.

[0025] In addition, to achieve the above-mentioned purpose, the present invention also proposes a storage medium, on which a video monitoring program under Smart Wenbo is stored. When the video monitoring program under Smart Wenbo is executed by the processor, the steps of the video monitoring method under Smart Wenbo as described above are implemented.

[0026] The present invention obtains images of cultural and museum venues collected by surveillance cameras installed at different locations of the smart cultural and museum venues; inputs the images of the cultural and museum venues into the improved S-NanoDet model for target detection, and obtains target detection results, the improved S-NanoDet model includes an S-GhostNet backbone network, a path aggregation network and a classification regression network connected in sequence, the improved S-NanoDet model also includes a CNSN structure inserted at the bottleneck layer, the S-GhostNet backbone network is used to extract image feature information and transmit the image feature information to the subsequent network, the CNSN structure is used to exchange the mean and variance of the feature map channel and adjust the mean and variance of the feature map channel through the attention mechanism, the loss function of the improved S-NanoDet model is the GFLV2 loss function; the smart cultural and museum venues are monitored and warned according to the target detection results. The present invention uses the improved S-NanoDet model with high robustness and low computational complexity to detect targets using image information collected by surveillance cameras in cultural and museum venues, thereby solving the technical problem that security accidents may occur due to the poor robustness of the deployment model and complex calculations of cameras in cultural and museum venues. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 It is a structural diagram of a video surveillance device in a smart cultural and museum hardware operating environment involved in an embodiment of the present invention;

[0028] Figure 2 This is a flow chart of a first embodiment of the video surveillance method for smart cultural relics according to the present invention;

[0029] Figure 3 This is a flow chart of a second embodiment of the video surveillance method for smart cultural relics according to the present invention;

[0030] Figure 4 Schematic diagram of the DGQP structure of the video monitoring method under the smart cultural heritage of the present invention;

[0031] Figure 5 This is a structural comparison diagram of the NanoDet model and the FCOS model in the video monitoring method under the smart cultural heritage of the present invention;

[0032] Figure 6 This is a schematic diagram of CrossNorm and SelfNorm in the video surveillance method under the smart cultural heritage of the present invention;

[0033] Figure 7 This is a schematic diagram of the CNSN structure flow in the video monitoring method under the smart cultural and museum system of the present invention;

[0034] Figure 8 A schematic diagram of the CNSN structure used in the video surveillance method under the smart museum of the present invention;

[0035] Figure 9 This is a structural block diagram of the first embodiment of the video surveillance device under the smart cultural heritage of the present invention.

[0036] Table 1:

[0037]

[0038] The purpose, features and advantages of the present invention will be further described with reference to the accompanying drawings and in conjunction with the embodiments. DETAILED DESCRIPTION

[0039] It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0040] Reference Figure 1 , Figure 1 This is a schematic diagram of the video surveillance device structure in the smart cultural and art museum hardware operating environment involved in the embodiment of the present invention.

[0041] like Figure 1 As shown, the video surveillance equipment under the smart cultural and museum system may include: a processor 1001, such as a central processing unit (CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. The communication bus 1002 is used to enable communication between these components. The user interface 1003 may include a display and an input unit such as a keyboard. Optionally, the user interface 1003 may also include a standard wired interface or a wireless interface. The network interface 1004 may optionally include a standard wired interface or a wireless interface (such as a wireless fidelity (Wi-Fi) interface). The memory 1005 may be a high-speed random access memory (RAM) or a stable non-volatile memory (NVM), such as a disk storage device. The memory 1005 may also be a storage device independent of the processor 1001.

[0042] Those skilled in the art will understand that Figure 1The structure shown in the figure does not constitute a limitation on the video surveillance equipment under the smart cultural and museum system, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0043] like Figure 1 As shown, the memory 1005 as a storage medium may include an operating system, a network communication module, a user interface module, and a video monitoring program under the smart cultural heritage.

[0044] exist Figure 1 In the video surveillance device under the smart cultural heritage shown, the network interface 1004 is mainly used for data communication with the network server; the user interface 1003 is mainly used for data interaction with the user; the processor 1001 and the memory 1005 in the video surveillance device under the smart cultural heritage of the present invention can be set in the video surveillance device under the smart cultural heritage, and the video surveillance device under the smart cultural heritage calls the video surveillance program under the smart cultural heritage stored in the memory 1005 through the processor 1001, and executes the video surveillance method under the smart cultural heritage provided by the embodiment of the present invention.

[0045] The embodiment of the present invention provides a video monitoring method under smart cultural and museum conditions, referring to Figure 2 , Figure 2 This is a flow chart of the first embodiment of the video surveillance method for smart cultural relics in the present invention.

[0046] In this embodiment, the video surveillance method under the smart cultural and museum system includes the following steps:

[0047] Step S10: Acquire images of the smart cultural and museum venue captured by surveillance cameras installed at different locations of the smart cultural and museum venue.

[0048] It should be noted that the execution entity of this embodiment can be a microcomputer, which can be installed in a surveillance camera. A smart cultural and museum venue is a digital museum that preserves cultural relics without spatial constraints, and eliminates the problem of being unable to display due to exhibition space or conditions. The surveillance camera can be specially designed for the smart cultural and museum venue and meet the venue's needs.

[0049] Step S20: Input the image of the cultural and museum venue into the improved S-NanoDet model for target detection, and obtain the target detection result. The improved S-NanoDet model includes an S-GhostNet backbone network, a path aggregation network and a classification regression network connected in sequence. The improved S-NanoDet model also includes a CNSN structure inserted at the bottleneck layer. The S-GhostNet backbone network is used to extract image feature information and transmit the image feature information to the subsequent network. The CNSN structure is used to exchange the mean and variance of the feature map channel and adjust the mean and variance of the feature map channel through the attention mechanism. The loss function of the improved S-NanoDet model is the GFLV2 loss function.

[0050] It should be noted that the improved S-NanoDet model is an improvement on the lightweight object detection model NanoDet. NanoDet is a single-stage anchor-free object detection model based on FCOS (Fully Convolutional One-Stage Object Detection). It uses the Adaptive Training Sample Selection (ATSS) strategy for object sampling and the Generalized Focal Loss (GFL) loss function for classification and bounding box regression.

[0051] It is understandable that before selecting the NanoDet model as the base model, a comparative experiment was conducted between the NanoDet model and the YOLOv3-tiny model and the YOLOv4-tiny model. The comparative experiment used the COCO dataset mAP as the evaluation standard. Table 1 shows the results of the comparative experiment. It can be seen from Table 1 that when the input resolution is 320×320, the image detection accuracy of the NanoDet model can reach 20.6%, which is 4% higher than the YOLOv3-tiny model and 1% lower than the YOLOv4-tiny model. When the input resolution is 416×416, the image detection accuracy of the NanoDet model is still higher than the YOLOv3-tiny model and is the same as the YOLOv4-tiny model. Through this comparative experiment, it can be seen that the NanoDet model is the most suitable model for surveillance cameras among the existing target detection models. Therefore, the improved S-NanoDet model in this embodiment selects the NanoDet model as the initial model.

[0052] It should be understood that the backbone network, loss function, and normalization method of the improved S-NanoDet model are different from those of the NanoDet model. The backbone network of the improved S-NanoDet model adopts the S-GhostNet algorithm, which further reduces the computational complexity of the backbone network. The loss function GFLV2 it adopts uses the statistics of the uncertainty of the bounding box to efficiently guide the positioning quality estimation, which can provide more precise detection box positioning, making target detection more accurate without incurring additional computational complexity. The normalization method used by the improved S-NanoDet model is to insert the CNSN structure at the bottleneck layer to improve the generalization performance of the model.

[0053] Step S30: Monitor and issue early warning for the smart cultural and museum venue based on the target detection result.

[0054] It should be noted that the target detection result can be divided into abnormal and non-abnormal. When the target detection result is abnormal, an alert will be sent to the staff of the cultural and museum venues through the Internet of Things so that the staff can conduct inspections and eliminate safety hazards.

[0055] This embodiment obtains images of the cultural and museum venues collected by surveillance cameras installed at different locations of the smart cultural and museum venues; inputs the images of the cultural and museum venues into the improved S-NanoDet model for target detection, and obtains target detection results. The improved S-NanoDet model includes an S-GhostNet backbone network, a path aggregation network, and a classification regression network connected in sequence. The improved S-NanoDet model also includes a CNSN structure inserted at the bottleneck layer. The S-GhostNet backbone network is used to extract image feature information and transmit the image feature information to the subsequent network. The CNSN structure is used to exchange the mean and variance of the feature map channel and adjust the mean and variance of the feature map channel through the attention mechanism. The loss function of the improved S-NanoDet model is the GFLV2 loss function; the smart cultural and museum venue is monitored and warned according to the target detection results. This embodiment uses the improved S-NanoDet model with high robustness and low computational complexity to detect targets using image information collected by surveillance cameras in cultural and museum venues, thereby solving the technical problem that security accidents may occur due to the poor robustness of the deployment model and complex calculations of cameras in cultural and museum venues.

[0056] refer to Figure 3 , Figure 3 This is a flow chart of the second embodiment of the video surveillance method for smart cultural relics in the present invention.

[0057] Based on the above-mentioned first embodiment, in this embodiment, the classification regression network includes a regression branch and a classification branch, and a link is set between the regression branch and the classification branch. The link is a lightweight network including 2 fully connected layers and corresponding ReLU and Sigmoid activation functions.

[0058] It should be noted that, referring to Figure 4 , Figure 4 The DGQP structure diagram of the video monitoring method under the smart cultural heritage of the present invention. The DGQP (Distribution-Guided Quality Predictor) structure contains two fully connected layers and corresponding ReLU and Sigmoid activation functions. By multiplying the output I of the DGQP structure with the classification branch C, the result J is supervised learning. Where J = C × J, C = [C1, C2, ..., Cm], C i ∈[0, 1] represents the classification representation of a total of m categories, and I∈[0, 1] is a scalar representing the IoU representation. The four boundaries of the built box are marked as l, r, t, and b respectively. The discrete probability of the ω edge can be defined as:

[0059] ;

[0060] The basic statistical feature F∈R is obtained by connecting the Top-k value and the average value of each distribution vector 4（k+1） :

[0061] ;

[0062] Among them, Top-km(·) represents the joint operation of calculating the Top-k value and its average, and Concat(·) represents channel concatenation. Using the statistical feature F of the general distribution as input, a subnetwork is designed to predict the final IoU quality evaluation. The subnetwork consists of two fully connected layers, one fully connected layer followed by a ReLU activation function, and the other fully connected layer followed by a Sigmoid activation function. Therefore, the formula for the IoU scalar I is:

[0063] ;

[0064] Where σ and δ refer to the Sigmoid activation function and the ReLU activation function respectively, W1∈R p×4（k+1） , W2∈R 1×p k represents the Top-k parameter, and p represents the number of channels in the hidden layer.

[0065] In this embodiment, before step S20, the following steps are further included:

[0066] Step S01: Obtain a smart museum image dataset;

[0067] Step S02: The S-NanoDet model to be trained is iteratively trained using the smart museum image dataset to obtain an improved S-NanoDet model.

[0068] It can be understood that after obtaining the S-NanoDet model to be trained, it needs to be trained with the corresponding training set so that it can accurately detect pictures of the corresponding type of the training set. The S-NanoDet model to be trained in this embodiment is applied to video surveillance of future cultural and museum venues, so it needs to be trained with the picture set of smart cultural and museum venues; the picture set of the smart cultural and museum venues can be obtained by taking relevant indoor and outdoor pictures of various future cultural and museum venues across the country, or it can be pictures of a specific smart cultural and museum venue. The improved S-NanoDet model obtained by training with pictures of the specific smart cultural and museum venue will be applied to the camera of the specific smart cultural and museum venue.

[0069] In this embodiment, before step S02, the method further includes:

[0070] Step S021: Replace the backbone network ShuffleNetV2 of the original NanoDet model with the S-GhostNet backbone network to obtain the initial S-NanoDe model;

[0071] Step S022: replacing the GFL loss function in the initial S-NanoDe model with the GFLV2 loss function to obtain a target S-NanoDet model;

[0072] Step S023: inserting the CNSN structure at the bottleneck layer preset position of each bottleneck layer of the target S-NanoDet model to replace the BN structure of the target S-NanoDet model to obtain the S-NanoDet model to be trained.

[0073] It should be noted that the reference Figure 5 , Figure 5 This is a structural comparison diagram of the NanoDet model and the FCOS model in the video monitoring method under the smart cultural heritage of the present invention; Figure 6 It can be seen that the original NanoDet model includes a backbone network, a path aggregation network and a classification regression network. Its backbone network is the ShuffleNetV2 network, the path aggregation network is the PAN network, and the loss function used in the classification regression network is the GFL loss function.

[0074] It can be understood that in order to reduce the amount of model calculation, the S-GhostNet backbone network is used in this embodiment to replace the backbone network ShuffleNetV2 of the original NanoDet model to obtain the initial S-NanoDet model; further, in order to improve the target detection accuracy of the model, the GFLV2 loss function is used to replace the GFL loss function in the original NanoDet model to obtain the target S-NanoDet model; in order to specifically improve the generalization performance of the model, the CNSN structure is inserted at the preset position of the bottleneck layer of each bottleneck layer of the target S-NanoDet model to replace the BN (BatchNormalization) structure of the target S-NanoDet model to obtain the S-NanoDet model to be trained.

[0075] It should be understood that the normalization method used in the original NanoDet model is to adopt the BN structure. The purpose of this method is to make each feature have a distribution with a mean of 0 and a variance of 1. When the training data and the test data come from the same distribution, BN makes the training more stable and the convergence speed faster. However, the distribution changes in real scenes are more complex, which makes the traditional normalization techniques like BN perform poorly. In this embodiment, the S-NanoDet model to be trained is applied to video surveillance of smart cultural and museum venues. Therefore, the distribution of target detection data is inconsistent and complex, so the BN structure is not suitable. In order to improve the generalization ability of the model, this embodiment adopts the CNSN structure. The CNSN structure includes the CrossNorm structure and the SelfNorm structure. The CrossNorm structure uses the image RGB mean and variance to transmit style features. It exchanges the mean and variance of the feature map channel. This method does not affect the category label. At the same time, it overcomes the problem of texture destruction during style conversion, aligns different domains, and enhances the distribution of training data. After enhancing the distribution of the training set, the model will still encounter style category detection targets that are not in the training set. Therefore, it is necessary to use the SelfNorm structure to use the attention mechanism to adjust the mean and variance of the feature map channel to achieve the purpose of reducing style differences. Figure 6 , Figure 6 This is the principle diagram of CrossNorm and SelfNorm in the video surveillance method under the smart cultural museum of the present invention. As can be seen from the figure, CrossNorm and SelfNorm are orthogonal, although their design directions are opposite, that is, CrossNorm transmits statistical data between channels, enriching the combination of standardized features and statistical data, while SelfNorm redesigns and calibrates statistical data, focusing only on necessary styles, reducing standardized features and statistical data diversity. The opposite operations of the two are mainly because they target different stages, refer to Figure 7 , Figure 7This figure shows the CNSN architecture flow diagram for the video surveillance method for Smart Cultural Heritage. CrossNorm is enabled only during training, while SelfNorm is learned during training and then recalibrated during testing. In video surveillance scenarios, images captured by cameras often differ in style from those in the training set. Therefore, the CNSN architecture effectively addresses this issue and optimizes the performance of the S-NanoDet model in video surveillance scenarios.

[0076] In this embodiment, in order to obtain the optimal setting position of the CNSN structure, a comparative experiment was conducted. In this comparative experiment, the CNSN structure was inserted into four different positions of the bottleneck layer of the improved S-NanoDet model. The four positions include "pre", "identity", "residual", and "post". Figure 8 Four different improved models were obtained and tested on the PASCAL VOC2012 dataset, which includes four major categories: people, common animals, vehicles, and indoor furniture. The dataset contains 11,540 training images and 3,000 test images. Based on the final experimental results, the improved S-NanoDet model achieved the best object detection performance by placing the CNSN structure at the pre-position of each bottleneck layer in the S-GhostNet backbone network.

[0077] It should be understood that in this embodiment, the smart museum image dataset used is the PASCAL VOC2012 dataset. The images in the PASCAL VOC2012 dataset include four major categories: people, common animals, transportation vehicles, and indoor furniture and supplies, including 11,540 training set images and 3,000 test set images.

[0078] Through continuous comparative experiments, this embodiment selected the S-GhostNet backbone network to reduce the model calculation amount, selected the GFLV2 loss function to optimize the model accuracy, adopted the CNSN structure for normalization, and set the CNSN at the pre position of the bottleneck layer header to maximize the effect of CNSN, thereby improving the generalization performance of the model, making the final improved S-NanoDet model more suitable for video surveillance scenarios in smart cultural and museum venues.

[0079] In addition, an embodiment of the present invention further proposes a storage medium, on which a video monitoring program for smart cultural relics is stored. When the video monitoring program for smart cultural relics is executed by a processor, the steps of the video monitoring method for smart cultural relics as described above are implemented.

[0080] Reference Figure 9 , Figure 9 This is a structural block diagram of the first embodiment of the video surveillance device under the smart cultural heritage of the present invention.

[0081] like Figure 9 As shown, the video surveillance device for smart cultural relics proposed in the embodiment of the present invention includes:

[0082] The image acquisition module 501 is used to acquire images of the smart cultural and museum venue collected by surveillance cameras installed at different locations of the smart cultural and museum venue.

[0083] It should be noted that the execution entity of this embodiment can be a microcomputer, which can be installed in a surveillance camera. A smart cultural and museum venue is a digital museum that preserves cultural relics without spatial constraints, and eliminates the problem of being unable to display due to exhibition space or conditions. The surveillance camera can be specially designed for the smart cultural and museum venue and meet the venue's needs.

[0084] The image detection module 502 is used to input the image of the cultural and museum venue into the improved S-NanoDet model for target detection and obtain the target detection result. The improved S-NanoDet model includes an S-GhostNet backbone network, a path aggregation network and a classification regression network connected in sequence. The improved S-NanoDet model also includes a CNSN structure inserted at the bottleneck layer. The S-GhostNet backbone network is used to extract image feature information and transmit the image feature information to the subsequent network. The CNSN structure is used to exchange the mean and variance of the feature map channel and adjust the mean and variance of the feature map channel through the attention mechanism. The loss function of the improved S-NanoDet model is the GFLV2 loss function.

[0085] It should be noted that the improved S-NanoDet model is an improvement on the lightweight object detection model NanoDet. NanoDet is a single-stage anchor-free object detection model based on FCOS (Fully Convolutional One-Stage Object Detection). It uses the Adaptive Training Sample Selection (ATSS) strategy for object sampling and the Generalized Focal Loss (GFL) loss function for classification and bounding box regression.

[0086] It is understandable that before choosing the NanoDet model as the base model, the NanoDet model was compared with the YOLOv3-tiny model and the YOLOv4-tiny model. Figure 3 , Figure 3 This is a model comparison experiment diagram of the first embodiment of the video surveillance method under the smart museum of the present invention; the comparison experiment uses the COCO dataset mAP as the evaluation standard. Figure 3 It can be seen that when the input resolution is 320×320, the image detection accuracy of the NanoDet model can reach 20.6%, which is 4% higher than the YOLOv3-tiny model and 1% lower than the YOLOv4-tiny model. When the input resolution is 416×416, the image detection accuracy of the NanoDet model is still higher than the YOLOv3-tiny model and is the same as the YOLOv4-tiny model. Through this comparative experiment, it can be seen that the NanoDet model is the most suitable model for surveillance cameras among the existing target detection models. Therefore, the improved S-NanoDet model in this embodiment selects the NanoDet model as the initial model.

[0087] It should be understood that the backbone network, loss function, and normalization method of the improved S-NanoDet model are different from those of the NanoDet model. The backbone network of the improved S-NanoDet model adopts the S-GhostNet algorithm, which further reduces the computational complexity of the backbone network. The loss function GFLV2 it adopts uses the statistics of the uncertainty of the bounding box to efficiently guide the positioning quality estimation, which can provide more precise detection box positioning, making target detection more accurate without incurring additional computational complexity. The normalization method used by the improved S-NanoDet model is to insert the CNSN structure at the bottleneck layer to improve the generalization performance of the model.

[0088] The venue early warning module 503 is used to monitor and issue early warnings to the smart cultural and museum venue based on the target detection results.

[0089] It should be noted that the target detection result can be divided into abnormal and non-abnormal. When the target detection result is abnormal, an alert will be sent to the staff of the cultural and museum venues through the Internet of Things so that the staff can conduct inspections and eliminate safety hazards.

[0090] This embodiment obtains images of the cultural and museum venues collected by surveillance cameras installed at different locations of the smart cultural and museum venues; inputs the images of the cultural and museum venues into the improved S-NanoDet model for target detection, and obtains target detection results. The improved S-NanoDet model includes an S-GhostNet backbone network, a path aggregation network, and a classification regression network connected in sequence. The improved S-NanoDet model also includes a CNSN structure inserted at the bottleneck layer. The S-GhostNet backbone network is used to extract image feature information and transmit the image feature information to the subsequent network. The CNSN structure is used to exchange the mean and variance of the feature map channel and adjust the mean and variance of the feature map channel through the attention mechanism. The loss function of the improved S-NanoDet model is the GFLV2 loss function; the smart cultural and museum venue is monitored and warned according to the target detection results. This embodiment uses the improved S-NanoDet model with high robustness and low computational complexity to detect targets using image information collected by surveillance cameras in cultural and museum venues, thereby solving the technical problem that security accidents may occur due to the poor robustness of the deployment model and complex calculations of cameras in cultural and museum venues.

[0091] Other embodiments or specific implementations of the video surveillance device under the smart cultural heritage of the present invention can refer to the above-mentioned method embodiments and will not be repeated here.

[0092] It should be noted that, in this document, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or system comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or system. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or system comprising the element.

[0093] The serial numbers of the above embodiments of the present invention are for description only and do not represent the advantages or disadvantages of the embodiments.

[0094] Through the above description of the embodiments, those skilled in the art will clearly understand that the above-mentioned embodiments and methods can be implemented by means of software plus the necessary general-purpose hardware platform. Of course, hardware can also be used, but in many cases the former is a more preferred embodiment. Based on this understanding, the technical solution of the present invention, or the portion that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as read-only memory / random access memory, a magnetic disk, or an optical disk) and includes a number of instructions for enabling a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0095] The above are only preferred embodiments of the present invention and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention description and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A video surveillance method for smart cultural relics, characterized in that: The method comprises the following steps: Obtain images of the smart cultural and museum venues collected by surveillance cameras installed at different locations in the venues; The image of the cultural and museum venue is input into the improved S-NanoDet model for target detection, and the target detection result is obtained. The improved S-NanoDet model includes an S-GhostNet backbone network, a path aggregation network, and a classification regression network connected in sequence. The improved S-NanoDet model also includes a CNSN structure inserted at the bottleneck layer. The S-GhostNet backbone network is used to extract image feature information and transmit the image feature information to the subsequent network. The CNSN structure is used to exchange the mean and variance of the feature map channel and adjust the mean and variance of the feature map channel through the attention mechanism. The loss function of the improved S-NanoDet model is the GFLV2 loss function; Performing monitoring and early warning on the smart cultural museum venue according to the target detection result; Before the step of inputting the image of the cultural and museum venues into the improved S-NanoDet model for target detection, the method further includes: Obtain the smart museum image dataset; The S-NanoDet model to be trained is iteratively trained using the smart museum image dataset to obtain an improved S-NanoDet model; Before the step of iteratively training the S-NanoDet model to be trained using the smart museum image dataset to obtain an improved S-NanoDet model, the method further includes: Replace the backbone network ShuffleNetV2 of the original NanoDet model with the S-GhostNet backbone network to obtain the initial S-NanoDe model; The GFL loss function in the initial S-NanoDe model is replaced by the GFLV2 loss function to obtain the target S-NanoDet model; The CNSN structure is inserted at the bottleneck layer preset position of each bottleneck layer of the target S-NanoDet model to replace the BN structure of the target S-NanoDet model to obtain the S-NanoDet model to be trained.

2. The video surveillance method for smart cultural relics according to claim 1, characterized in that: The classification regression network includes a regression branch and a classification branch. A link is set between the regression branch and the classification branch. The link is a lightweight network including two fully connected layers and corresponding ReLU and Sigmoid activation functions.

3. The video surveillance method for smart cultural relics according to claim 1, characterized in that: The CNSN structure includes a CrossNorm structure and a SelfNorm structure. The CrossNorm structure is used to exchange the mean and variance of the feature map channel, and the SelfNorm structure is used to adjust the mean and variance of the feature map channel using the attention mechanism.

4. The video surveillance method for smart cultural relics according to claim 3, characterized in that: The CNSN structure is set at the header pre position of each bottleneck layer in the S-GhostNet backbone network.

5. The video surveillance method for a smart cultural heritage site according to any one of claims 3 to 4, characterized in that: The smart museum image dataset is the PASCAL VOC2012 dataset. The images in the PASCAL VOC2012 dataset include four major categories: people, common animals, transportation vehicles, and indoor furniture and supplies.

6. A video surveillance device for smart cultural relics, characterized in that: The video surveillance device under the smart cultural and museum system includes: An image acquisition module is used to acquire images of the smart cultural and museum venues collected by surveillance cameras installed at different locations in the venues; An image detection module is used to input the image of the cultural and museum venue into the improved S-NanoDet model for target detection and obtain target detection results. The improved S-NanoDet model includes an S-GhostNet backbone network, a path aggregation network, and a classification regression network connected in sequence. The improved S-NanoDet model also includes a CNSN structure inserted at the bottleneck layer. The S-GhostNet backbone network is used to extract image feature information and transmit the image feature information to the subsequent network. The CNSN structure is used to exchange the mean and variance of the feature map channel and adjust the mean and variance of the feature map channel through the attention mechanism. The loss function of the improved S-NanoDet model is the GFLV2 loss function. A venue early warning module is used to monitor and warn the smart cultural and museum venue based on the target detection results; The video surveillance device under the smart museum is used to obtain a smart museum image dataset; the smart museum image dataset is used to iteratively train the S-NanoDet model to be trained to obtain an improved S-NanoDet model; The video surveillance device under the smart cultural heritage is used to replace the backbone network ShuffleNetV2 of the original NanoDet model with the S-GhostNet backbone network to obtain the initial S-NanoDet model; replace the GFL loss function in the initial S-NanoDet model with the GFLV2 loss function to obtain the target S-NanoDet model; insert the CNSN structure at the bottleneck layer preset position of each bottleneck layer of the target S-NanoDet model to replace the BN structure of the target S-NanoDet model to obtain the S-NanoDet model to be trained.

7. A video surveillance device for smart cultural relics, characterized in that: The device includes: a memory, a processor, and a video monitoring program for smart cultural relics stored in the memory and executable on the processor. The video monitoring program for smart cultural relics is configured to implement the steps of the video monitoring method for smart cultural relics as described in any one of claims 1 to 5.

8. A storage medium, characterized in that: The storage medium stores a video monitoring program for smart cultural relics, and when the video monitoring program for smart cultural relics is executed by the processor, the steps of the video monitoring method for smart cultural relics as described in any one of claims 1 to 5 are implemented.

Images