Training base security monitoring method and system based on deep learning

Abstract

The invention relates to the technical field of training base security and protection monitoring methods, in particular to a training base security and protection monitoring method and system based on deep learning, and the method comprises the following steps: obtaining a monitoring region map and a semantic map of each scene at each moment; carrying out noise reduction processing on each frame of image of each scene at each moment by adopting a median filtering method; obtaining a plurality of initial high-dimensional feature maps of each frame of image of each scene at each moment; obtaining a plurality of final high-dimensional feature maps of each scene at each moment; obtaining low-rank characteristics of the same position at all moments; obtaining combined features of each scene at each moment; obtaining a combined video stream feature of each scene; the method comprises the following steps: acquiring an abnormal detection result of each scene, introducing a monitoring area graph and a semantic graph to more accurately position a key attention area, understanding semantic information of objects and personnel in the scene, processing high-dimensional features by adopting a low-rank initialization model, effectively removing redundant information, and extracting an essential behavior mode.

Description

Technical Field

[0001] The present invention relates to the technical field of training base security monitoring methods, and in particular to a training base security monitoring method and system based on deep learning. Background Art

[0002] As the scale of training bases continues to expand and their functions become increasingly complex, traditional security monitoring methods can no longer meet the current security management needs. In recent years, intelligent security systems based on deep learning have gradually become a research hotspot. Such systems analyze video data to identify abnormal behaviors and provide strong support for security management.

[0003] At present, mainstream intelligent security systems mostly use convolutional neural networks (CNN) to extract features of video frames, and recurrent neural networks (RNN) or long short-term memory networks (LSTM) to capture time series information. This method has achieved good results in detecting abnormal behavior in a short period of time, but it still has many shortcomings when dealing with long time series and complex scenes.

[0004] First, existing methods tend to focus on abnormal behaviors at a single moment, ignoring abnormal patterns over a long time scale. For example, a person's short stay in a specific area may be normal behavior, but if it occurs frequently or lasts for too long, it may mean a potential threat. Existing systems have difficulty effectively capturing such long-term behavior patterns.

[0005] Secondly, current intelligent security systems perform poorly in dealing with complex environments with multiple scenarios and scales. Training bases usually contain multiple functional areas, such as training grounds, dormitory areas, and armories, and the normal behavior patterns and potential risks of each area are different. Existing technologies are difficult to take into account the characteristics of these different scenarios at the same time, resulting in high false alarm rates or frequent missed alarms.

[0006] Furthermore, existing methods have deficiencies in feature extraction and fusion. They usually directly use the features extracted by CNN, fail to fully utilize the semantic information of the scene and the importance of the region, resulting in the extracted features being incomplete and inaccurate. At the same time, in the feature fusion process, simple splicing or weighted averaging is often used, which fails to effectively integrate the information of spatial and temporal dimensions.

[0007] In addition, existing systems perform poorly in dealing with noise and redundant information. The surveillance videos of training bases often contain a lot of repeated and irrelevant information. How to effectively remove these interferences and extract truly valuable features is another challenge facing existing technologies.

[0008] Finally, current anomaly detection algorithms are mostly based on predefined rules or simple statistical models, which are difficult to adapt to complex and changing actual scenarios. This results in the system's lack of ability to identify new types of anomalies and difficulty in making flexible adjustments based on specific circumstances.

[0009] In view of the above problems, a new method for comprehensive, accurate and flexible security monitoring of training bases is urgently needed. The present invention is proposed to solve these technical problems. Summary of the invention

[0010] In response to the above problems, the present invention proposes a training base security monitoring method and system based on deep learning. By innovatively combining multi-level feature extraction, low-rank feature learning, multi-level feature fusion and other technologies, the present invention aims to build an intelligent security system that can capture short-term abnormal behaviors and identify long-term abnormal patterns.

[0011] The present invention proposes a training base security monitoring method based on deep learning, comprising the following steps: First, an image of each scene at each moment is obtained, and the image is sequentially input into a monitoring area recognition model and a semantic recognition model to obtain a monitoring area map and a semantic map of each scene at each moment respectively; Then, each frame of each scene at each moment is subjected to noise reduction processing using the median filter method to remove interference noise; Next, each frame of the image after denoising is input into a pre-trained deep learning model to obtain multiple initial high-dimensional feature maps of each frame of the image in each scene at each moment; Subsequently, the initial high-dimensional feature map is upsampled multiple times by using an interpolation or proximity search method so that its dimension is consistent with the monitoring area map, thereby obtaining multiple final high-dimensional feature maps for each scene at each moment; Next, the final high-dimensional feature map at the same position at all times is input into the trained low-rank initialization model to obtain the low-rank features at the same position at all times; Then, the monitoring area map and the semantic map of each scene at each moment are subjected to a first feature fusion, and the multiple final high-dimensional feature maps and low-rank features of each scene at each moment are subjected to a second feature fusion, so as to obtain a combined feature of each scene at each moment; Then, the combined features of all positions in each scene are spliced ​​in time order to obtain the combined video stream features of each scene; Finally, the combined video stream features of each scene are input into the trained anomaly detection network model to obtain the anomaly detection results for each scene.

[0012] Preferably, the step of performing a first feature fusion on the monitoring area map and the semantic map of each scene at each moment specifically includes: First, the monitoring area map and the semantic map of each scene at each moment are multiplied to obtain the semantic area map of each scene at each moment; Then, the semantic region map is fuzzy processed by using a mean filter with a preset ratio; Finally, the blurred semantic region map is concatenated with the original semantic map to obtain the features after the first fusion.

[0013] Preferably, the step of performing a second feature fusion of the multiple final high-dimensional feature maps and low-rank features of each scene at each moment specifically includes: First, multiple final high-dimensional feature maps of each scene at each moment are concatenated to obtain a multi-dimensional high-dimensional feature map of each scene at each moment; Then, the multi-dimensional high-dimensional feature map is concatenated with the low-rank features of the same scene at the same time to obtain the features after the second fusion.

[0014] Preferably, the training process of the low-rank initialization model includes: The low-rank initialization model is used to perform low-rank initialization on the high-dimensional feature map of the complete video stream consisting of all high-dimensional feature maps at all times and positions. The specific steps are: Based on the low-rank initialization model and a preset number of iterations, the alternating direction multiplier method is used to perform low-rank initialization on the high-dimensional feature map of the complete video stream, and finally obtain low-rank features of all positions.

[0015] Preferably, the objective function of the low-rank initialization model includes a data fidelity term and a nuclear norm term, wherein the data fidelity term is used to measure the feature similarity before and after the low-rank initialization process.

[0016] A deep learning-based training base security monitoring system for executing the method includes: An image acquisition module is used to acquire images of each scene at each moment; The image processing module includes a monitoring area recognition submodule and a semantic recognition submodule, wherein the monitoring area recognition submodule is used to input the image of each scene at each moment into the monitoring area recognition model to obtain the monitoring area map of each scene at each moment, and the semantic recognition submodule is used to input the image of each scene at each moment into the semantic recognition model to obtain the semantic map of each scene at each moment; Image denoising module, used to denoise each frame of each scene at each moment using median filtering to remove interference noise; The feature extraction module is used to input each frame of the denoised image into a pre-trained deep learning model to obtain an initial high-dimensional feature map, and perform multiple upsampling on it to obtain a final high-dimensional feature map; A low-rank feature extraction module is used to input the final high-dimensional feature map at the same position at all times into the trained low-rank initialization model to obtain the low-rank features at the same position at all times; The feature fusion module is used to perform multi-level fusion of the monitoring area map, semantic map, final high-dimensional feature map and low-rank features to obtain combined features; The video stream feature generation module is used to splice the combined features of all positions in each scene in chronological order to obtain the combined video stream features of each scene; The anomaly detection module is used to input the combined video stream features of each scene into the anomaly detection network model to obtain the anomaly detection results of each scene.

[0017] Preferably, the feature fusion module includes: The first fusion submodule is used to perform point multiplication, fuzzy processing and splicing on the monitoring area map and semantic map of each scene at each moment to obtain the features after the first fusion; The second fusion submodule is used to splice multiple final high-dimensional feature maps of each scene at each moment, and splice them with the low-rank features of the same scene at the same moment to obtain the features after the second fusion.

[0018] Preferably, the low-rank feature extraction module comprises: The low-rank initialization submodule is used to use the low-rank initialization model to perform low-rank initialization on the high-dimensional feature map of the complete video stream consisting of all high-dimensional feature maps at all times and positions, and uses the alternating direction multiplication method to optimize based on a preset number of iterations to ultimately obtain low-rank features for all positions.

[0019] Preferably, it also includes a model training module for training and optimizing the monitoring area recognition model, the semantic recognition model, the pre-trained deep learning model, the low-rank initialization model and the anomaly detection network model.

[0020] Preferably, the anomaly detection module further includes a visualization submodule for presenting the anomaly detection result in a graphical manner and marking the specific location and time of the anomaly occurrence.

[0021] The beneficial effects of the present invention are mainly reflected in the following aspects: The core of the present invention lies in its unique feature extraction and fusion mechanism. First, by introducing the monitoring area map and semantic map, the system can more accurately locate the key focus area and understand the semantic information of objects and people in the scene. Secondly, the low-rank initialization model is used to process high-dimensional features, which effectively removes redundant information and extracts the essential behavior patterns. Furthermore, through multi-level feature fusion, the system can organically combine the information of spatial and temporal dimensions to form a comprehensive and accurate feature representation.

[0022] This innovative technical solution brings about many beneficial effects. In terms of accuracy, the present invention significantly improves the accuracy of anomaly detection and greatly reduces the false alarm rate and missed alarm rate. In particular, the present invention shows obvious advantages in identifying long-term abnormal behavior patterns. In terms of flexibility, the system can adapt to different types of training base scenarios and has good recognition capabilities for newly emerging anomaly types. In terms of efficiency, by effectively removing redundant information, the system can process large-scale video data more quickly and achieve real-time monitoring.

[0023] More importantly, there are synergistic and complementary effects between the various technical features of the present invention. For example, the introduction of the monitoring area map and the semantic map provides more valuable input for low-rank feature learning, and the low-rank features provide more essential time series information for subsequent feature fusion. This multi-level synergy enables the system to comprehensively capture various potential safety hazards when dealing with complex and changeable training base environments.

[0024] In general, the present invention not only solves the problems existing in the prior art, but also achieves a qualitative leap in the accuracy, flexibility and efficiency of anomaly detection, and provides a new solution for intelligent security monitoring of training bases. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 It is a logic block diagram of the whole system of the present invention.

[0026] Figure 2 It is a logic block diagram of the image processing module of the present invention.

[0027] Figure 3 It is a logic block diagram of the feature extraction module of the present invention.

[0028] Figure 4 It is a logic block diagram of the feature fusion module of the present invention.

[0029] Figure 5 It is a logic block diagram of the abnormality detection module of the present invention.

[0030] Figure 6 It is an overall logic block diagram of the method of the present invention. DETAILED DESCRIPTION

[0031] In order to further explain the technical means and effects adopted by the present invention to achieve the predetermined invention purpose, the specific implementation methods, structures, features and effects are described in detail below in conjunction with the accompanying drawings and preferred embodiments. In the following description, different "one embodiment" or "another embodiment" does not necessarily refer to the same embodiment. In addition, specific features, structures or characteristics in one or more embodiments may be combined in any suitable form.

[0032] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.

[0033] See also Figure 1-6 The present invention discloses a training base security monitoring method based on deep learning, comprising the following steps: Obtain a monitoring area map and a semantic map corresponding to the image of each scene at each moment; perform a first process and a second process on each frame of each scene at each moment to obtain multiple high-dimensional feature maps of each scene at each moment, wherein the first process is to input each frame of each scene at each moment into a pre-trained deep learning model to obtain multiple high-dimensional feature maps of each frame of each scene at each moment, and the second process is to perform multiple upsampling on the high-dimensional feature map of each frame of each scene at each moment to obtain multiple high-dimensional feature maps of each scene at each moment, wherein the dimensions of the multiple high-dimensional feature maps are consistent with the monitoring area map; input the high-dimensional feature map at the same position at all moments into a trained low-rank initialization model to obtain low-rank features at the same position at all moments; perform feature fusion on the monitoring area map and the semantic map of each scene at each moment, and perform feature fusion on the multiple high-dimensional feature maps and the low-rank features of each scene at each moment to obtain the combined features of each scene at each moment; splice the combined features of all positions in each scene to obtain the combined video stream features of each scene; input the combined video stream features of each scene into a trained anomaly detection network model to obtain anomaly detection results for each scene.

[0034] In one embodiment of the present invention, first, it is necessary to obtain the monitoring area map and semantic map corresponding to the image of each scene at each moment of the training base. This step usually involves installing high-definition cameras in key areas of the base (such as entrances and exits, training grounds, dormitory areas, etc.) to collect video images in real time. Preferably, these cameras can use 360-degree panoramic cameras to obtain more comprehensive monitoring coverage.

[0035] Next, the system will process each frame of each scene at each moment twice. The first processing is to input the image into a pre-trained deep learning model. In one embodiment of the present invention, this pre-trained deep learning model can be a classic convolutional neural network model such as ResNet50 or DenseNet121. Through this model, we can obtain multiple initial high-dimensional feature maps for each frame of each scene at each moment. The second processing is to upsample these initial high-dimensional feature maps multiple times. The purpose of this step is to make the dimension of the high-dimensional feature map consistent with the monitoring area map, so as to facilitate subsequent feature fusion. Preferably, the upsampling multiple can be set to 2 or 4, which can be determined according to the size difference between the initial high-dimensional feature map and the monitoring area map.

[0036] Then, the system will input the high-dimensional feature map at the same position at all times into the trained low-rank initialization model to obtain the low-rank features at the same position at all times. The purpose of this step is to capture the temporal correlation in the video sequence and remove redundant information. In the monitoring scenario of the training base, this step can effectively extract abnormal behavior patterns on a long time scale, such as staying in a prohibited area for a long time, repeatedly entering and exiting a certain area, etc.

[0037] Next, the system will perform two feature fusions. The first is to fuse the monitoring area map and semantic map of each scene at each moment, and the second is to fuse multiple high-dimensional feature maps and low-rank features of each scene at each moment. Through these two fusions, we obtain the combined features of each scene at each moment, which contain both spatial and temporal information.

[0038] Subsequently, the system concatenates the combined features of all locations in each scene to obtain the combined video stream features of each scene. The purpose of this step is to further integrate spatial and temporal information to form a complete video feature representation.

[0039] Finally, the system will input the combined video stream features of each scene into the trained anomaly detection network model to obtain the anomaly detection results for each scene. In the actual application of the training base, this step can detect various abnormal situations, such as unauthorized entry, abnormal gatherings, and suspicious items left behind.

[0040] In an embodiment of the present invention, obtaining the monitoring area map and semantic map corresponding to the image of each scene at each moment includes: inputting the image of each scene at each moment into a monitoring area recognition model to obtain the monitoring area map of each scene at each moment; inputting the image of each scene at each moment into a semantic recognition model to obtain the semantic map of each scene at each moment.

[0041] In this step, the system first inputs the images of each scene at each moment into the monitoring area recognition model to obtain the monitoring area map of each scene at each moment. Then, the system inputs the same images into the semantic recognition model to obtain the semantic map of each scene at each moment. The separation of these two steps helps to improve the professionalism and accuracy of the model. For example, in the application of a training base, the monitoring area recognition model can accurately locate areas that need to be focused on, such as armories and confidential document storage areas, while the semantic recognition model can accurately identify objects such as people, vehicles, and equipment in the scene.

[0042] In an embodiment of the present invention, before obtaining multiple high-dimensional feature maps of each scene at each moment: a median filtering method is used to reduce noise on each frame of each scene at each moment to remove interference noise.

[0043] Before obtaining the high-dimensional feature map, the system will also use the median filter method to reduce noise for each frame of each scene at each moment to remove interference noise. This step is particularly important in the actual application of the training base, because outdoor scenes may be affected by factors such as weather and lighting, generating various noises. In one embodiment of the present invention, the window size of the median filter can be set to 3x3 or 5x5, which can be adjusted specifically according to the resolution and noise level of the image. Preferably, for 1080p high-definition video, a window size of 5x5 can be selected, which can effectively remove noise while retaining image details.

[0044] In an embodiment of the present invention, the multiple upsampling of the high-dimensional feature map of each frame image of each scene at each moment includes: multiple upsampling of the high-dimensional feature map of each frame image of each scene at each moment using interpolation or proximity search method.

[0045] This step details the upsampling process of the high-dimensional feature map. The system uses interpolation or proximity search to multiply upsample the high-dimensional feature map of each frame of each scene at each moment. In the application of the training base, this step ensures that features of different scales (such as large-scale crowd activities and small-scale individual behaviors) can be accurately captured and represented. Preferably, bilinear interpolation can be used for upsampling because it strikes a good balance between computational efficiency and effect. For 4x upsampling, 2x upsampling can be performed first, and then 2x upsampling can be performed again, which can reduce the distortion caused by interpolation.

[0046] In an embodiment of the present invention, the high-dimensional feature maps at the same position at all times are input into a trained low-rank initialization model to obtain the low-rank features at the same position at all times, including: inputting the high-dimensional feature maps at the same position at all times into a trained low-rank initialization model to obtain the low-rank features at the same position at all times, and the trained low-rank initialization model is obtained by using the low-rank initialization model to perform low-rank initialization on all high-dimensional feature maps of the video stream composed of all high-dimensional feature maps at all positions at all times.

[0047] This step describes the low-rank feature extraction process in detail. The system inputs the high-dimensional feature map at the same position at all times into the trained low-rank initialization model to obtain the low-rank features at the same position at all times. This trained low-rank initialization model is obtained by low-rank initialization of the high-dimensional feature map of the complete video stream consisting of high-dimensional feature maps at all times and all positions. In the monitoring scenario of the training base, this step can effectively capture behavioral patterns over long time scales, such as patrol routes, daily training processes, etc., making it easier to identify abnormal behaviors that deviate from normal patterns.

[0048] Preferably, the low-rank initialization model can adopt the following objective function:

[0049] Among them, X represents the high-dimensional feature map of the input, L represents the low-rank feature, represents the Frobenius norm, represents the nuclear norm, and λ is a balance parameter. λ can be set to a value between 0.1 and 1, and the optimal value can be determined by cross-validation. In the application of the training base, if we pay more attention to long-term behavior patterns, we can set λ to a larger value, such as 0.5 or 0.8; if we pay more attention to short-term abnormal behavior, we can set λ to a smaller value, such as 0.1 or 0.3.

[0050] In an embodiment of the present invention, the feature fusion of the monitoring area map and the semantic map of each scene at each moment includes: performing dot product between the monitoring area map and the semantic map of each scene at each moment to obtain the semantic area map of each scene at each moment; blurring the semantic area map of each scene at each moment using a mean filter with a preset ratio; and splicing the processed semantic area map of each scene at each moment with the semantic map of each scene at each moment.

[0051] This step describes the process of the first feature fusion in detail. First, the system performs a dot product between the monitoring area map and the semantic map for each scene at each moment to obtain the semantic area map for each scene at each moment. Then, the semantic area map is blurred using a mean filter with a preset ratio. Finally, the processed semantic area map and the original semantic map are spliced. In actual applications at the training base, this process can highlight the semantic information of important areas while retaining the details of the original semantic map. For example, it can emphasize the activities of personnel in the armory area while retaining the overall situation of the surrounding area.

[0052] Preferably, the window size of the mean filter can be set to 5x5 or 7x7, which can be adjusted according to the resolution of the image. For 1080p high-definition video, a window size of 7x7 can be selected, which can effectively smooth local details while retaining the main features. The preset ratio can be set to 0.7, which means that the blurred semantic area map accounts for 70% of the weight, while the original semantic map accounts for 30% of the weight. This ratio setting can achieve a good balance between retaining the original information and highlighting important areas.

[0053] In an embodiment of the present invention, the feature fusion of the multiple high-dimensional feature maps and low-rank features of each scene at each moment to obtain the combined features of each scene at each moment includes: splicing the multiple high-dimensional feature maps of each scene at each moment to obtain a multi-dimensional high-dimensional feature map of each scene at each moment; splicing the multi-dimensional high-dimensional feature map of each scene at each moment with the low-rank features of each scene at the same moment to obtain the combined features of each scene at each moment.

[0054] This step describes the process of the second feature fusion. First, the system will splice multiple high-dimensional feature maps of each scene at each moment to obtain a multi-dimensional high-dimensional feature map of each scene at each moment. Then, this multi-dimensional high-dimensional feature map is spliced ​​with the low-rank features of the same scene at the same moment to obtain the combined features of each scene at each moment. In the monitoring of the training base, this step can organically combine the rich spatial semantic information with the correlation of the time series, providing a comprehensive and accurate feature representation for subsequent anomaly detection.

[0055] Preferably, when performing feature splicing, a channel dimension splicing method can be used. For example, if the high-dimensional feature map has 256 channels and the low-rank feature has 64 channels, then the spliced ​​combined feature will have 320 channels. This method can retain all feature information while allowing the subsequent anomaly detection network model to automatically learn how to best utilize these features. In the application of the training base, this combined feature can capture both short-term abnormal behaviors (such as sudden intrusions) and long-term abnormal patterns (such as suspicious patrol routes), thereby improving the comprehensiveness and accuracy of anomaly detection.

[0056] In an embodiment of the present invention, the step of splicing the combined features of all positions in each scene to obtain the combined video stream features of each scene includes: splicing the combined features of all positions in each scene to obtain the combined video stream features of each scene.

[0057] This step further clarifies the process of obtaining the combined video stream features. The system will splice the combined features of all positions in each scene in chronological order to obtain the combined video stream features of each scene. This step is particularly important in the monitoring of training bases because it can capture complex behavior patterns across space and time, such as multi-person collaborative training, long-term equipment use, etc.

[0058] In one embodiment of the present invention, a sliding window method can be used for splicing. For example, a 30-second time window can be set, sliding forward 5 seconds each time. In this way, each combined video stream feature contains 30 seconds of information, and there is a 25-second overlap between adjacent features. This method can effectively capture continuous behavior patterns, while also detecting sudden abnormal events. For the monitoring of the training base, this setting can well balance real-time and comprehensiveness, and can both quickly respond to emergencies and identify potential long-term abnormal behaviors.

[0059] In an embodiment of the present invention, a low-rank initialization model is used to perform low-rank initialization on all high-dimensional feature maps of all video streams consisting of all high-dimensional feature maps at all times and all positions, including: based on the low-rank initialization model and a preset number of iterations, the alternating direction multiplication method is used to perform low-rank initialization on the high-dimensional feature maps of the video stream to obtain low-rank features of all positions.

[0060] This step details the low-rank initialization process. Based on the low-rank initialization model and the preset number of iterations, the system uses the alternating direction multiplier method (ADMM) to perform low-rank initialization on the high-dimensional feature map of the video stream to obtain low-rank features at all positions. In the application of the training base, this step can effectively remove noise and redundant information in the video and extract the essential behavior pattern.

[0061] Preferably, the number of iterations can be set between 100 and 500, which can be adjusted according to the length and complexity of the video. For example, for an 8-hour surveillance video, the number of iterations can be set to 300. If the convergence speed is slow, the number of iterations can be appropriately increased; if the convergence speed is fast, the number of iterations can be appropriately reduced. In practical applications, a convergence threshold can be set. When the difference between the results of two adjacent iterations is less than this threshold, the iteration is stopped. This threshold can be set to 1e-6, which can avoid unnecessary calculations while ensuring accuracy.

[0062] Corresponding to the method, the present invention also discloses a training base security monitoring system based on deep learning. The system includes the following modules: The image acquisition module 1 is used to acquire the image of each scene at each moment. In practical applications, this module can be a monitoring system composed of multiple high-definition cameras.

[0063] The image processing module 2 includes a monitoring area recognition submodule 21 and a semantic recognition submodule 22. The monitoring area recognition submodule 21 is used to input the image of each scene at each moment into the monitoring area recognition model to obtain the monitoring area map of each scene at each moment. The semantic recognition submodule 22 is used to input the image of each scene at each moment into the semantic recognition model to obtain the semantic map of each scene at each moment.

[0064] The image noise reduction module 3 is used to reduce the noise of each frame of each scene at each moment by using the median filtering method to remove the interference noise.

[0065] The feature extraction module 4 is used to input each frame of the denoised image into a pre-trained deep learning model to obtain an initial high-dimensional feature map, and perform multiple upsampling on it to obtain a final high-dimensional feature map.

[0066] The low-rank feature extraction module 5 is used to input the final high-dimensional feature map at the same position at all times into the trained low-rank initialization model to obtain the low-rank features at the same position at all times.

[0067] The feature fusion module 6 is used to perform multi-level fusion of the monitoring area map, semantic map, final high-dimensional feature map and low-rank features to obtain combined features. The module includes a first fusion submodule 61 and a second fusion submodule 62. The first fusion submodule 61 is used to perform dot multiplication, fuzzy processing and splicing on the monitoring area map and semantic map of each scene at each moment to obtain the features after the first fusion. The second fusion submodule 62 is used to splice multiple final high-dimensional feature maps of each scene at each moment, and splice them with the low-rank features of the same scene at the same moment to obtain the features after the second fusion.

[0068] The video stream feature generation module 7 is used to splice the combined features of all positions in each scene in chronological order to obtain the combined video stream features of each scene.

[0069] The anomaly detection module 8 is used to input the combined video stream features of each scene into the anomaly detection network model to obtain the anomaly detection results of each scene. The module also includes a visualization submodule 81, which is used to present the anomaly detection results in a graphical manner and mark the specific location and time of the anomaly.

[0070] In addition, the system also includes a model training module 9, which is used to train and optimize the monitoring area recognition model, semantic recognition model, pre-trained deep learning model, low-rank initialization model and anomaly detection network model. In practical applications, these models can be pre-trained using historical monitoring data from the training base, and then continuously fine-tuned during actual use to meet the needs of specific scenarios.

[0071] The training base security monitoring method and system based on deep learning of the present invention have the following advantages: First, through multi-level feature extraction and fusion, the spatial and temporal information of video data is fully utilized to improve the accuracy of anomaly detection. In particular, low-rank feature extraction is introduced to effectively capture the temporal correlation of video sequences and remove redundant information.

[0072] Secondly, multiple deep learning models are adopted, such as the monitoring area recognition model, the semantic recognition model, the pre-trained deep learning model and the anomaly detection network model, so that the system has strong learning ability and adaptability, and can cope with the complex and changeable training base environment.

[0073] Thirdly, through the design of the feature fusion module, the organic combination of monitoring area information, semantic information, high-dimensional features and low-rank features is achieved, providing rich feature representation for anomaly detection.

[0074] Finally, the modular design of the system allows each processing link to be optimized and upgraded independently, with good scalability and maintainability.

[0075] In practical applications, the method and system of the present invention can effectively detect various abnormal situations in the training base, such as unauthorized entry, abnormal gathering, suspicious items left behind, etc. For example, when the system detects the presence of unauthorized personnel in a restricted area, an alarm can be triggered immediately and the abnormal location can be marked on the visual interface. For another example, when the system finds that the density of people in a certain area is abnormally high, it can remind the management staff to pay attention to possible mass incidents.

[0076] In summary, the present invention provides a training base security monitoring method and system based on deep learning, which realizes efficient and accurate abnormal behavior detection through innovative technologies such as multi-level feature extraction, low-rank feature extraction, and multi-level feature fusion, and provides strong technical support for the security management of the training base.

[0077] It should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A training base security monitoring method based on deep learning, characterized in that: The following steps are involved: First, an image of each scene at each moment is obtained, and the image is sequentially input into a monitoring area recognition model and a semantic recognition model to obtain a monitoring area map and a semantic map of each scene at each moment respectively; Then, each frame of each scene at each moment is subjected to noise reduction processing using the median filter method to remove interference noise; Next, each frame of the image after denoising is input into a pre-trained deep learning model to obtain multiple initial high-dimensional feature maps of each frame of the image in each scene at each moment; Subsequently, the initial high-dimensional feature map is upsampled multiple times by using an interpolation or proximity search method so that its dimension is consistent with the monitoring area map, thereby obtaining multiple final high-dimensional feature maps for each scene at each moment; Next, the final high-dimensional feature map at the same position at all times is input into the trained low-rank initialization model to obtain the low-rank features at the same position at all times; Then, the monitoring area map and the semantic map of each scene at each moment are subjected to a first feature fusion, and the multiple final high-dimensional feature maps and low-rank features of each scene at each moment are subjected to a second feature fusion, so as to obtain a combined feature of each scene at each moment; Then, the combined features of all positions in each scene are spliced ​​in time order to obtain the combined video stream features of each scene; Finally, the combined video stream features of each scene are input into the trained anomaly detection network model to obtain the anomaly detection results for each scene.

2. The method according to claim 1, characterized in that The step of performing the first feature fusion of the monitoring area map and the semantic map of each scene at each moment specifically includes: First, the monitoring area map and the semantic map of each scene at each moment are multiplied to obtain the semantic area map of each scene at each moment; Then, the semantic region map is fuzzy processed by using a mean filter with a preset ratio; Finally, the blurred semantic region map is concatenated with the original semantic map to obtain the features after the first fusion.

3. The method according to claim 1, characterized in that The step of performing a second feature fusion of multiple final high-dimensional feature maps and low-rank features of each scene at each moment specifically includes: First, multiple final high-dimensional feature maps of each scene at each moment are concatenated to obtain a multi-dimensional high-dimensional feature map of each scene at each moment; Then, the multi-dimensional high-dimensional feature map is concatenated with the low-rank features of the same scene at the same time to obtain the features after the second fusion.

4. The method according to claim 1, characterized in that: The training process of the low-rank initialization model includes: The low-rank initialization model is used to perform low-rank initialization on the high-dimensional feature map of the complete video stream consisting of all high-dimensional feature maps at all times and positions. The specific steps are: Based on the low-rank initialization model and a preset number of iterations, the alternating direction multiplier method is used to perform low-rank initialization on the high-dimensional feature map of the complete video stream, and finally obtain low-rank features of all positions.

5. The method according to claim 4, characterized in that The objective function of the low-rank initialization model includes a data fidelity term and a nuclear norm term, wherein the data fidelity term is used to measure the feature similarity before and after the low-rank initialization process.

6. A training base security monitoring system based on deep learning that implements the method according to any one of claims 1 to 5, characterized in that: include: An image acquisition module is used to acquire images of each scene at each moment; The image processing module includes a monitoring area recognition submodule and a semantic recognition submodule, wherein the monitoring area recognition submodule is used to input the image of each scene at each moment into the monitoring area recognition model to obtain the monitoring area map of each scene at each moment, and the semantic recognition submodule is used to input the image of each scene at each moment into the semantic recognition model to obtain the semantic map of each scene at each moment; Image denoising module, used to denoise each frame of each scene at each moment using median filtering to remove interference noise; The feature extraction module is used to input each frame of the denoised image into a pre-trained deep learning model to obtain an initial high-dimensional feature map, and perform multiple upsampling on it to obtain a final high-dimensional feature map; A low-rank feature extraction module is used to input the final high-dimensional feature map at the same position at all times into the trained low-rank initialization model to obtain the low-rank features at the same position at all times; The feature fusion module is used to perform multi-level fusion of the monitoring area map, semantic map, final high-dimensional feature map and low-rank features to obtain combined features; The video stream feature generation module is used to splice the combined features of all positions in each scene in chronological order to obtain the combined video stream features of each scene; The anomaly detection module is used to input the combined video stream features of each scene into the anomaly detection network model to obtain the anomaly detection results of each scene.

7. The system according to claim 6, characterized in that The feature fusion module includes: The first fusion submodule is used to perform point multiplication, fuzzy processing and splicing on the monitoring area map and semantic map of each scene at each moment to obtain the features after the first fusion; The second fusion submodule is used to splice multiple final high-dimensional feature maps of each scene at each moment, and splice them with the low-rank features of the same scene at the same moment to obtain the features after the second fusion.

8. The system according to claim 6, characterized in that The low-rank feature extraction module comprises: The low-rank initialization submodule is used to use the low-rank initialization model to perform low-rank initialization on the high-dimensional feature map of the complete video stream consisting of all high-dimensional feature maps at all times and positions, and uses the alternating direction multiplication method to optimize based on a preset number of iterations to ultimately obtain low-rank features for all positions.

9. The system according to claim 6, characterized in that It also includes a model training module for training and optimizing monitoring area recognition models, semantic recognition models, pre-trained deep learning models, low-rank initialization models, and anomaly detection network models.

10. The system according to claim 6, characterized in that The anomaly detection module also includes a visualization submodule for presenting the anomaly detection result in a graphical manner and marking the specific location and time of the anomaly occurrence.

Images