Real-time detection method and device for abnormal intruder in monitoring scene

Abstract

The application provides a real-time detection method and device for abnormal intruder in a monitoring scene, comprising: acquiring a monitoring video in real time; inputting the monitoring video into a preset abnormal intruder detection model for detection; when it is detected that there is an abnormal intruder in a monitoring image corresponding to the monitoring video, obtaining a monitoring image containing the abnormal intruder output by the abnormal intruder detection model and a category to which the abnormal intruder belongs. The application can monitor in real time whether there is an abnormal intruder in the environment in the monitoring range, which will hinder normal production, affect safety and cause information leakage, so as to timely remind by voice and video bounding box when it is detected that there is an abnormal intruder, so as to ensure the safety of the monitoring area.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a method and apparatus for real-time detection of abnormal intruders in a monitoring scenario. Background Technology

[0002] Safety is a focal point of social concern in modern life, and comprehensive monitoring systems have been established across various regions to ensure the smooth and orderly operation of society. Video surveillance systems are one of the most widely used components. With the widespread adoption of high-definition cameras and the development of mobile networks, video surveillance systems are increasingly being applied to various scenarios in modern production and daily life, especially in sensitive and high-risk areas where video surveillance coverage is extremely high. Thanks to the continuous breakthroughs in computer vision technology, particularly deep learning, in recent years, computers can analyze and process video images under the guidance of algorithmic models, extracting valuable information for judgment and output, understanding the scene like a human, thus replacing human staff in completing some tedious tasks and improving efficiency and accuracy. However, this massive amount of video data also presents significant challenges to real-time intelligent monitoring; conventional methods struggle to achieve high detection and recognition accuracy under real-time processing conditions.

[0003] In the field of security, visual surveillance methods for foreign object intrusion detection can be divided into two types: one is motion target detection algorithms based on background modeling, such as static difference, Gaussian mixture model, and ViBe algorithm. These algorithms require building a background model; pixels that conform to the background model are considered the background and are used as input to update the model. Pixels that do not conform to the model are considered the foreground, i.e., the detected target. Background modeling-based algorithms have the advantages of fast computation speed and good generalization, but their disadvantages include a high false alarm rate and poor detection performance for small, distant targets in complex scenes. The other type is target detection algorithms based on deep learning, such as Faster R-CNN, SSD, and the YOLO series. Deep learning-based target detection algorithms do not rely on information between video frames and can perform target detection on a single image. The advantages of deep learning-based target detection algorithms are high detection accuracy and low false alarm rate, but their disadvantages include poor detection performance for distant and small targets, high computer resource consumption, and low real-time performance. Summary of the Invention

[0004] To address the problems existing in the prior art, embodiments of the present invention provide a method and apparatus for real-time detection of abnormal intruders in a monitoring scenario.

[0005] In a first aspect, embodiments of the present invention provide a method for real-time detection of abnormal intruders in a monitoring scenario, comprising:

[0006] Real-time acquisition of surveillance video;

[0007] The surveillance video is input into a preset abnormal intrusion detection model for detection. When an abnormal intrusion is detected in the surveillance image corresponding to the surveillance video, the abnormal intrusion detection model outputs a surveillance image containing the abnormal intrusion and the category to which the abnormal intrusion belongs. The surveillance image containing the abnormal intrusion carries a bounding box, which is used to frame the detected abnormal object. The preset abnormal intrusion detection model is trained based on a machine learning algorithm, using surveillance video samples as input data, surveillance images containing abnormal intrusions corresponding to the surveillance video samples, and the category to which the abnormal intrusion belongs as output data.

[0008] Furthermore, it also includes:

[0009] Construct an anomalous intruder dataset and label it to determine the category to which the anomalous intruders belong;

[0010] The dataset of anomalous intruders is expanded and enhanced;

[0011] Based on the expanded and enhanced abnormal intruder dataset, a clustering algorithm is used to obtain anchor box sizes that conform to the preset scale characteristics of the target to be detected;

[0012] Based on the clustered anomalous intruder dataset, redundant boxes are suppressed by improving the nonmaximum suppression method.

[0013] The anomalous intruder dataset is trained and detected based on YOLO_v4 and / or YOLO_v5 neural networks.

[0014] Furthermore, it also includes:

[0015] Calculate the distance between the center points of the two anchor point frames;

[0016] Based on the calculation results, redundant frames in the anchor point frame are suppressed.

[0017] Furthermore, the expansion and enhancement of the anomalous intruder dataset specifically includes:

[0018] The abnormal intruder dataset is expanded and enhanced by using transformations such as rotation, brightness, scale, angle, noise, and blurring.

[0019] Secondly, embodiments of the present invention provide a real-time detection device for abnormal intrusions in a monitoring scenario, comprising:

[0020] The acquisition module is used to acquire surveillance video in real time.

[0021] The detection module is used to input the surveillance video into a preset abnormal intrusion detection model for detection. When an abnormal intrusion is detected in the surveillance image corresponding to the surveillance video, the module obtains the surveillance image containing the abnormal intrusion and the category of the abnormal intrusion, output by the abnormal intrusion detection model. The surveillance image containing the abnormal intrusion carries a bounding box, which is used to enclose the detected abnormal object. The preset abnormal intrusion detection model is trained based on a machine learning algorithm, using surveillance video samples as input data, the surveillance image containing the abnormal intrusion corresponding to the surveillance video sample, and the category of the abnormal intrusion as output data.

[0022] Furthermore, it also includes a model building module:

[0023] The model building module is used to build an abnormal intruder dataset and label it to determine the category to which the abnormal intruder belongs;

[0024] The dataset of anomalous intruders is expanded and enhanced;

[0025] Based on the expanded and enhanced abnormal intruder dataset, a clustering algorithm is used to obtain anchor box sizes that conform to the preset scale characteristics of the target to be detected;

[0026] Based on the clustered anomalous intruder dataset, redundant boxes are suppressed by improving the nonmaximum suppression method.

[0027] The anomalous intruder dataset is trained and detected based on YOLO_v4 and / or YOLO_v5 neural networks.

[0028] Furthermore, the model building module is also used for:

[0029] Calculate the distance between the center points of the two anchor point frames;

[0030] Based on the calculation results, redundant frames in the anchor point frame are suppressed.

[0031] Furthermore, when performing the expansion and enhancement of the anomalous intruder dataset, the model building module is specifically used for:

[0032] The abnormal intruder dataset is expanded and enhanced by using transformations such as rotation, brightness, scale, angle, noise, and blurring.

[0033] Thirdly, embodiments of the present invention also provide an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the real-time detection method for abnormal intruders in the monitoring scenario described in the first aspect above.

[0034] Fourthly, embodiments of the present invention also provide a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the real-time detection method for abnormal intruders in the monitoring scenario described in the first aspect above.

[0035] As can be seen from the above technical solution, the real-time intrusion detection method and device in a monitoring scenario provided by the embodiments of the present invention acquires monitoring video in real time; inputs the monitoring video into a preset intrusion detection model for detection; when an intrusion is detected in the monitoring image corresponding to the monitoring video, a monitoring image containing the intrusion, output by the intrusion detection model, and the category to which the intrusion belongs are obtained. The present invention can monitor in real time whether there are intrusions in the environment within the monitoring range that may hinder normal production, affect safety, or cause information leakage, thereby providing timely voice and video annotation box alerts when an intrusion is detected, to ensure the safety of the monitored area. Attached Figure Description

[0036] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0037] Figure 1 This is a flowchart illustrating a real-time detection method for abnormal intruders in a monitoring scenario provided by an embodiment of the present invention.

[0038] Figure 2 This is a schematic diagram of the structure of a real-time detection device for abnormal intrusions in a monitoring scenario provided by an embodiment of the present invention;

[0039] Figure 3 This is a schematic diagram of the physical structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0040] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention. The real-time detection method for abnormal intruders in monitoring scenarios provided by the present invention will be explained and described in detail below through specific embodiments.

[0041] Figure 1 This is a flowchart illustrating a real-time detection method for abnormal intruders in a monitoring scenario according to an embodiment of the present invention; as follows: Figure 1 As shown, the method includes:

[0042] Step 101: Acquire surveillance video in real time.

[0043] Step 102: Input the surveillance video into a preset abnormal intrusion detection model for detection. When an abnormal intrusion is detected in the surveillance image corresponding to the surveillance video, obtain the surveillance image containing the abnormal intrusion and the category of the abnormal intrusion output by the abnormal intrusion detection model. The surveillance image containing the abnormal intrusion carries a bounding box, which is used to frame the detected abnormal object. The preset abnormal intrusion detection model is trained based on a machine learning algorithm, using surveillance video samples as input data, surveillance images containing abnormal intrusions corresponding to the surveillance video samples, and the category of the abnormal intrusion as output data.

[0044] In this embodiment, it should be noted that abnormal intruders, such as drones, vehicles, balloons, kites, and floating objects (such as plastic bags, etc.), may hinder normal production, affect the safety of personnel and machines, and may cause information leakage.

[0045] In this embodiment, regarding the preset abnormal intrusion detection model, it should be noted that the preset abnormal intrusion detection model is used to monitor and issue warnings for foreign objects such as drones, vehicles, balloons, kites, and floating objects in the air. The preset abnormal intrusion detection model is trained based on a machine learning algorithm, using surveillance video samples as input data, surveillance images containing abnormal intrusions corresponding to the surveillance video samples, and the category to which the abnormal intrusions belong as output data. Specifically, regarding the construction of the preset abnormal intrusion detection model, for example, firstly, a corresponding dataset is constructed for foreign objects that may intrude into the scene. The dataset source may include real data from the monitoring scene, or relevant data collected from the Internet through web crawling technology. Data augmentation is used to expand the data to improve its diversity. Then, a clustering algorithm is used to obtain anchor box sizes that conform to the scale characteristics of the target to be detected. Then, non-maximum suppression is improved to suppress redundant boxes. Finally, training and testing are completed on the YOLO_v4 and / or YOLO_v5 network architecture, so that the constructed preset abnormal intrusion detection model can monitor multiple abnormal intrusions simultaneously, maintaining real-time performance while improving detection accuracy.

[0046] As can be seen from the above technical solution, the real-time intrusion detection method in a monitoring scenario provided by this invention acquires monitoring video in real time; inputs the monitoring video into a preset intrusion detection model for detection; when an intrusion is detected in the monitoring image corresponding to the monitoring video, the monitoring image containing the intrusion, output by the intrusion detection model, and the category of the intrusion are obtained. This invention can monitor in real time whether there are intrusions in the environment within the monitoring range that may hinder normal production, affect safety, or cause information leakage, thereby providing timely voice and video annotation box alerts when an intrusion is detected to ensure the safety of the monitored area.

[0047] Based on the above embodiments, in this embodiment, an abnormal intruder dataset is constructed and labeled to determine the category to which the abnormal intruder belongs;

[0048] The dataset of anomalous intruders is expanded and enhanced;

[0049] Based on the expanded and enhanced abnormal intruder dataset, a clustering algorithm is used to obtain anchor box sizes that conform to the preset scale characteristics of the target to be detected;

[0050] Based on the clustered anomalous intruder dataset, redundant boxes are suppressed by improving the nonmaximum suppression method.

[0051] The anomalous intruder dataset is trained and detected based on YOLO_v4 and / or YOLO_v5 neural networks.

[0052] In this embodiment, it can be understood that constructing the abnormal intruder dataset includes constructing a training set and a test set, labeling them, and generating an XML file, wherein the XML file includes the coordinate information of the labeled bounding boxes and their respective categories.

[0053] In this embodiment, it can be understood that the enhancement methods include rotation, brightness, angle, scale transformation, and noise and blur processing transformations, so as to make full use of the limited data (i.e. the original data sample) and perform data augmentation on the original data sample to form enhanced training samples.

[0054] In this embodiment, it should be noted that the clustering algorithm automatically groups similar samples into a category to obtain the anchor box size that conforms to the scale characteristics of the target to be detected.

[0055] In this embodiment, it should be noted that the anchor point box design of the current target scale characteristics is obtained by determining whether there is a target object within the fixed grid and the distance between the predicted box and the actual distance.

[0056] In this embodiment, it should be noted that the backbone network of YOLO_v4 and / or YOLO_v5 adopts the CSPDarknet53 (BottlenneckCSP) network structure to solve the problem of repeated learning of gradient information during network deepening and improve detection speed. An SPP structure is added after the network structure to allow the predicted bounding box to acquire a larger receptive field, effectively separating important feature information from the data. PANet path aggregation uses a combination of upsampling and downsampling; upsampling improves the detection performance of small targets, downsampling strengthens the feature pyramid, and finally, prediction is performed through multi-scale feature layers.

[0057] In this embodiment, it should be noted that, based on the YOLOv4 and / or YOLOv5 network, an abnormal intrusion object dataset conforming to the monitoring scenario is constructed. This dataset can be obtained from real visual surveillance video or from the internet, and data augmentation is performed on it to fully extract feature information from limited data, which can help the model generalize better. Clustering algorithms are used to obtain anchor box sizes that conform to the scale characteristics of the target to be detected, which can improve the detection accuracy of the model. The real-time abnormal intrusion detection method in monitoring scenarios provided by this embodiment of the invention maintains both high accuracy and real-time performance.

[0058] Based on the above embodiments, this embodiment further includes:

[0059] Calculate the distance between the center points of the two anchor point frames;

[0060] Based on the calculation results, redundant frames in the anchor point frame are suppressed.

[0061] In this embodiment, it can be understood that the non-maximum suppression is improved to suppress redundant boxes, taking into account the information of the center point positions of the two boxes. The original IoU (Intersection over Union) is replaced in the non-maximum suppression process, which is more in line with the actual situation.

[0062] In this embodiment, it should be noted that the suppression criterion should consider not only the overlapping area, but also the distance between the center points of the two bounding boxes, thus improving the non-maximum suppression to suppress redundant boxes.

[0063] For example, the improved nonmaximum suppression can better reflect the overlap between two boxes, establish a correlation between factors such as the distance, overlap rate and scale between the predicted box and the ground truth box, directly minimize the distance between their center points. When the predicted box and the ground truth box are adjacent vertically or horizontally, it can optimize the non-intersecting bounding boxes, provide the predicted box with a direction of movement, make its position regression faster and more accurate, and ultimately retain the predicted box with the more accurate position, improve the accuracy of the model in predicting the target position, and make the results obtained by nonmaximum suppression more reasonable.

[0064] Based on the above embodiments, in this embodiment, the expansion and enhancement of the anomalous intruder dataset specifically includes:

[0065] The abnormal intruder dataset is expanded and enhanced by using transformations such as rotation, brightness, scale, angle, noise, and blurring.

[0066] As can be seen from the above technical solution, the real-time detection method for abnormal intruders in monitoring scenarios provided by the embodiments of the present invention is beneficial to make full use of limited data (i.e., original data samples) and to expand the data on the original data samples to form enhanced training samples.

[0067] To better understand the present invention, the following embodiments further illustrate the content of the present invention, but the present invention is not limited to the following embodiments.

[0068] I. Constructing an abnormal intruder detection model:

[0069] This invention, through training on abnormal intruders, obtains training weights for abnormal intruders and constructs an abnormal intruder detection model. The training phase specifically comprises the following steps:

[0070] Step S1: Construct five representative datasets of the autonomous driving environment, namely drones, vehicles, balloons, kites, and floating objects. Divide the dataset into training and testing sets, and then use LabelImg or Labelme to label the data. The generated label file contains the coordinate information of the label box and the category to which the data belongs.

[0071] Step S2: Perform data augmentation on the current dataset. Data augmentation methods include rotation, brightness, scale, angle transformation, and noise and blurring transformations. Data augmentation expands the data, prevents it from stopping learning features due to overfitting, and improves its generalization ability and robustness.

[0072] Step S3: Use a clustering algorithm to obtain the anchor box size that matches the scale characteristics of the target to be detected. The specific calculation steps are as follows:

[0073] Appropriately and randomly select k initial centers of classes;

[0074] In each iteration, for any sample, calculate its Euclidean distance to each of the k centers, and assign the sample to the class containing the center with the shortest distance.

[0075] Update the center values ​​of the k classes using the mean method;

[0076] For all k cluster centers, repeat steps two and three. The iteration ends and the classification is completed when the movement distance of the cluster center values ​​meets a certain condition.

[0077] Step S4: Improve non-maximum suppression to suppress redundant boxes. This step considers the information of the center point positions of the two boxes. Based on the calculated distance between the center points of the two anchor boxes, redundant boxes are suppressed. The specific calculation method is shown in the following formula:

[0078]

[0079] The improved nonmaximum suppression removes box B by simultaneously considering the overlapping region and the distance between the center points of the two boxes. i When the highest-scoring predicted box M and other boxes B i When the difference in the distance between the center points is small, B i Score s i Remain unchanged; otherwise, s i A value of 0 indicates that the element is filtered out. Where s... i R represents the confidence score for each class, IOU is the intersection-union ratio, which indicates the degree of overlap between the predicted bounding box and the ground truth bounding box. DIoU It is the penalty term of the DIoU (normalized distance between two boxes) loss function, where M represents the box with the highest confidence among all predicted boxes, and B... i This represents all the predicted bounding boxes being compared in the current category, and ε represents a manually set threshold, typically 0.5.

[0080] Step S5: Apply the improved algorithm to the YOLOv4 or YOLOv5 network to train the data. In this embodiment of the invention, five types of data were collected. The entire training process is as follows: The input image enters the YOLOv4 network at a resolution of 608×608. After passing through the backbone network CSPDarknet53, it enters the SPP network. The SPP network in YOLO is used to increase the receptive field of the network, realizing max pooling of 5×5, 9×9, and 13×13 on the previous layer. After pooling, Concat (tensor concatenation, which can expand the dimensions of two tensors) is performed to connect them into a feature map. The feature map is then reduced to 512 channels through 1×1. After upsampling by PANet, it is used for the final feature concatenation. Finally, it is output as 76×76×30, 38×38×30, and 19×19×30 by the YOLOv3 Head.

[0081] II. Testing the constructed abnormal intruder detection model:

[0082] In the detection phase of this invention, abnormal intruders are detected by applying trained weights. The specific detection principle is as follows: Taking an input image resolution of 608×608 and a feature scale of 19×19 as an example, the detection process of the YOLOv4 detection model is explained. The model uniformly divides the entire image into a 19×19 network. When the center of a target falls into the network, the network is responsible for detecting the target. Each grid predicts three bounding boxes of different scales. Each bounding box needs to predict the center point coordinates, width, height, confidence score, and the probability that the target belongs to each category. The clustering method described in step S3 of the training phase is used to obtain bounding box sizes that better match the target data. Then, the improved non-maximum suppression method described in step S4 of the training phase is used to remove redundant predictions, resulting in the final target detection box.

[0083] The above principles are used to detect the constructed anomalous intruder model:

[0084] Real-time acquisition of surveillance video;

[0085] The surveillance video is input into a preset abnormal intrusion detection model for detection. For example, if an abnormal intrusion (drone) is detected in the surveillance image corresponding to the surveillance video at a certain moment, the surveillance image containing the abnormal intrusion is output by the abnormal intrusion detection model, and the category of the abnormal intrusion (i.e., the output surveillance image contains the drone in the frame).

[0086] III. Applying the pre-constructed abnormal intruder detection model:

[0087] This invention can detect and identify abnormal intrusions and their relative positions in images within a security monitoring scenario. By comprehensively analyzing the identified results, the system can effectively perceive the external environment, thereby ensuring the security of the monitored area.

[0088] Figure 2 This is a schematic diagram of the structure of a real-time intrusion detection device in a monitoring scenario provided by an embodiment of the present invention, as shown below. Figure 2 As shown, the device includes: an acquisition module 201 and a detection module 202, wherein:

[0089] Among them, the acquisition module 201 is used to acquire monitoring video in real time;

[0090] The detection module 202 is used to input the surveillance video into a preset abnormal intrusion detection model for detection. When an abnormal intrusion is detected in the surveillance image corresponding to the surveillance video, the module obtains the surveillance image containing the abnormal intrusion and the category of the abnormal intrusion, output by the abnormal intrusion detection model. The surveillance image containing the abnormal intrusion carries a bounding box, which is used to frame the detected abnormal object. The preset abnormal intrusion detection model is trained based on a machine learning algorithm, using surveillance video samples as input data, surveillance images containing abnormal intrusions corresponding to the surveillance video samples, and the category of the abnormal intrusion as output data.

[0091] The real-time detection device for abnormal intruders in the monitoring scenario provided in this embodiment of the invention can be used to execute the real-time detection method for abnormal intruders in the monitoring scenario described in the above embodiment. Its technical principle and beneficial effects are similar, and can be found in the above embodiment. It will not be repeated here.

[0092] Based on the same inventive concept, embodiments of the present invention provide an electronic device, see [link to relevant documentation]. Figure 3 The electronic device specifically includes the following components: processor 301, communication interface 303, memory 302, and communication bus 304;

[0093] The processor 301, communication interface 303, and memory 302 communicate with each other via communication bus 304. Communication interface 303 is used to transmit information between various modeling software and intelligent manufacturing equipment module libraries and other related devices. The processor 301 calls the computer program in memory 302. When the processor executes the computer program, it implements the methods provided in the above-mentioned method embodiments. For example, when the processor executes the computer program, it implements the following steps: real-time acquisition of monitoring video; inputting the monitoring video into a preset abnormal intrusion detection model for detection; when an abnormal intrusion is detected in the monitoring image corresponding to the monitoring video, obtaining the monitoring image containing the abnormal intrusion and the category of the abnormal intrusion output by the abnormal intrusion detection model; wherein the monitoring image containing the abnormal intrusion carries a bounding box, which is used to frame the detected abnormal object; the preset abnormal intrusion detection model is trained based on a machine learning algorithm, using monitoring video samples as input data, monitoring images containing abnormal intrusions corresponding to the monitoring video samples, and the category of the abnormal intrusion as output data.

[0094] Based on the same inventive concept, another embodiment of the present invention provides a non-transitory computer-readable storage medium storing a computer program thereon. When executed by a processor, the computer program is implemented to perform the methods provided in the above-described method embodiments, such as: acquiring surveillance video in real time; inputting the surveillance video into a preset abnormal intrusion detection model for detection; when an abnormal intrusion is detected in the surveillance image corresponding to the surveillance video, obtaining a surveillance image containing the abnormal intrusion and the category to which the abnormal intrusion belongs, output by the abnormal intrusion detection model; wherein the surveillance image containing the abnormal intrusion carries a bounding box, the bounding box being used to frame the detected abnormal object; the preset abnormal intrusion detection model is trained based on a machine learning algorithm, using surveillance video samples as input data, surveillance images containing abnormal intrusions corresponding to the surveillance video samples, and the category to which the abnormal intrusion belongs as output data.

[0095] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0096] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., including several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods of various embodiments or some parts of embodiments.

[0097] Furthermore, in this invention, terms such as "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined with "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this invention, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0098] Furthermore, in this invention, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes the element.

[0099] Furthermore, in the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Furthermore, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0100] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for real-time detection of abnormal intruders in a monitoring scenario, characterized in that, include: Real-time acquisition of surveillance video; The surveillance video is input into a preset abnormal intrusion detection model for detection. When an abnormal intrusion is detected in the surveillance image corresponding to the surveillance video, the abnormal intrusion detection model outputs a surveillance image containing the abnormal intrusion and the category of the abnormal intrusion. The surveillance image containing the abnormal intrusion includes a bounding box to enclose the detected abnormal object. The preset abnormal intrusion detection model is trained using a surveillance video sample as input data, a surveillance image containing the abnormal intrusion corresponding to the video sample, and the category of the abnormal intrusion as output data, based on a machine learning algorithm. The category of the abnormal intrusion includes one or more of drones, vehicles, balloons, kites, and floating objects. The method further includes: Construct an anomalous intruder dataset and label it to determine the category to which the anomalous intruders belong; The dataset of anomalous intruders is expanded and enhanced; Based on the expanded and enhanced abnormal intruder dataset, a clustering algorithm is used to obtain anchor box sizes that conform to the preset scale characteristics of the target to be detected; Based on the clustered anomalous intruder dataset, redundant boxes are suppressed by improving the nonmaximum suppression method. The anomalous intruder dataset is trained and detected based on YOLO_v4 and / or YOLO_v5 neural networks; The method further includes: Calculate the distance between the center points of the two anchor point frames; Based on the calculation results, redundant frames in the anchor point frame are suppressed.

2. The real-time detection method for abnormal intruders in a monitoring scenario according to claim 1, characterized in that, The expansion and enhancement of the anomalous intruder dataset specifically includes: The abnormal intruder dataset is expanded and enhanced by transformations involving rotation, brightness, scale, angle, noise, and blurring.

3. A real-time detection device for abnormal intrusions in a monitoring scenario, characterized in that, include: The acquisition module is used to acquire surveillance video in real time. The detection module is used to input the surveillance video into a preset abnormal intrusion detection model for detection. When an abnormal intrusion is detected in the surveillance image corresponding to the surveillance video, the module outputs a surveillance image containing the abnormal intrusion and the category of the abnormal intrusion. The surveillance image containing the abnormal intrusion includes a bounding box to enclose the detected abnormal object. The preset abnormal intrusion detection model is trained using a surveillance video sample as input data, a surveillance image containing the abnormal intrusion corresponding to the video sample, and the category of the abnormal intrusion as output data. The category of the abnormal intrusion includes one or more of drones, vehicles, balloons, kites, and floating objects. The device also includes a model building module: The model building module is used to build an abnormal intruder dataset and label it to determine the category to which the abnormal intruder belongs; The dataset of anomalous intruders is expanded and enhanced; Based on the expanded and enhanced abnormal intruder dataset, a clustering algorithm is used to obtain anchor box sizes that conform to the preset scale characteristics of the target to be detected; Based on the clustered anomalous intruder dataset, redundant boxes are suppressed by improving the nonmaximum suppression method. The anomalous intruder dataset is trained and detected based on YOLO_v4 and / or YOLO_v5 neural networks; The model building module is also used for: Calculate the distance between the center points of the two anchor point frames; Based on the calculation results, redundant frames in the anchor point frame are suppressed.

4. The real-time detection method for abnormal intruders in a monitoring scenario according to claim 3, characterized in that, When the model building module performs expansion and enhancement on the anomalous intruder dataset, it is specifically used for: The abnormal intruder dataset is expanded and enhanced by transformations involving rotation, brightness, scale, angle, noise, and blurring.

5. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the real-time detection method for abnormal intruders in a monitoring scenario as described in any one of claims 1 or 2.

6. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the real-time detection method for abnormal intrusions in a monitoring scenario as described in any one of claims 1 or 2.

Images