Group abnormal behavior detection method and device, electronic equipment and storage medium

Abstract

The invention provides a group abnormal behavior detection method and device, electronic equipment and a storage medium, in the method, an infrared camera carried by an unmanned aerial vehicle is adopted to collect a real-time infrared video stream of a to-be-monitored area, compared with traditional ground fixed monitoring equipment, the method can cover a wider field of view, and the detection accuracy is improved. The method is advantaged in that monitoring comprehensiveness and continuity are improved, a pedestrian detection model can be employed to carry out pedestrian detection on each frame of infrared video frame image of the acquired real-time infrared video stream, a clustering method is employed to carry out clustering on pedestrian coordinate information obtained through pedestrian detection, group abnormal behavior identification is carried out according to a clustering result, and group abnormal behavior identification accuracy is improved. And group abnormal behaviors can be identified timely and effectively, and early warning can be given out.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a detection method, device, electronic equipment and storage medium for abnormal group behavior. Background Art

[0002] Large public places (such as squares, commercial streets, scenic spots, etc.) have always been the focus of inspections by relevant departments, because these areas are prone to abnormal group behaviors that affect public order, such as gatherings of people and fights. However, traditional manual patrol and monitoring methods have many limitations. Patrol officers have limited energy and cannot monitor the entire venue all day and all directions; although surveillance cameras have a wide coverage, they can only passively record the occurrence of events and cannot actively discover and warn of possible disputes. All these make it difficult for relevant departments to discover and deal with such emergencies in a timely manner, posing a hidden danger to social security. Timely discovery and reporting of abnormal group events such as gatherings of people and fights in public places can not only enable relevant departments to take quick response measures to avoid the escalation of the situation, but also help maintain social stability and public safety.

[0003] At present, public places cannot achieve full camera coverage and surveillance mainly due to the following reasons:

[0004] 1. Cost issues. In large public places, such as commercial streets and parks, comprehensive coverage requires a large number of surveillance camera equipment, which is very expensive and often exceeds the budget capacity of the management party;

[0005] 2. Technical limitations. Even if a large amount of money is invested, it is difficult to achieve complete coverage without blind spots. Some special obstructions and dark corners are difficult for cameras to cover. Moreover, the existing monitoring system is also difficult to achieve real-time analysis and processing of all images;

[0006] 3. Difficult maintenance. Large-scale monitoring systems are also a big challenge for daily maintenance and management, requiring a lot of manpower and financial investment. Once a failure or damage occurs, it is difficult to detect and repair it in time.

[0007] Therefore, the monitoring systems in public places are more targeted at crowded or key areas, and there are currently major practical obstacles to achieving full coverage.

[0008] The existing pedestrian monitoring and behavior identification technologies mainly include the following solutions: a) Pedestrian detection based on surveillance cameras in public places. This method of monitoring pedestrian gatherings cannot fully cover public scenes due to its limited monitoring range, so it cannot identify pedestrian gatherings and fights in real time; b) Traditional monitoring systems can only passively record events and cannot promptly detect and warn of abnormal situations such as gatherings of people, fights, etc.; c) There is a lack of intelligent event detection and warning mechanisms, and management departments cannot take timely response measures, making it difficult to effectively maintain social stability and public safety.

[0009] In summary, traditional technologies are unable to conduct comprehensive and effective detection of abnormal group behaviors in a timely manner and lack an early warning mechanism. Summary of the invention

[0010] In view of this, the purpose of the present invention is to provide a method, device, electronic device and storage medium for detecting abnormal group behavior, so as to alleviate the technical problems that traditional technology cannot timely and comprehensively and effectively detect abnormal group behavior and lacks an early warning mechanism.

[0011] In a first aspect, an embodiment of the present invention provides a method for detecting abnormal group behavior, comprising:

[0012] Acquire a real-time infrared video stream of the area to be monitored through an infrared camera carried by the drone, and acquire an infrared video frame image of the real-time infrared video stream;

[0013] Using a pedestrian detection model to perform pedestrian detection on the infrared video frame image to obtain information about pedestrians contained in the infrared video frame image;

[0014] Normalizing the pedestrian coordinate information in the pedestrian information, and clustering the normalized pedestrian coordinate information;

[0015] Determine whether the number of pedestrians in each cluster in the clustering result is greater than a preset number threshold;

[0016] If the number of pedestrians in the target cluster is greater than the preset number threshold, it is determined that the pedestrians in the target cluster have group abnormal behavior, and an early warning is issued to the management department.

[0017] Furthermore, the method further comprises:

[0018] The area corresponding to the target cluster is marked by means of a heat map, and the heat map is sent to the command center system of the management department for display.

[0019] Furthermore, before normalizing the pedestrian coordinate information in the pedestrian information, the method further includes:

[0020] Determining whether the number of pedestrians included in the infrared video frame image is greater than the preset number threshold according to the information of the pedestrians;

[0021] If the number is greater than the preset threshold, normalizing the pedestrian coordinate information in the pedestrian information;

[0022] If it is not greater than the preset number threshold, the next infrared video frame image is obtained, and the next infrared video frame image is used as the infrared video frame image, and the process returns to the step of performing pedestrian detection on the infrared video frame image using a pedestrian detection model.

[0023] Furthermore, the method further comprises:

[0024] If the number of pedestrians not in the target cluster is greater than the preset number threshold, it is determined that there is no abnormal group behavior in the infrared video frame image, and then the next infrared video frame image is obtained, and the next infrared video frame image is used as the infrared video frame image, and the step of performing pedestrian detection on the infrared video frame image using the pedestrian detection model is returned.

[0025] Furthermore, the pedestrian detection model includes: a structure of FPN combined with PAN.

[0026] Furthermore, there are multiple drones.

[0027] In a second aspect, an embodiment of the present invention further provides a detection device for abnormal group behavior, comprising:

[0028] An acquisition unit is used to acquire a real-time infrared video stream of the area to be monitored through an infrared camera carried by the drone, and to acquire an infrared video frame image of the real-time infrared video stream;

[0029] A pedestrian detection unit, configured to perform pedestrian detection on the infrared video frame image using a pedestrian detection model to obtain information about pedestrians contained in the infrared video frame image;

[0030] A clustering unit, used for normalizing the pedestrian coordinate information in the pedestrian information, and clustering the normalized pedestrian coordinate information;

[0031] A judging unit, used to judge whether the number of pedestrians in each cluster in the clustering result is greater than a preset number threshold;

[0032] The determination unit is configured to determine that the pedestrians in the target cluster have abnormal group behavior if the number of pedestrians in the target cluster is greater than the preset number threshold, and issue an early warning to the management department.

[0033] Furthermore, the device is also used for:

[0034] The area corresponding to the target cluster is marked by means of a heat map, and the heat map is sent to the command center system of the management department for display.

[0035] In a third aspect, an embodiment of the present invention further provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of any one of the methods described in the first aspect when executing the computer program.

[0036] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores machine-executable instructions. When the machine-executable instructions are called and executed by a processor, the machine-executable instructions prompt the processor to execute any method described in the first aspect above.

[0037] In an embodiment of the present invention, a method for detecting abnormal group behavior is provided, including: obtaining a real-time infrared video stream of a monitored area through an infrared camera carried by a drone, and obtaining an infrared video frame image of the real-time infrared video stream; performing pedestrian detection on the infrared video frame image using a pedestrian detection model to obtain information about pedestrians contained in the infrared video frame image; normalizing the pedestrian coordinate information in the pedestrian information, and clustering the normalized pedestrian coordinate information; judging whether the number of pedestrians in each cluster in the clustering result is greater than a preset number threshold; if the number of pedestrians in a target cluster is greater than the preset number threshold, it is determined that the pedestrians in the target cluster have abnormal group behavior, and an early warning is issued to the management department. From the above description, it can be seen that in the detection method of abnormal group behavior of the present invention, the infrared camera carried by the drone is used to collect the real-time infrared video stream of the monitored area. Compared with the traditional ground fixed monitoring equipment, it can cover a wider field of view and improve the comprehensiveness and continuity of monitoring. Subsequently, the pedestrian detection model can be used to perform pedestrian detection on each frame of the infrared video frame image of the collected real-time infrared video stream, and then the clustering method is used to cluster the pedestrian coordinate information obtained by the pedestrian detection, and then the abnormal group behavior is identified according to the clustering results. The abnormal group behavior can be identified in a timely and effective manner, and an early warning can be issued, which alleviates the technical problem that the traditional technology cannot perform comprehensive and effective detection of abnormal group behavior in a timely manner and lacks an early warning mechanism. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0039] Figure 1 A flow chart of a method for detecting abnormal group behavior provided by an embodiment of the present invention;

[0040] Figure 2 A schematic diagram of the structure of a pedestrian detection model provided by an embodiment of the present invention;

[0041] Figure 3 A schematic diagram of an FPN structure provided by an embodiment of the present invention;

[0042] Figure 4 A schematic diagram of a PAN structure provided by an embodiment of the present invention;

[0043] Figure 5 A schematic diagram of a detection device for abnormal group behavior provided by an embodiment of the present invention;

[0044] Figure 6 A schematic diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0045] The technical solution of the present invention will be clearly and completely described below in conjunction with the embodiments. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0046] Traditional technologies are unable to conduct comprehensive and effective detection of abnormal group behaviors in a timely manner and lack an early warning mechanism.

[0047] Based on this, in the detection method of abnormal group behavior of the present invention, an infrared camera carried by a drone is used to collect real-time infrared video streams of the area to be monitored. Compared with traditional ground fixed monitoring equipment, it can cover a wider field of view and improve the comprehensiveness and continuity of monitoring. Subsequently, a pedestrian detection model can be used to perform pedestrian detection on each frame of infrared video frame image of the collected real-time infrared video stream, and then a clustering method is used to cluster the pedestrian coordinate information obtained by the pedestrian detection, and then abnormal group behavior is identified based on the clustering results. Abnormal group behavior can be identified in a timely and effective manner, and an early warning can be issued.

[0048] To facilitate understanding of this embodiment, a method for detecting abnormal group behavior disclosed in an embodiment of the present invention is first introduced in detail.

[0049] Embodiment 1:

[0050] According to an embodiment of the present invention, an embodiment of a method for detecting abnormal group behavior is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0051] Figure 1 is a flow chart of a method for detecting abnormal group behavior according to an embodiment of the present invention. Figure 1 As shown, the method comprises the following steps:

[0052] Step S102, obtaining a real-time infrared video stream of the area to be monitored through the infrared camera carried by the drone, and obtaining an infrared video frame image of the real-time infrared video stream;

[0053] In recent years, drone technology has developed rapidly and has been widely used in patrol and other fields. Drones are flexible and have a wide coverage (drones have mobility and a global perspective, and multiple drones can monitor the status of pedestrians in the monitored area in real time), making them very suitable for dynamic monitoring and inspections in public places.

[0054] At the same time, computer vision technologies such as deep learning have made significant breakthroughs in target detection and behavior recognition, and can be applied to the analysis of video data collected by drones to automatically detect abnormal group events such as gatherings of people and fights. This intelligent monitoring system based on drones and computer vision can make up for the shortcomings of traditional monitoring methods and improve the efficiency of relevant departments in discovering and handling emergency events.

[0055] Specifically, the real-time infrared video stream of the monitored area is collected by the infrared camera carried by the drone. One drone can be equipped with multiple infrared cameras, and one monitored area can include multiple drones. The specific size can be determined according to the size of the monitored area to achieve full-area detection of the monitored area.

[0056] After obtaining the real-time infrared video stream of the area to be monitored collected by each infrared camera, each infrared video frame image in each real-time infrared video stream is obtained, which can be obtained by frame extraction.

[0057] Step S104, using a pedestrian detection model to perform pedestrian detection on the infrared video frame image to obtain information about pedestrians contained in the infrared video frame image;

[0058] Specifically, the pedestrian detection model is used to detect pedestrians in infrared video frames, and a set PERSON containing pedestrian target coordinate information is obtained, that is, the information of pedestrians contained in the infrared video frame. Here, only one infrared video frame is used as an example for explanation, and the group abnormal behavior detection process is performed for each infrared video frame.

[0059] It should be noted that in order to improve the accuracy of detection, when using an infrared camera to collect real-time infrared video streams, a visible light camera is also used to collect real-time video streams. The collected real-time video streams are framed to obtain video frame images. Face detection is performed on the video frame images through a face detection model. After determining that it is a person, a pedestrian detection model is used to perform pedestrian detection on the infrared video frame images.

[0060] The pedestrian detection model includes: the structure of FPN combined with PAN.

[0061] The specific structure of the pedestrian detection model is as follows Figure 2 As shown, in Figure 2 In the present invention, the detection network can be divided into four parts: Backbone (backbone network), Neck (neck feature fusion structure), and Head (detection head structure). The backbone network of YOLOv5 is the basis of the entire pedestrian detection model, which is responsible for extracting useful features from the input image. The backbone network of YOLOv5 adopts the CSP-Darknet53 architecture. During the network implementation process, the Mish activation function is adopted, which is a smoother activation function than the traditional ReLU activation function, which helps to provide a more stable gradient flow during the training process. The backbone network of YOLOv5 enables the model to detect targets of different sizes (such as pedestrians) at the same time through multi-scale feature fusion. The detection efficiency of the target is guaranteed for the present invention.

[0062] The Neck structure is the middle part connecting the Backbone and the Head, and is responsible for further fusion and transmission of features. The neck feature fusion module includes the FPN and PAN structures. The specific implementation details of the present invention are as follows: Figure 3 , Figure 4 As shown in the figure, through the FPN+PAN structure, the improved YOLOV5 model of the present invention can transmit high-level semantic information through a top-down path, and then transmit low-level detail information through a bottom-up path. It can effectively improve the detection efficiency of pedestrian targets in the picture.

[0063] Step S106, normalizing the pedestrian coordinate information in the pedestrian information, and clustering the normalized pedestrian coordinate information;

[0064] Specifically, we can first determine whether the set PERSON is an empty set. If it is an empty set, return to the next infrared video frame image and execute the step of using the pedestrian detection model to perform pedestrian detection on the infrared video frame image; if it is not an empty set, then the coordinate information in the set PERSON is normalized, that is, the pedestrian coordinate information in the pedestrian information is normalized, and the normalized pedestrian coordinate information is clustered.

[0065] Step S108, determining whether the number of pedestrians in each cluster in the clustering result is greater than a preset number threshold;

[0066] Specifically, the preset number threshold may be 7, and the embodiment of the present invention does not specifically limit the preset number threshold.

[0067] Step S110: If the number of pedestrians in the target cluster is greater than a preset number threshold, it is determined that the pedestrians in the target cluster have group abnormal behavior, and an early warning is issued to the management department.

[0068] In an embodiment of the present invention, a method for detecting abnormal group behavior is provided, including: obtaining a real-time infrared video stream of a monitored area through an infrared camera carried by a drone, and obtaining an infrared video frame image of the real-time infrared video stream; performing pedestrian detection on the infrared video frame image using a pedestrian detection model to obtain information about pedestrians contained in the infrared video frame image; normalizing the pedestrian coordinate information in the pedestrian information, and clustering the normalized pedestrian coordinate information; judging whether the number of pedestrians in each cluster in the clustering result is greater than a preset number threshold; if the number of pedestrians in a target cluster is greater than the preset number threshold, it is determined that the pedestrians in the target cluster have abnormal group behavior, and an early warning is issued to the management department. From the above description, it can be seen that in the detection method of abnormal group behavior of the present invention, the infrared camera carried by the drone is used to collect the real-time infrared video stream of the monitored area. Compared with the traditional ground fixed monitoring equipment, it can cover a wider field of view and improve the comprehensiveness and continuity of monitoring. Subsequently, the pedestrian detection model can be used to perform pedestrian detection on each frame of the infrared video frame image of the collected real-time infrared video stream, and then the clustering method is used to cluster the pedestrian coordinate information obtained by the pedestrian detection, and then the abnormal group behavior is identified according to the clustering results. The abnormal group behavior can be identified in a timely and effective manner, and an early warning can be issued, which alleviates the technical problem that the traditional technology cannot perform comprehensive and effective detection of abnormal group behavior in a timely manner and lacks an early warning mechanism.

[0069] The above content briefly introduces the detection method of abnormal group behavior of the present invention. The specific contents involved are described in detail below.

[0070] In an optional embodiment of the present invention, the method further comprises the following steps:

[0071] The area corresponding to the target cluster is marked by means of a heat map, and the heat map is sent to the command center system of the management department for display.

[0072] Specifically, the area corresponding to the group's abnormal behavior is marked on the heat map, and then the heat map is sent to the command center system of the management department for display. Relevant personnel can intuitively present the location of the incident, providing an important reference for subsequent handling. This visual method can better guide the emergency response of relevant departments.

[0073] In an optional embodiment of the present invention, before normalizing the pedestrian coordinate information in the pedestrian information, the method further includes the following steps:

[0074] (1) determining whether the number of pedestrians included in the infrared video frame image is greater than a preset number threshold according to the information of the pedestrians;

[0075] (2) If the number is greater than a preset threshold, normalizing the pedestrian coordinate information in the pedestrian information;

[0076] (3) If the number is not greater than the preset threshold, the next infrared video frame image is obtained, and the next infrared video frame image is used as the infrared video frame image, and the step of performing pedestrian detection on the infrared video frame image using the pedestrian detection model is returned.

[0077] Specifically, in order to avoid the subsequent normalization processing and clustering calculation amount, before the pedestrian coordinate information in the pedestrian information is normalized, it is first determined based on the pedestrian information whether the number of pedestrians contained in the infrared video frame image is greater than the preset number threshold. If not, there is no need to normalize the pedestrian coordinate information of the pedestrian information contained in the infrared video frame image. In other words, it is directly determined that there is no abnormal group behavior in the infrared video frame image, which greatly reduces the subsequent calculation amount and improves the detection efficiency.

[0078] In an optional embodiment of the present invention, the method further comprises the following steps:

[0079] If the number of pedestrians without target clusters is greater than the preset threshold, it is determined that there is no abnormal group behavior in the infrared video frame image, and then the next infrared video frame image is obtained, and the next infrared video frame image is used as the infrared video frame image, and the step of performing pedestrian detection on the infrared video frame image using the pedestrian detection model is returned.

[0080] In an optional embodiment of the present invention, there are multiple drones.

[0081] Specifically, each drone works according to the remote deployment and control of the management personnel in the command center system, and the management personnel can view the monitoring screen in real time through the PC or mobile device.

[0082] The method for detecting abnormal group behavior of the present invention has the following advantages:

[0083] (1) UAVs are highly maneuverable and have a wide coverage area. Using flexible UAVs for all-weather patrols in public places can quickly cover a large area, making up for the limitations of manual patrols and fixed surveillance cameras.

[0084] (2) Using advanced computer vision technologies such as deep learning to automatically analyze real-time video data collected by drones, it can accurately detect and identify abnormal group behaviors such as gatherings of people and fighting;

[0085] (3) Intelligent event warning. The system can monitor the dynamic situation of public places in real time. Once abnormal group behavior such as gathering of people, fighting, etc. is discovered, it will immediately issue a warning to the management department to provide timely information support for taking countermeasures;

[0086] (4) It can realize remote linkage with the command center system of the management department. Managers can view the monitoring screen in real time through PC or mobile device, and remotely deploy and control the drone;

[0087] (5) Reduce labor costs and risks. Compared with traditional manual patrols, drone monitoring can significantly reduce manpower input and also reduce safety risks for law enforcement personnel.

[0088] Therefore, the technical solution proposed in the present invention can overcome the limitations of existing group abnormal behavior detection solutions, realize comprehensive and real-time monitoring of the monitored area, provide timely and effective support for relevant departments, and has strong technical advantages and application value.

[0089] The present invention has the following effects:

[0090] The present invention adopts the method of real-time video monitoring by unmanned aerial vehicles, which can cover a wider field of view than traditional ground fixed monitoring equipment, and improve the comprehensiveness and continuity of monitoring. Unmanned aerial vehicles can be quickly deployed to areas that need to be monitored, making up for the inflexible layout of ground monitoring equipment. Traditional public place monitoring mainly relies on fixed cameras. Once abnormal group behavior occurs, it needs to be discovered and alarmed manually, and the response speed is slow. The present invention adopts the method of real-time video analysis, which can capture the occurrence of abnormal group behavior in time, shorten the time interval from event discovery to alarm, and improve the timeliness of safety accident prevention. Through video image analysis, the present invention can mark the specific area where the abnormal group behavior occurs on the heat map, intuitively present the location of the event, and provide an important reference for subsequent disposal. Compared with a single alarm prompt, this visualization method can better guide the emergency response of the accident handling department. At present, public places mostly rely on manual inspection and monitoring, which has high work intensity and low efficiency. The present invention realizes automatic detection and early warning through video analysis, which can greatly reduce the pressure of manual inspection and improve the efficiency and accuracy of monitoring work.

[0091] The key technical points of the present invention include:

[0092] (1) Pedestrian recognition based on target detection: The invention uses a pedestrian detection model to detect pedestrian targets in each frame of video image and record their coordinate information to form a PERSON set. This pedestrian target detection link is the basis for subsequent clustering analysis. The key lies in the ability to accurately and real-time extract pedestrian target information from the image.

[0093] (2) The present invention improves the feature fusion module based on the yolov5 algorithm. Since the detection of small targets is easily lost when the drone is flying at high altitude, a method combining FPN and PAN structure is improved to improve the efficiency of small target detection. It is applied to the task of identifying abnormal group behavior in the present invention.

[0094] (3) Crowd gathering detection based on cluster analysis. The present invention uses a clustering algorithm to group pedestrian targets in the PERSON set, counts the number of samples in each cluster, and determines that there is a crowd gathering phenomenon (i.e., abnormal group behavior) when it exceeds a preset number threshold. This cluster analysis link is the core of discovering crowd gatherings, and the key lies in being able to effectively identify crowd gathering areas.

[0095] (4) Visualization presentation technology based on heat map. The present invention marks the specific area where the gathering of people occurs through heat map, and intuitively presents the location of the event. This visualization method can better guide the subsequent emergency response work, which is a highlight of the present invention.

[0096] Embodiment 2:

[0097] The embodiment of the present invention further provides a detection device for abnormal group behavior. The detection device for abnormal group behavior is mainly used to execute the detection method for abnormal group behavior provided in the first embodiment of the present invention. The following is a specific introduction to the detection device for abnormal group behavior provided in the embodiment of the present invention.

[0098] Figure 5 is a schematic diagram of a detection device for abnormal group behavior according to an embodiment of the present invention, such as Figure 5 As shown, the device includes: an acquisition unit 10, a pedestrian detection unit 20, a clustering unit 30, a judgment unit 40 and a determination unit 50, wherein:

[0099] An acquisition unit is used to acquire a real-time infrared video stream of the area to be monitored through an infrared camera carried by the drone, and to acquire an infrared video frame image of the real-time infrared video stream;

[0100] A pedestrian detection unit, used to perform pedestrian detection on the infrared video frame image using a pedestrian detection model to obtain information about pedestrians contained in the infrared video frame image;

[0101] A clustering unit, used for normalizing the pedestrian coordinate information in the pedestrian information, and clustering the normalized pedestrian coordinate information;

[0102] A judging unit, used to judge whether the number of pedestrians in each cluster in the clustering result is greater than a preset number threshold;

[0103] The determination unit is used to determine that the pedestrians in the target cluster have abnormal group behavior if the number of pedestrians in the target cluster is greater than a preset number threshold, and issue an early warning to the management department.

[0104] In an embodiment of the present invention, a detection device for abnormal group behavior is provided, including: obtaining a real-time infrared video stream of a monitored area through an infrared camera carried by a drone, and obtaining an infrared video frame image of the real-time infrared video stream; performing pedestrian detection on the infrared video frame image using a pedestrian detection model to obtain information about pedestrians contained in the infrared video frame image; normalizing the pedestrian coordinate information in the pedestrian information, and clustering the normalized pedestrian coordinate information; judging whether the number of pedestrians in each cluster in the clustering result is greater than a preset number threshold; if the number of pedestrians in a target cluster is greater than the preset number threshold, it is determined that the pedestrians in the target cluster have abnormal group behavior, and an early warning is issued to the management department. From the above description, it can be seen that in the detection device for abnormal group behavior of the present invention, the infrared camera carried by the drone is used to collect the real-time infrared video stream of the monitored area. Compared with the traditional ground fixed monitoring equipment, it can cover a wider field of view and improve the comprehensiveness and continuity of monitoring. Subsequently, the pedestrian detection model can be used to perform pedestrian detection on each frame of the infrared video frame image of the collected real-time infrared video stream, and then the clustering method is used to cluster the pedestrian coordinate information obtained by the pedestrian detection, and then the abnormal group behavior is identified according to the clustering results. The abnormal group behavior can be identified in a timely and effective manner, and an early warning can be issued, which alleviates the technical problem that the traditional technology cannot perform comprehensive and effective detection of abnormal group behavior in a timely manner and lacks an early warning mechanism.

[0105] Optionally, the device is also used to: mark the area corresponding to the target cluster by means of a heat map, and send the heat map to the command center system of the management department for display.

[0106] Optionally, the device is also used to: determine whether the number of pedestrians contained in the infrared video frame image is greater than a preset number threshold based on the pedestrian information; if it is greater than the preset number threshold, normalize the pedestrian coordinate information in the pedestrian information; if it is not greater than the preset number threshold, obtain the next infrared video frame image, and use the next infrared video frame image as the infrared video frame image, and return to execute the step of performing pedestrian detection on the infrared video frame image using the pedestrian detection model.

[0107] Optionally, the device is also used to: if the number of pedestrians not in the target cluster is greater than a preset threshold, determine that there is no abnormal group behavior in the infrared video frame image, and then obtain the next infrared video frame image, and use the next infrared video frame image as the infrared video frame image, and return to execute the step of performing pedestrian detection on the infrared video frame image using a pedestrian detection model.

[0108] Optionally, the pedestrian detection model includes: a structure of FPN combined with PAN.

[0109] Optionally, there are multiple drones.

[0110] The device provided in the embodiment of the present invention has the same implementation principle and technical effects as those of the aforementioned method embodiment. For the sake of brief description, for matters not mentioned in the device embodiment, reference may be made to the corresponding contents in the aforementioned method embodiment.

[0111] like Figure 6 As shown, an electronic device 600 provided in an embodiment of the present application includes: a processor 601, a memory 602 and a bus, wherein the memory 602 stores machine-readable instructions executable by the processor 601. When the electronic device is running, the processor 601 communicates with the memory 602 through the bus, and the processor 601 executes the machine-readable instructions to perform the steps of the above-mentioned method for detecting abnormal group behavior.

[0112] Specifically, the above-mentioned memory 602 and processor 601 can be general-purpose memory and processor, which are not specifically limited here. When the processor 601 runs the computer program stored in the memory 602, the above-mentioned group abnormal behavior detection method can be executed.

[0113] The processor 601 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit or software instructions in the processor 601. The above processor 601 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the embodiments of the present application can be directly embodied as a hardware decoding processor for execution, or a combination of hardware and software modules in the decoding processor for execution. The software module may be located in a storage medium mature in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory 602, and the processor 601 reads the information in the memory 602 and completes the steps of the above method in combination with its hardware.

[0114] Corresponding to the above-mentioned method for detecting abnormal group behavior, an embodiment of the present application also provides a computer-readable storage medium, which stores machine-executable instructions. When the computer-executable instructions are called and executed by a processor, the computer-executable instructions prompt the processor to execute the steps of the above-mentioned method for detecting abnormal group behavior.

[0115] The detection device for abnormal group behavior provided in the embodiment of the present application can be specific hardware on the device or software or firmware installed on the device. The device provided in the embodiment of the present application, its implementation principle and the technical effect produced are the same as those in the aforementioned method embodiment. For the sake of brief description, the parts not mentioned in the device embodiment can refer to the corresponding contents in the aforementioned method embodiment. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working processes of the systems, devices and units described above can all refer to the corresponding processes in the aforementioned method embodiment, and will not be repeated here.

[0116] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0117] For another example, the flowchart and block diagram in the accompanying drawings show the possible architecture, function and operation of the device, method and computer program product according to multiple embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of the code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or the flowchart, and the combination of the boxes in the block diagram and / or the flowchart can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.

[0118] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0119] In addition, each functional unit in the embodiments provided in the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0120] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can essentially or partly contribute to the prior art or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including a number of instructions for an electronic device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the vehicle marking method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, and other media that can store program codes.

[0121] It should be noted that similar numbers and letters represent similar items in the following figures. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. In addition, the terms "first", "second", "third", etc. are only used to distinguish the description and are not to be understood as indicating or implying relative importance.

[0122] Finally, it should be noted that the above-described embodiments are only specific implementation methods of the present application, which are used to illustrate the technical solution of the present application, rather than to limit it. The protection scope of the present application is not limited thereto. Although the present application is described in detail with reference to the aforementioned embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solution recorded in the aforementioned embodiments within the technical scope disclosed in the present application, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solution deviate from the scope of the technical solution of the embodiment of the present application. They should all be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A method for detecting abnormal group behavior, characterized in that: include: Acquire a real-time infrared video stream of the area to be monitored through an infrared camera carried by the drone, and acquire an infrared video frame image of the real-time infrared video stream; Using a pedestrian detection model to perform pedestrian detection on the infrared video frame image to obtain information about pedestrians contained in the infrared video frame image; Normalizing the pedestrian coordinate information in the pedestrian information, and clustering the normalized pedestrian coordinate information; Determine whether the number of pedestrians in each cluster in the clustering result is greater than a preset number threshold; If the number of pedestrians in the target cluster is greater than the preset number threshold, it is determined that the pedestrians in the target cluster have group abnormal behavior, and an early warning is issued to the management department.

2. The method according to claim 1, characterized in that The method further comprises: The area corresponding to the target cluster is marked by means of a heat map, and the heat map is sent to the command center system of the management department for display.

3. The method according to claim 1, characterized in that Before normalizing the pedestrian coordinate information in the pedestrian information, the method further includes: Determining whether the number of pedestrians included in the infrared video frame image is greater than the preset number threshold according to the information of the pedestrians; If the number is greater than the preset threshold, normalizing the pedestrian coordinate information in the pedestrian information; If it is not greater than the preset number threshold, the next infrared video frame image is obtained, and the next infrared video frame image is used as the infrared video frame image, and the process returns to the step of performing pedestrian detection on the infrared video frame image using a pedestrian detection model.

4. The method according to claim 1, characterized in that: The method further comprises: If the number of pedestrians not in the target cluster is greater than the preset number threshold, it is determined that there is no abnormal group behavior in the infrared video frame image, and then the next infrared video frame image is obtained, and the next infrared video frame image is used as the infrared video frame image, and the step of performing pedestrian detection on the infrared video frame image using the pedestrian detection model is returned.

5. The method according to claim 1, characterized in that The pedestrian detection model includes: a structure of FPN combined with PAN.

6. The method according to claim 1, characterized in that The number of the drones is multiple.

7. A detection device for abnormal group behavior, characterized in that: include: An acquisition unit is used to acquire a real-time infrared video stream of the area to be monitored through an infrared camera carried by the drone, and to acquire an infrared video frame image of the real-time infrared video stream; A pedestrian detection unit, configured to perform pedestrian detection on the infrared video frame image using a pedestrian detection model to obtain information about pedestrians contained in the infrared video frame image; A clustering unit, used for normalizing the pedestrian coordinate information in the pedestrian information, and clustering the normalized pedestrian coordinate information; A judging unit, used to judge whether the number of pedestrians in each cluster in the clustering result is greater than a preset number threshold; The determination unit is configured to determine that the pedestrians in the target cluster have abnormal group behavior if the number of pedestrians in the target cluster is greater than the preset number threshold, and issue an early warning to the management department.

8. The device according to claim 7, characterized in that The device is also used for: The area corresponding to the target cluster is marked by means of a heat map, and the heat map is sent to the command center system of the management department for display.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores machine-executable instructions. When the machine-executable instructions are called and executed by a processor, the machine-executable instructions cause the processor to execute the method according to any one of claims 1 to 6.

Images