Abnormal behavior detection method, device and equipment based on swarm intelligence

Abstract

The present application discloses a method for detecting abnormal behavior based on swarm intelligence, comprising: obtaining a video of a group activity; performing a behavior analysis on the group activity video based on a group behavior analysis model of a swarm intelligence algorithm to determine whether the group activity in the group activity video may have abnormal behavior; and performing image recognition on the video frames in the group activity video based on an image recognition model to determine whether the video frames may contain abnormal behavior related objects; if the above results are all yes, then outputting a video detection result as abnormal. The above method is used to solve the problem that the prior art easily misdetects abnormal behavior of a crowd.

Description

Technical Field

[0001] The present application relates to the field of computer vision technology, and specifically to a method for detecting abnormal behavior based on swarm intelligence, an abnormal behavior detection device based on swarm intelligence, an electronic device, and a storage device. Background Art

[0002] In today's urban life, the density of people in public places is high, which easily causes safety problems. In this regard, relevant personnel must obtain relevant information and respond in a timely manner.

[0003] Under the existing technology, the detection algorithm based on the target detection model can generally only identify the targets in the scene individually, and make judgments by setting a threshold for the length of time the number of people stay.

[0004] There are some defects in the abnormal behavior-related object detection algorithm under the existing technology: the ability to judge the relationship between people and abnormal behavior-related objects is lacking, and the abnormal behavior-related objects are easily confused with other objects in the environment; for example, for construction road occupation, construction tools such as shovels and hoes may not be seen due to visual occlusion and other reasons, and related construction signs, engineering vehicles, etc. are easily confused with other objects in road traffic. The above situations are likely to cause false detection or missed detection.

[0005] In summary, the existing technology is prone to false alarms for abnormal crowd behavior. Summary of the invention

[0006] The present application provides a method, device, electronic device and storage device for detecting abnormal behavior based on swarm intelligence to solve the problem that the prior art easily misdetects abnormal behavior of a crowd.

[0007] The present application provides a method for detecting abnormal behavior based on swarm intelligence, comprising:

[0008] Get videos of group activities;

[0009] Based on a group behavior analysis model of a group intelligence algorithm, a behavior analysis is performed on the group activity video to determine whether the group activity in the group activity video may contain abnormal behavior; and

[0010] Based on the image recognition model, image recognition is performed on the video frames in the group activity video to determine whether the video frames may contain objects related to abnormal behavior;

[0011] If all the above results are yes, the output video detection result is abnormal.

[0012] As an implementation mode, the group behavior analysis model includes:

[0013] A group behavior simulation model, used to obtain a trajectory of group activities based on the group activity video; the trajectory of group activities includes an estimate of the trajectory of group activities in the next time period;

[0014] The classification model is used to judge whether there may be abnormal behavior in the group activity according to the feature vector composed of the parameters of the group behavior simulation model.

[0015] As an implementation mode, the group behavior simulation model is trained and obtained by the following method:

[0016] Get videos of group activities;

[0017] Extracting structured data of group activities according to the group activity video data;

[0018] Accumulating structured data of the group activity for a predetermined time threshold;

[0019] providing the structured data of the group activity accumulated to a predetermined time threshold as training data to an initial group behavior simulation model to train the group behavior simulation model;

[0020] The trained group behavior simulation model is used as the current group behavior simulation model.

[0021] As an implementation mode, the classification model is obtained in the following manner:

[0022] Collect videos of group activities that meet the quantity requirements and mark whether there are any abnormal behaviors in them;

[0023] Corresponding to each group activity video, the corresponding group behavior simulation model is obtained, and the parameters therein are extracted to form a feature vector;

[0024] Providing the feature vector to an initial classification model, and training the initial classification model in combination with the annotation of whether there is abnormal behavior;

[0025] After the training of the classification model reaches a predetermined standard, the trained classification model is used in the group behavior analysis model.

[0026] As an implementation, the output of the classification model includes at least one of the following two types:

[0027] The judgment result of the existence or non-existence of abnormal behavior in group activities, and the corresponding confidence level;

[0028] The judgment result of whether abnormal behavior exists or does not exist in group activities.

[0029] As an implementation manner, extracting structured data of group activities according to the group activity video includes:

[0030] Pre-establish a planar scene graph of the area;

[0031] Performing target recognition on video frames of the group activity video to obtain active individuals therein;

[0032] According to each video frame of the group activity video and the position of the camera device that obtains the video frame, the position of each active individual is marked in the plane scene graph, and a structured position parameter is formed and stored in a simulation queue to form simulation queue structured data;

[0033] The structured data of the group activity is extracted based on the simulated queue structured data accumulated for a predetermined time length.

[0034] As an implementation mode, the image recognition model is used to perform image recognition on the images in the group activity video, and in the step of determining whether the images contain abnormal behavior-related objects, if it is determined that the images may contain abnormal behavior-related objects, then the possible abnormal behavior-related objects are identified; and the group behavior analysis model based on the group intelligence algorithm performs behavior analysis on the group activity video, and in the step of determining whether the group activity in the group activity video may contain abnormal behavior, if it is determined that there is no abnormal behavior, then the identified possible abnormal behavior-related objects are filtered out.

[0035] As an implementation manner, the performing of image recognition on the video frames in the group activity video based on the image recognition model to determine whether the video frames may contain abnormal behavior related objects includes:

[0036] Performing target recognition on the current video frame image of the group activity video;

[0037] Providing the identified target object to a pre-trained abnormal behavior related object detection model to perform abnormal behavior related object identification;

[0038] If possible abnormal behavior correlates are identified and the likelihood exceeds a specified threshold, it is determined that the abnormal behavior correlates may be included.

[0039] As an implementation method, it also includes: displaying the plane scene diagram on a screen; and marking the positions of each of the active individuals in the plane scene diagram on the screen.

[0040] The present application also provides an abnormal behavior detection device based on swarm intelligence, comprising:

[0041] A video acquisition unit, used to acquire group activity videos;

[0042] an abnormal behavior determination unit, configured to perform behavior analysis on the group activity video based on a group behavior analysis model of a group intelligence algorithm, and determine whether the group activity in the group activity video may contain abnormal behavior;

[0043] an abnormal behavior-related object determination unit, configured to perform image recognition on video frames in the group activity video based on an image recognition model to determine whether the video frames may contain abnormal behavior-related objects;

[0044] The detection result output unit is used to output the video detection result as abnormal when the output results of the abnormal behavior determination unit and the abnormal behavior related object determination unit are both yes.

[0045] The present application also provides an electronic device, including:

[0046] Processor; and

[0047] The memory is used to store a program of the abnormal behavior detection method based on swarm intelligence. After the device is powered on and the program of the abnormal behavior detection method based on swarm intelligence is run by the processor, the following steps are performed:

[0048] Get videos of group activities;

[0049] Based on a group behavior analysis model of a group intelligence algorithm, a behavior analysis is performed on the group activity video to determine whether the group activity in the group activity video may contain abnormal behavior; and

[0050] Based on the image recognition model, image recognition is performed on the video frames in the group activity video to determine whether the video frames may contain objects related to abnormal behavior;

[0051] If all the above results are yes, the output video detection result is abnormal.

[0052] The present application also provides a storage device storing a program of an abnormal behavior detection method based on swarm intelligence, wherein the program is executed by a processor to perform the following steps:

[0053] Get videos of group activities;

[0054] Based on a group behavior analysis model of a group intelligence algorithm, a behavior analysis is performed on the group activity video to determine whether the group activity in the group activity video may contain abnormal behavior; and

[0055] Based on the image recognition model, image recognition is performed on the video frames in the group activity video to determine whether the video frames may contain objects related to abnormal behavior;

[0056] If all the above results are yes, the output video detection result is abnormal.

[0057] Compared with the prior art, the abnormal behavior detection method based on swarm intelligence provided by the present invention includes: obtaining a group activity video; performing a behavior analysis on the group activity video based on a group behavior analysis model of a swarm intelligence algorithm to determine whether the group activity in the group activity video may have abnormal behavior; and, based on an image recognition model, performing image recognition on the video frames in the group activity video to determine whether it may contain abnormal behavior related objects; if the above results are all yes, then the output video detection result is abnormal. The present application performs image recognition on the images in the group activity video to determine whether it may contain abnormal behavior related objects, and at the same time introduces a swarm intelligence-based behavior analysis model to perform swarm intelligence analysis on the behavior of the group activities of pedestrians in the image to determine whether there is abnormal behavior, and combines the results of the two to output the video detection result. The combination of the two recognition and analysis methods significantly reduces false detections and improves the accuracy of the detection results compared with the prior art that relies solely on image recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Figure 1A It is a schematic diagram of a scenario provided in the first embodiment of the present application.

[0059] Figure 1 This is a flow chart of an abnormal behavior detection method based on swarm intelligence provided in the first embodiment of the present application.

[0060] Figure 2 A flowchart for extracting structured data of group activities based on the crowd activity video is provided for the first embodiment of the present application.

[0061] Figure 3 A schematic diagram of an abnormal behavior detection output result of swarm intelligence provided in the first embodiment of the present application.

[0062] Figure 4 A flowchart for performing image recognition on video frames in the group activity video to determine whether the video frames may contain abnormal behavior-related objects is provided in the first embodiment of the present application.

[0063] Figure 5 A flowchart of a specific embodiment provided for the first embodiment of the present application.

[0064] Figure 6 A schematic diagram of an abnormal behavior detection device based on swarm intelligence provided in the second embodiment of the present application. DETAILED DESCRIPTION

[0065] In order to more clearly understand the above-mentioned purposes, features and advantages of the present application, the present application is further described in detail below in conjunction with the accompanying drawings and specific implementation methods. It should be noted that the embodiments of the present application and the features in the embodiments can be combined with each other without conflict.

[0066] Many specific details are described in the following description to facilitate a full understanding of the present application. However, the present application can be implemented in many other ways different from those described herein, and those skilled in the art can make similar generalizations without violating the connotation of the present application. Therefore, the present application is not limited by the specific implementation disclosed below.

[0067] In order to more clearly demonstrate the present application, the application scenario of the image processing method provided in the embodiment of the present application is first introduced.

[0068] Some embodiments provided in this application are applied to a monitoring system composed of a video acquisition device, a server and a monitoring video terminal, typically, such as a road traffic monitoring system or a community monitoring system. The video acquisition device generally refers to a camera installed at a road traffic scene.

[0069] like Figure 1 As shown, this figure is a schematic diagram of a typical application system provided by the present application. The video acquisition device is connected to the server 1, and the group activity video of the acquisition site is sent to the server 1 in real time. The server 1 is configured with a system for abnormal behavior detection based on swarm intelligence. The system first acquires the group activity video through the video acquisition unit 101, and then the abnormal behavior judgment unit 102 performs behavior analysis on the group activity video based on the group behavior analysis model of the swarm intelligence algorithm to determine whether the group activity in the group activity video may have abnormal behavior; and the abnormal behavior related object judgment unit 103 performs image recognition on the video frame in the group activity video based on the image recognition model to determine whether it may contain abnormal behavior related objects; finally, the detection result output unit 104 determines whether the output video detection result is abnormal according to the output results of the abnormal behavior judgment unit 102 and the abnormal behavior related object judgment unit 103; specifically, when the abnormal behavior judgment unit 102 judges that there is abnormal behavior, and the abnormal behavior related object judgment unit 103 judges that there is abnormal behavior related object in the image, the output video detection result is abnormal. In the above application system, the video acquisition device may be a number of cameras distributed at road traffic scenes or other places where people gather (such as squares and shopping malls), and it is not excluded that the video acquisition device may be a movable video acquisition device carried by relevant staff.

[0070] As a typical application scenario, the above system is applied to the judgment of abnormal situations in a public gathering place of a certain group of people. The video acquisition equipment is a large number of cameras fixedly installed at various street lights, electric poles, road traffic poles, etc. to collect videos of the public place. It is necessary to use multiple cameras related to the gathering place of the crowd to participate in the judgment. The subsequent embodiments of this application are described by taking the above typical application scenario as an example.

[0071] Specific abnormal behaviors and abnormal behavior-related objects can vary greatly depending on the application scenario; for example, abnormal behaviors can include behaviors such as construction occupying the road; abnormal behavior-related objects are items related to abnormal behaviors, for example, abnormal behavior-related objects of construction occupying the road are construction tools such as shovels and hoes, as well as engineering vehicles, construction signs and other items used in construction. It should be clear that the above application scenario is only a specific embodiment of the image processing method described in this application. The purpose of providing the application scenario embodiment is to facilitate understanding of the method provided in this application for identifying abnormal crowd behavior using a swarm intelligence algorithm, and is not used to limit the scope of application of this application.

[0072] The first embodiment of the present application provides an abnormal behavior detection method based on swarm intelligence. Figure 1 Provide explanation.

[0073] Step S101, obtaining group activity video.

[0074] The group activity video may refer to a real-time video of crowd activities captured by a video capture device such as a camera in a specific scene. For example, a video of crowd activities captured by a camera set up on a road. The specific scene may also include public places such as squares and shopping malls.

[0075] The execution subject of the first embodiment of the present application can be a server, and it is not excluded that it can be a client. If the execution subject of the first embodiment of the present application is a server, the group activity video can also be obtained from the client. Generally, the first embodiment of the present application is applied to a system having a plurality of cameras for collecting videos and a server, and the group activity videos obtained by the cameras for collecting videos are all transmitted to the server for use by the server.

[0076] It should be explained that in general scenarios, group activity videos are obtained from multiple camera devices installed at different locations in real monitoring scenarios such as road traffic. The group activity videos obtained by these devices reflect the activities of a group observed from different angles - generally a human group, but also animal groups, such as monitoring elephant groups and monkey groups - in a certain period of time. For the situation obtained by each camera, the activities of each individual reflected in the obtained activity video should be reflected on the regional plan corresponding to the scene according to the specific scene, as well as the different installation positions and shooting angles of each camera, so as to place the group activity videos obtained by different cameras at the same perspective and obtain unified processing; the specific processing process will be explained in the subsequent steps.

[0077] Step S102 : Based on the group behavior analysis model of the group intelligence algorithm, a behavior analysis is performed on the group activity video to determine whether the group activity in the group activity video may contain abnormal behavior.

[0078] The abnormal behavior mentioned in this application refers to the behavior of group activities that is different from normal situations, which is generally manifested as abnormality of the group status rather than abnormality of a certain individual. For example, it may include the behavior of pedestrians changing their movement mode (such as gathering to watch or bypassing).

[0079] The group behavior analysis model, in this embodiment, refers to an artificial intelligence model that mainly uses a group intelligence algorithm to analyze group behavior.

[0080] The so-called swarm intelligence algorithm (Swarm intelligence) originally mainly refers to the simulation of the group behavior of insects, animal flocks, bird flocks and fish schools. These groups search for food in a cooperative way, and each member of the group constantly changes the direction of the search by learning from its own experience and the experience of other members. The outstanding feature of the swarm intelligence optimization algorithm is to use the group wisdom of the population for collaborative search, so as to find the optimal solution in a short time. Common swarm intelligence optimization algorithms include ant colony algorithm, particle swarm optimization algorithm, bacterial colony optimization algorithm, frog leaping algorithm and artificial bee colony algorithm. In this application, the swarm intelligence algorithm is understood as a type of machine intelligence method that gathers group wisdom and collaboratively solves large-scale complex problems.

[0081] For this application, the swarm intelligence algorithm mainly realizes the simulation of group activity trajectories; further, based on the simulation of group activity trajectories, the group activities are classified through machine learning algorithms to identify whether there are abnormal behaviors. Specifically for the scenario of this application, the main thing achieved is crowd simulation; crowd simulation is the process of simulating the movement of a large number of entities or characters, which is usually used for crisis training, construction and urban planning, and evacuation simulation, and can also be used to create virtual scenes in movies or video games.

[0082] Corresponding to the above functions, the swarm intelligence behavior analysis model includes: a swarm behavior simulation model, and a classification model.

[0083] The group behavior simulation model is used to obtain the trajectory of group activities based on the group activity video, that is, to simulate the group activity; the trajectory of the group activity includes a description of the historical trajectory of each individual in the group reflected by the group activity video, and more importantly, also includes an estimate of the trajectory of the group activity in the next time period. In this application, a general group intelligence algorithm model is selected to implement the group behavior simulation model.

[0084] The classification model is used to judge whether there may be abnormal behavior in group activities; the input data based on which the judgment is based can be used in a variety of ways. In this embodiment, the following method is used: according to the characteristic vector composed of the parameters of the group behavior simulation model, whether there may be abnormal behavior in group activities is judged. The reason for using the characteristic vector composed of the parameters of the group behavior simulation model is that since the above-mentioned group behavior simulation model can simulate group behavior, its parameters must reflect the characteristics of group behavior. Therefore, it is possible to judge whether the group behavior is abnormal based on the characteristic vector composed of its parameters; that is, the group behavior simulation model that can simulate the trajectory of group activities, the characteristic parameters therein can reflect whether the group behavior is normal; through a machine model trained by appropriate machine learning, the group behavior simulation model can be used to judge whether the group behavior is normal.

[0085] Of course, the above classification model can also directly extract parameters reflecting the trajectory as input based on the trajectory of the group activity to determine whether there may be abnormal behavior in the group activity.

[0086] As an implementation mode, the group behavior simulation model is obtained by training on the basis of a general group intelligence algorithm model; for example, a typical ant colony system algorithm model is adopted, and the group behavior simulation model is obtained by training through the following method:

[0087] Acquire group activity videos; extract structured data of group activities based on the group activity video data; accumulate the structured data of group activities until a predetermined time threshold is reached; provide the structured data of group activities accumulated to the predetermined time threshold as training data to an initial group behavior simulation model, and train the group behavior simulation model; use the trained group behavior simulation model as the current group behavior simulation model.

[0088] It should be noted that the group behavior simulation models are all trained under specific space and specific time conditions (referred to as specific scenarios), which can describe the behaviors that have occurred in each individual in the specific scenario, and estimate the actions of each individual in the future time period; therefore, the characteristic parameters in the group behavior simulation model also reflect the characteristics of the specific scenario, and can be used in subsequent classification models to determine whether there is abnormal behavior.

[0089] Taking the ant colony system algorithm model as an example, the specific training process is as follows:

[0090] For the current spatial plan, a traffic coefficient is assigned to each location on the plane (the default traffic coefficient is 0, the default traffic coefficient is 1, and the default traffic coefficient is a decimal between 0 and 1 for steps and other places that affect traffic). The traffic coefficient of the intersection can change according to the status of the traffic light.

[0091] The parameters needed in the model, such as information heuristic factor, expected value heuristic factor, detour coefficient, etc., are initialized to default values; these parameters themselves are learnable parameters and need to be learned and determined in the subsequent training process.

[0092] Given the starting and ending position information of each individual (obtained from the input structured data, for example, in step S101, the 1-minute video information is obtained, the starting position of an individual can be obtained from the first video frame, and the ending position of the individual can be obtained from the last video frame), the behavior pattern (path, speed) is estimated through the initialized model and compared with the visual observation results (structured data obtained from the observation of the obtained group activity video and input into the model), and each learnable parameter is iteratively solved by the numerical optimization method until the estimation error of the behavior pattern is less than the preset threshold or the number of iterations reaches the set upper limit, then the training is considered to be completed;

[0093] The learnable parameters include the above-mentioned information heuristic factor, expected value heuristic factor, detour coefficient and other parameters, and some traffic coefficients (such as the traffic coefficient of the area where an accident may occur on the road can be set to be learnable). The group behavior simulation model trained for a specific scene (the so-called specific scene refers to a specific space and time) can reproduce the group behavior that has been collected and can also predict the group behavior in the next time period; therefore, these parameters can be feature parameters, and these parameters can be combined into feature vectors, which can be used in subsequent classification models to determine whether the group behavior simulation model using this group of parameters has abnormal behavior.

[0094] In fact, for a certain set of learnable parameters, the reasoning process of the group behavior simulation model is actually still an iterative optimization process, so a double numerical optimization iteration process is actually carried out during training.

[0095] The structured data, i.e., model input, can be divided into two categories: one is the location-related data of each target, which actually only needs to include the location information of each target at each moment. In the early preprocessing, the results belonging to each target at each moment need to be distinguished; the rest of the speed, starting and ending points, retention time and other information can be immediately calculated and obtained in the model simulation stage; the other is the attribute-related data of each target and traffic element, including the type of vehicle, the shape, height, gait, etc. of the person, the status of the traffic light, and the meaning of the temporary traffic signs (such as no driving during construction, etc.); when constructing the model, the attributes of the person / vehicle can be used to personalize the configuration of their initial velocity, acceleration and other parameters, and the status of the traffic lights and traffic signs can be used to automatically change the traffic coefficient of a certain area.

[0096] It should be noted that the above-mentioned group behavior simulation model is aimed at the group activity video data obtained according to a specific time period in a specific scene, and simulates the current group activity in the scene, including the group activity in a certain time period in the future; therefore, the training data based on the group behavior simulation model must be the data accumulated for the specific time period that needs to be simulated in the scene, so that the members of the group will not change much, and the rules of group behavior are easy to determine. In the preferred mode, the entire group behavior simulation model is to obtain group activity videos as observation data, perform model training, and apply the model, continuously accumulate data, train the model, and continuously apply the current model. For example, for the group behavior simulation model established in the same area, the characteristic parameters contained in the previous and current ones 10 minutes ago may be different, because the activity subjects have changed a lot; therefore, the group behavior model is always being updated, and the so-called update can be simply considered to be the continuous acquisition of new values ​​of various characteristic parameters.

[0097] In the above steps, the step of extracting structured data of group activities based on the group activity video data is very critical, which is described in detail below.

[0098] Figure 2 A flowchart for extracting structured data of group activities based on the crowd activity video is provided in an embodiment of the present application.

[0099] like Figure 2 As shown, in step S201, a planar scene graph of the area is pre-established.

[0100] The plane scene map of the area is to convert the scene to be monitored from the three-dimensional form of the real scene to the plane form corresponding to the real scene, so as to express the position of each individual in the group and express the movement characteristics. Figure 3On the left side, typically, the plan scene diagram includes the direction of the street and the layout of the video acquisition equipment.

[0101] like Figure 2 As shown, in step S202, target recognition is performed on the video frames of the group activity video to obtain active individuals therein.

[0102] In this step, targets in the video frames of the group activity video are identified, and a pre-trained detection module may be used for target identification, the purpose of which is to obtain the active individuals contained therein so as to determine the position and movement status of each active individual.

[0103] like Figure 2 As shown, in step S203, according to each video frame of the group activity video and the position of the camera device that obtains the video frame, the position of each active individual is marked in the planar scene graph, and structured position parameters are formed and stored in the simulation queue to form simulation queue structured data.

[0104] The purpose of this step is to place the active individuals in the video at appropriate positions in the plane scene graph; specifically, according to each video frame in the obtained group activity video, the active individuals identified in the video frame are converted in combination with the position information of the camera device that obtains the video frame, and the corresponding points in the plane scene graph are obtained, so that the positions of each active individual are marked in the plane scene graph; after marking the position points, the corresponding structured position parameters can be obtained, and the structured position parameters corresponding to each video frame are stored in the simulation queue. Finally, the structured position parameters corresponding to each video frame are arranged in time series to form simulation queue structured data. The so-called structured data refers to standard data containing several fields in a predetermined format. For example, in this step, a set of position data reflecting its coordinate position in the plane scene graph is given to each active individual as a structured position parameter.

[0105] like Figure 2 As shown, in step S204, the structured data of the group activity is extracted based on the simulated queue structured data accumulated for a predetermined time length.

[0106] The structured data may include group activity trajectory parameters and group activity location parameters. The specific data of these parameters have a specified data format, can be logically expressed and implemented by a two-dimensional table structure, and strictly follow the data format and length specifications, and belong to structured data.

[0107] After the data in step S203 is accumulated for a certain period of time, the trajectory information of the group activity can be further obtained according to the time series relationship between the video frames. Therefore, the structured data of the group activity can be further extracted, including group activity trajectory parameters and group activity position parameters; the group activity trajectory parameters are parameters that reflect the dynamic information of the active individuals. For example, according to the position change of a certain active individual in the video frames at different time points, the parameters such as the movement speed and movement direction of the active individual are obtained.

[0108] After extracting the structured data of group activities according to the group activity video data, it is necessary to accumulate the structured data of the group activities to reach a predetermined time threshold, and then further provide the structured data of the group activities accumulated to the predetermined time threshold as training data to the initial group behavior simulation model to train the group behavior simulation model; the trained group behavior simulation model can be used as the current group behavior group simulation model. Among them, the initial group behavior simulation model can be implemented using a typical group intelligence algorithm model, such as an ant colony system algorithm model. By using the above-mentioned accumulated structured data of group activities for training, a group behavior simulation model that can simulate the trajectory of the group activities of the current group in the scene can be obtained, and the trajectory of the group activities can be obtained by analyzing the continuously collected group activity videos, which includes the trajectory of the group activities that have been reflected by the previous group activity videos, and also includes the estimation of the trajectory of the group activities in the next time period.

[0109] After obtaining the trajectory of group activities, we can further judge whether there is abnormal behavior. The specific judgment method can be classified and judged using a classification model built by a machine learning algorithm.

[0110] As an implementation method, the classification model can be obtained in the following manner:

[0111] Collect a number of group activity videos and mark whether there are abnormal behaviors in them; obtain a corresponding group behavior simulation model for each group activity video, extract parameters therein and form a feature vector; provide the feature vector to an initial classification model, and train the initial classification model in combination with the marking of whether there are abnormal behaviors; after the training of the classification model reaches a predetermined standard, use the trained classification model in the group behavior analysis model.

[0112] Among them, manual labeling can generally be used to label abnormal behaviors.

[0113] The implementation process of the step of "corresponding to each group activity video, obtaining a corresponding group behavior simulation model, extracting parameters therein, and forming a feature vector" has been described above and will not be repeated here.

[0114] The initial classification model can adopt various machine learning algorithm models. In the prior art, there are many ways to implement it, which will not be described in detail. The initial classification model is trained using the feature vectors of the group activity video previously labeled. Through repeated training of positive and negative samples, a classification model with a recognition accuracy rate that meets the standard can be obtained.

[0115] As an implementation method, the output of the classification model may include: a judgment result of whether there is abnormal behavior in the group activity, and the corresponding confidence level. For example, "abnormal behavior exists, and the confidence level is 0.9"; or "abnormal behavior does not exist, and the confidence level of abnormal behavior is 0"; of course, it is also possible to only provide a judgment of whether there is abnormal behavior, for example, "existence" indicates that there is abnormal behavior, and "non-existence" indicates that there is no abnormal behavior.

[0116] For example, if the abnormal behavior is construction occupying the road, the classification model can output a judgment result that the crowd may have construction occupying the road as yes, as well as the confidence level of the construction occupying the road.

[0117] As an implementation mode, the output of the classification model may only include: the confidence level of the existence of abnormal behavior in group activities, or the confidence level of the absence of abnormal behavior in group activities, without including the judgment result of whether abnormal behavior is likely to exist or not; this implementation mode is substantially the same as the method of simultaneously outputting the judgment result of whether abnormal behavior is likely to exist or not, and the corresponding confidence level.

[0118] For example, if there is a certain possibility that a crowd of people is occupying the road for construction, the classification model can output the confidence level that the crowd is likely to be occupying the road for construction.

[0119] As an implementation mode, the group behavior analysis model is used to obtain the behavior analysis of the group activity according to the group activity video, and to determine whether the group activity may have abnormal behavior. The complete process can also be described as follows:

[0120] According to the group activity video, obtaining structured data of the group activity that reaches a predetermined time threshold;

[0121] Using the structured data of the group activity that reaches a predetermined time threshold duration to train a group behavior simulation model, obtain a trained group behavior simulation model, and obtain parameters of the group behavior simulation model;

[0122] The parameters of the trained group simulation model are combined into a vector, and input into the trained classification model to obtain the confidence that there is abnormal behavior in the group activity;

[0123] According to the confidence level, it is determined whether there may be abnormal behavior in the group activity.

[0124] Step S103: Based on the image recognition model, image recognition is performed on the video frames in the group activity video to determine whether the video frames may contain abnormal behavior related objects.

[0125] The abnormal behavior related objects refer to objects associated with the abnormal behavior. For example, if the abnormal behavior is occupying the road for construction, the abnormal behavior related objects may be one, two or more of various construction tools, construction vehicles, and construction signs.

[0126] The identification of objects related to abnormal behavior mainly uses image recognition in video frames of group activity videos. For example, objects that meet the conditions of construction tools, construction vehicles, and construction signs are identified. If it is determined that the video frame contains objects related to abnormal behavior, they are marked.

[0127] Since the abnormal behavior related objects identified in the above image recognition may be misidentified, they can be verified in combination with the recognition results of the group behavior analysis model. For example, in the step of "obtaining a behavior analysis of the group activity based on the group activity video by using the group behavior analysis model of the group intelligence algorithm to determine whether the group activity may have abnormal behavior", if it is determined that there is no abnormal behavior, the identified possible abnormal behavior related objects are filtered out.

[0128] When using the pre-trained swarm intelligence crowd behavior analysis model, the behavior analysis of group activities is obtained based on the group activity video. When it is determined that there is no abnormal behavior, the identified possible abnormal behavior related objects are filtered out, which reflects the filtering effect of the swarm intelligence crowd behavior analysis model on the detection system, which can reduce the occurrence of false detection when relying solely on the target detection algorithm and improve the accuracy of the monitoring results. In particular, the identification of the same abnormal behavior related objects may exist in different video frames, and they can be filtered out, which can effectively reduce false detections.

[0129] For example, image recognition is performed on images in group activity videos to determine that they may contain construction projects occupying the road. The analysis result of the group intelligent crowd behavior analysis model is that there is no abnormal behavior of construction projects occupying the road. In this case, the construction tools such as shovels and hoes detected and identified are regarded as false detections, and the possible construction tools such as shovels and hoes identified are filtered out.

[0130] The above-mentioned image recognition model is based on which the video frames in the group activity video are subjected to image recognition to determine whether the video frames may contain abnormal behavior related objects. The specific implementation method thereof can be seen in Figure 4 .

[0131] like Figure 4As shown, in step S401, target recognition is performed on the current video frame image of the group activity video.

[0132] like Figure 4 As shown, in step S402, the identified target object is provided to a pre-trained abnormal behavior related object detection model to perform abnormal behavior related object identification. The specific type of the abnormal behavior related object can be determined according to the purpose of executing the method in a specific application scenario; it can include one or more. For example, if the target is construction road occupation behavior, the abnormal behavior related object is construction equipment, such as one, two or more of various construction tools, construction vehicles, construction signs and other items; if the target is traffic accident identification, the abnormal behavior related object is stationary cars that collide with each other.

[0133] like Figure 4 As shown, in step S403, if possible abnormal behavior related objects are identified and the possibility exceeds a specified threshold, it is determined that the abnormal behavior related objects may be included.

[0134] The abnormal behavior related object detection model refers to a target detection model used to detect abnormal behavior related objects. Object detection is a branch of computer technology closely related to computer vision and image processing. Its goal is to detect specific semantic target entities in digital images and videos, such as people, buildings, cars, etc. Usually, in order to facilitate manual observation, a rectangular frame tightly enclosing the target entity will be output on the display screen. Object detection has applications in many computer vision fields such as image retrieval and video monitoring.

[0135] As an implementation method, the first embodiment of the present application may also include: displaying the planar scene diagram on the screen; and marking the positions of the individual activities in the planar scene diagram on the screen. The monitoring camera live image may also be output, and the detection frame of the abnormal behavior related object captured by the abnormal behavior related object detection model and the schematic diagram of the human flow trajectory in the area may be output on the live image. Figure 3 The figure shows a schematic diagram of the output results of the construction road occupation detection.

[0136] It should be noted that, in order to save time, step S102 and step S103 can be executed in parallel; in order to reduce threads, step S102 and step S103 can also be executed in series.

[0137] like Figure 1 As shown, in step S104, if the results of step S102 and step S103 are both yes, the output video detection result is abnormal.

[0138] In step S104, only when the group behavior analysis model is used to determine that the group activity may have abnormal behavior, and the image recognition is performed on the image in the group activity video to determine that it may contain abnormal behavior related objects, the video detection result is output as abnormal. At this time, other measures can be further taken, including alarm processing, further adding monitoring measures, etc.

[0139] Compared with the prior art which only uses image recognition to determine whether an image may contain abnormal behavior-related objects and then issues an alarm, the present application embodies the abnormal behavior information obtained by group intelligence analysis of group behavior, which can check the judgment results of abnormal behavior-related objects. Furthermore, it can also filter out abnormal behavior-related objects that are misidentified, reduce the occurrence of false detection when relying solely on target detection algorithms, and improve the accuracy of detection results.

[0140] In order to explain the present application more clearly, a specific embodiment is introduced below in combination with the construction road occupation detection scenario.

[0141] like Figure 5 As shown, in step S501, a group activity video is obtained;

[0142] like Figure 5 As shown, in step S502, the video frame of the group activity video is read;

[0143] like Figure 5 As shown, in step S503, structured data of group activities are extracted according to the video frames of the group activity video;

[0144] like Figure 5 As shown, in step S504, it is determined whether the accumulated structured data of the group activity reaches a predetermined time threshold, and if so, step S505 is executed; if not, the process returns to S502;

[0145] like Figure 5 As shown, in step S505, it is identified whether at least one, or two or more of the abnormal behavior related objects such as construction tools, construction vehicles, and construction signs exist in the image;

[0146] like Figure 5 As shown, in step S506, a group behavior analysis model is constructed, and the group behavior analysis model is used to determine whether the group activity includes construction road occupation behavior according to the group activity video;

[0147] like Figure 5 As shown, in step S507, when the results in step S505 and step S506 are both yes, an exception process is performed, which includes alarming and strengthening monitoring.

[0148] The above is an introduction to the first embodiment of the present application. The first embodiment of the present application performs image recognition on images in group activity videos to determine whether they may contain objects related to abnormal behavior (for example, construction tools, construction vehicles, or construction signs). At the same time, a group intelligence crowd behavior analysis model is introduced to analyze the behavior of group activities of pedestrians in the image to determine whether there is abnormal behavior (for example, construction road occupation behavior). The results of the two are combined to determine whether an alarm is needed, which reflects the filtering effect of the group intelligence crowd behavior analysis model on image recognition, reduces the occurrence of false detection when relying solely on the target detection algorithm, and improves the accuracy of the detection results.

[0149] Corresponding to the abnormal behavior detection method based on swarm intelligence provided in the first embodiment of the present application, the second embodiment of the present application provides an abnormal behavior detection device based on swarm intelligence.

[0150] like Figure 6 As shown, the abnormal behavior detection device based on swarm intelligence includes:

[0151] The video acquisition unit 601 is used to acquire group activity videos;

[0152] The abnormal behavior determination unit 602 is used to perform behavior analysis on the group activity video based on the group behavior analysis model of the group intelligence algorithm to determine whether the group activity in the group activity video may have abnormal behavior;

[0153] The abnormal behavior related object determination unit 603 is used to perform image recognition on the video frames in the group activity video based on the image recognition model to determine whether the video frames may contain abnormal behavior related objects;

[0154] The detection result output unit 604 is used to output the video detection result as abnormal when the output results of the abnormal behavior determination unit and the abnormal behavior related object determination unit are both yes.

[0155] As an implementation method, the swarm intelligence crowd behavior analysis model includes:

[0156] A group behavior simulation model, used to obtain a trajectory of group activities based on the group activity video; the trajectory of group activities includes an estimate of the trajectory of group activities in the next time period;

[0157] The classification model is used to judge whether there may be abnormal behavior in the group activity according to the feature vector composed of the parameters of the group behavior simulation model.

[0158] As an embodiment, the device comprises: a group behavior simulation model training unit, which is used to:

[0159] Get videos of group activities;

[0160] Extracting structured data of group activities according to the group activity video data;

[0161] Accumulating structured data of the group activity for a predetermined time threshold;

[0162] providing the structured data of the group activity accumulated to a predetermined time threshold as training data to an initial group behavior simulation model to train the group behavior simulation model;

[0163] The trained group behavior simulation model is used as the current group behavior simulation model.

[0164] As an embodiment, the device includes a classification model obtaining unit, which is used to:

[0165] Collect several group activity videos and mark whether there are any abnormal behaviors in them;

[0166] Corresponding to each group activity video, the corresponding group behavior simulation model is obtained, and the parameters therein are extracted to form a feature vector;

[0167] Providing the feature vector to an initial classification model, and training the initial classification model in combination with the annotation of whether there is abnormal behavior;

[0168] After the training of the classification model reaches a predetermined standard, the trained classification model is used in the group behavior analysis model.

[0169] As an implementation mode, the output of the classification model includes: a judgment of whether the group activity may have abnormal behavior or not, and a corresponding confidence level.

[0170] As an implementation mode, the swarm intelligence crowd simulation model training unit is specifically used for:

[0171] Pre-establish a planar scene graph of the area;

[0172] Performing target recognition on video frames of the crowd activity video to obtain active individuals therein;

[0173] According to each video frame of the crowd activity video and the position of the camera device that obtains the video frame, the position of each of the active individuals is marked in the plane scene graph, and a structured position parameter is formed and stored in a simulation queue to form simulation queue structured data;

[0174] According to the simulation queue structured data accumulated for a predetermined time length, the structured data of the group activity is extracted, including group activity trajectory parameters and group activity position parameters.

[0175] As an implementation mode, the abnormal behavior related object determination unit is specifically used to determine that there may be abnormal behavior related objects, and then identify the possible abnormal behavior related objects; and, in the abnormal behavior determination unit, if it is determined that there is no abnormal behavior, then the identified possible abnormal behavior related objects are filtered out.

[0176] As an implementation manner, the abnormal behavior related object determination unit is specifically used for:

[0177] Performing target recognition on the current video frame image of the group activity video;

[0178] Providing the identified target object to a pre-trained abnormal behavior related object detection model to perform abnormal behavior related object identification;

[0179] If possible abnormal behavior correlates are identified and the likelihood exceeds a specified threshold, it is determined that the abnormal behavior correlates may be included.

[0180] As an embodiment, the device further includes a display unit, which is used to display the planar scene diagram on a screen; and mark the positions of each of the active individuals in the planar scene diagram on the screen.

[0181] It should be noted that for the detailed description of the device provided in the second embodiment of the present application, reference can be made to the relevant description of the first embodiment of the present application, which will not be repeated here.

[0182] Corresponding to the abnormal behavior detection method based on swarm intelligence provided in the first embodiment of the present application, the third embodiment of the present application provides an electronic device.

[0183] The electronic device comprises:

[0184] Processor; and

[0185] The memory is used to store a program of the abnormal behavior detection method based on swarm intelligence. After the device is powered on and the program of the abnormal behavior detection method based on swarm intelligence is run by the processor, the following steps are performed:

[0186] Get videos of group activities;

[0187] Based on a group behavior analysis model of a group intelligence algorithm, a behavior analysis is performed on the group activity video to determine whether the group activity in the group activity video may contain abnormal behavior; and

[0188] Based on the image recognition model, image recognition is performed on the video frames in the group activity video to determine whether the video frames may contain objects related to abnormal behavior;

[0189] If all the above results are yes, the output video detection result is abnormal.

[0190] As an implementation mode, the group behavior analysis model includes:

[0191] A group behavior simulation model, used to obtain a trajectory of group activities based on the group activity video; the trajectory of group activities includes an estimate of the trajectory of group activities in the next time period;

[0192] The classification model is used to judge whether there may be abnormal behavior in the group activity according to the feature vector composed of the parameters of the group behavior simulation model.

[0193] As an implementation mode, the swarm intelligence crowd simulation model is trained and obtained by the following method:

[0194] Get videos of group activities;

[0195] Extracting structured data of group activities according to the group activity video data;

[0196] Accumulating structured data of the group activity for a predetermined time threshold;

[0197] providing the structured data of the group activity accumulated to a predetermined time threshold as training data to an initial group behavior simulation model to train the group behavior simulation model;

[0198] The trained group behavior simulation model is used as the current group behavior simulation model.

[0199] As an implementation mode, the classification model is obtained in the following manner:

[0200] Collect several videos of crowd activities and mark whether there are any abnormal behaviors in them;

[0201] Corresponding to each group activity video, the corresponding group behavior simulation model is obtained, and the parameters therein are extracted to form a feature vector;

[0202] Providing the feature vector to an initial classification model, and training the initial classification model in combination with the annotation of whether there is abnormal behavior;

[0203] After the training of the classification model reaches a predetermined standard, the trained classification model is used in the group behavior analysis model.

[0204] As an implementation mode, the output of the classification model includes: a judgment of whether the group activity may have abnormal behavior or not, and a corresponding confidence level.

[0205] As an implementation manner, extracting structured data of group activities based on the crowd activity video includes:

[0206] Pre-establish a planar scene graph of the area;

[0207] Performing target recognition on video frames of the crowd activity video to obtain active individuals therein;

[0208] According to each video frame of the crowd activity video and the position of the camera device that obtains the video frame, the position of each of the active individuals is marked in the plane scene graph, and a structured position parameter is formed and stored in a simulation queue to form simulation queue structured data;

[0209] According to the simulation queue structured data accumulated for a predetermined time length, the structured data of the group activity is extracted, including group activity trajectory parameters and group activity position parameters.

[0210] As an implementation mode, the image recognition model is used to perform image recognition on the images in the group activity video, and in the step of determining whether the images contain abnormal behavior-related objects, if it is determined that the images may contain abnormal behavior-related objects, then the possible abnormal behavior-related objects are identified; and the group behavior analysis model based on the group intelligence algorithm performs behavior analysis on the group activity video, and in the step of determining whether the group activity in the group activity video may contain abnormal behavior, if it is determined that there is no abnormal behavior, then the identified possible abnormal behavior-related objects are filtered out.

[0211] As an implementation manner, the image recognition model is used to perform image recognition on the images in the group activity video to determine whether the images may contain abnormal behavior related objects, including:

[0212] Performing target recognition on the current video frame image of the group activity video;

[0213] Providing the identified target object to a pre-trained abnormal behavior related object detection model to perform abnormal behavior related object identification;

[0214] If possible abnormal behavior correlates are identified and the likelihood exceeds a specified threshold, it is determined that the abnormal behavior correlates may be included.

[0215] As an implementation method, it also includes: displaying the plane scene diagram on a screen; and marking the positions of each of the active individuals in the plane scene diagram on the screen.

[0216] It should be noted that for the detailed description of the electronic device provided in the third embodiment of the present application, reference can be made to the relevant description of the first embodiment of the present application, which will not be repeated here.

[0217] A fourth embodiment of the present application provides a storage device storing a program of an abnormal behavior detection method based on swarm intelligence, wherein the program is executed by a processor to perform the following steps:

[0218] Get videos of group activities;

[0219] Based on a group behavior analysis model of a group intelligence algorithm, a behavior analysis is performed on the group activity video to determine whether the group activity in the group activity video may contain abnormal behavior; and

[0220] Based on the image recognition model, image recognition is performed on the video frames in the group activity video to determine whether the video frames may contain objects related to abnormal behavior;

[0221] If all the above results are yes, the output video detection result is abnormal.

[0222] It should be noted that for the detailed description of the storage device provided in the fourth embodiment of the present application, reference can be made to the relevant description of the first embodiment of the present application, which will not be repeated here.

[0223] Although the present application is disclosed as above in the form of a preferred embodiment, it is not intended to limit the present application. Any technical personnel in this field may make possible changes and modifications without departing from the spirit and scope of the present application. Therefore, the scope of protection of the present application shall be based on the scope defined by the claims of the present application.

[0224] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0225] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0226] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include non-transitory media such as modulated data signals and carrier waves.

[0227] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

Claims

1. A method for detecting abnormal behavior based on swarm intelligence, characterized in that: include: Get videos of group activities; Based on a group behavior analysis model of a group intelligence algorithm, a behavior analysis is performed on the group activity video to determine whether the group activity in the group activity video may contain abnormal behavior; as well as, Based on the image recognition model, image recognition is performed on the video frames in the group activity video to determine whether the video frames may contain objects related to abnormal behavior; If the group activity may have abnormal behavior, and the video frame may contain abnormal behavior related objects, then the output video detection result is abnormal; Wherein, the group behavior analysis model includes: A group behavior simulation model, used to obtain a trajectory of group activities based on the group activity video; the trajectory of group activities includes an estimate of the trajectory of group activities in the next time period; The classification model is used to judge whether there may be abnormal behavior in the group activity according to the feature vector composed of the parameters of the group behavior simulation model.

2. The abnormal behavior detection method based on swarm intelligence according to claim 1 is characterized in that: The group behavior simulation model is trained by the following method: Get videos of group activities; Extracting structured data of group activities according to the group activity video data; Accumulating structured data of the group activity for a predetermined time threshold; providing the structured data of the group activity accumulated to a predetermined time threshold as training data to an initial group behavior simulation model to train the group behavior simulation model; The trained group behavior simulation model is used as the current group behavior simulation model.

3. The abnormal behavior detection method based on swarm intelligence according to claim 1 is characterized in that: The classification model is obtained in the following way: Collect videos of group activities that meet the quantity requirements and mark whether there are any abnormal behaviors in them; Corresponding to each group activity video, the corresponding group behavior simulation model is obtained, and the parameters therein are extracted to form a feature vector; Providing the feature vector to an initial classification model, and training the initial classification model in combination with the annotation of whether there is abnormal behavior; After the training of the classification model reaches a predetermined standard, the trained classification model is used in the group behavior analysis model.

4. The abnormal behavior detection method based on swarm intelligence according to claim 1 is characterized in that: The output of the classification model includes at least one of the following two types: The judgment result of the existence or non-existence of abnormal behavior in group activities, and the corresponding confidence level; The judgment result of whether abnormal behavior exists or does not exist in group activities.

5. The abnormal behavior detection method based on swarm intelligence according to claim 2 is characterized in that: The step of extracting structured data of group activities according to the group activity video includes: Pre-establish a planar scene graph of the area; Performing target recognition on video frames of the group activity video to obtain active individuals therein; According to each video frame of the group activity video and the position of the camera device that obtains the video frame, the position of each active individual is marked in the plane scene graph, and a structured position parameter is formed and stored in a simulation queue to form simulation queue structured data; The structured data of the group activity is extracted based on the simulated queue structured data accumulated for a predetermined time length.

6. The abnormal behavior detection method based on swarm intelligence according to claim 1 is characterized in that: The image recognition model is based on the step of performing image recognition on the images in the group activity video to determine whether the images contain abnormal behavior-related objects. If it is judged that the images may contain abnormal behavior-related objects, the possible abnormal behavior-related objects are identified; and the group behavior analysis model based on the group intelligence algorithm performs behavior analysis on the group activity video to determine whether the group activity in the group activity video may contain abnormal behavior. If it is judged that there is no abnormal behavior, the identified possible abnormal behavior-related objects are filtered out.

7. The abnormal behavior detection method based on swarm intelligence according to claim 1, characterized in that: The performing image recognition on the video frames in the group activity video based on the image recognition model to determine whether the video frames may contain abnormal behavior related objects includes: Performing target recognition on the current video frame image of the group activity video; Providing the identified target object to a pre-trained abnormal behavior related object detection model to perform abnormal behavior related object identification; If possible abnormal behavior correlates are identified and the likelihood exceeds a specified threshold, it is determined that the abnormal behavior correlates may be included.

8. The abnormal behavior detection method based on swarm intelligence according to claim 5 is characterized in that: Also includes: Displaying the planar scene graph on a screen; And, the positions of each of the active individuals are marked in the plane scene graph on the screen.

9. An abnormal behavior detection device based on swarm intelligence, characterized in that: include: A video acquisition unit, used to acquire group activity videos; an abnormal behavior determination unit, configured to perform behavior analysis on the group activity video based on a group behavior analysis model of a group intelligence algorithm, and determine whether the group activity in the group activity video may contain abnormal behavior; The group behavior analysis model includes: a group behavior simulation model and a classification model, wherein the group behavior simulation model is used to obtain the trajectory of the group activity according to the group activity video, and the trajectory of the group activity includes an estimation of the trajectory of the group activity in the next time period; the classification model is used to judge whether the group activity may have abnormal behavior according to the feature vector composed of the parameters of the group behavior simulation model; an abnormal behavior-related object determination unit, configured to perform image recognition on video frames in the group activity video based on an image recognition model to determine whether the video frames may contain abnormal behavior-related objects; The detection result output unit is used to output the video detection result as abnormal when the output results of the abnormal behavior determination unit and the abnormal behavior related object determination unit are both yes.

10. An electronic device comprising: processor; as well as The memory is used to store a program of the abnormal behavior detection method based on swarm intelligence. After the device is powered on and the program of the abnormal behavior detection method based on swarm intelligence is run by the processor, the following steps are performed: Get videos of group activities; Based on a group behavior analysis model of a group intelligence algorithm, a behavior analysis is performed on the group activity video to determine whether the group activity in the group activity video may contain abnormal behavior; as well as, Based on the image recognition model, image recognition is performed on the video frames in the group activity video to determine whether the video frames may contain objects related to abnormal behavior; If the group activity may have abnormal behavior, and the video frame may contain abnormal behavior related objects, then the output video detection result is abnormal; Wherein, the group behavior analysis model includes: A group behavior simulation model, used to obtain a trajectory of group activities based on the group activity video; the trajectory of group activities includes an estimate of the trajectory of group activities in the next time period; The classification model is used to judge whether there may be abnormal behavior in the group activity according to the feature vector composed of the parameters of the group behavior simulation model.

11. A storage device storing a program of an abnormal behavior detection method based on swarm intelligence, wherein the program is executed by a processor to perform the following steps: Get videos of group activities; Based on a group behavior analysis model of a group intelligence algorithm, a behavior analysis is performed on the group activity video to determine whether the group activity in the group activity video may contain abnormal behavior; as well as, Based on the image recognition model, image recognition is performed on the video frames in the group activity video to determine whether the video frames may contain objects related to abnormal behavior; If the group activity may have abnormal behavior, and the video frame may contain abnormal behavior related objects, then the output video detection result is abnormal; Wherein, the group behavior analysis model includes: A group behavior simulation model, used to obtain a trajectory of group activities based on the group activity video; the trajectory of group activities includes an estimate of the trajectory of group activities in the next time period; The classification model is used to judge whether there may be abnormal behavior in the group activity according to the feature vector composed of the parameters of the group behavior simulation model.

Images