A detection method, device, equipment and storage medium for group abnormal events

Abstract

The present invention discloses a method, apparatus, device and storage medium for detecting group abnormal events. The method includes: for each image frame in video data, obtaining the image optical flow field data and at least one object area image corresponding to the current image frame; for each object area image, determining at least one motion particle point corresponding to the object area image, where the motion particle point is used to characterize the motion unit corresponding to the object area image; for each motion particle point, based on the image optical flow field data, determining the interaction power corresponding to the motion particle point, where the interaction power is used to characterize the motion stability state of the motion particle point; based on at least one interaction power, determining whether the current image frame belongs to an image frame with group abnormal events. The present invention solves the problem that the existing methods for detecting group abnormal events are inaccurate and improves the detection efficiency of group abnormal events.

Description

Technical Field

[0001] The present invention relates to the technical field of video surveillance, and in particular, to a method, device, equipment and storage medium for detecting group abnormal events. Background Art

[0002] Group abnormal event detection refers to discovering unique information that can depict the abnormal behaviors of a crowd from surveillance video data, such as crowd density, crowd behavior characteristics, etc. By analyzing this information characterizing the crowd, people can be reminded to handle abnormal events in a timely manner and maintain public safety.

[0003] The current group abnormal event detection methods are mainly divided into two categories. One is the detection method based on individual targets, which obtains the behavior characteristics of the entire crowd by analyzing the motion characteristics of each individual, such as motion trajectories, postures, etc. The other is the detection method based on group characteristics, which extracts the crowd characteristic information in each frame of the video, and trains a learning model based on the crowd characteristic information to achieve the classification of normal and abnormal situations.

[0004] The existing detection methods based on individual targets are not applicable to environments with large crowd density and complex scenes, and the detection effect is poor. The existing detection methods based on group characteristics rely on the collection of training data sets and the model training process takes a long time. Summary of the Invention

[0005] The present invention provides a method, device, equipment and storage medium for detecting group abnormal events to improve the detection efficiency and the accuracy of the detection effect of group abnormal events.

[0006] According to one aspect of the present invention, there is provided a method for detecting group abnormal events, the method comprising:

[0007] For each image frame in the video data, obtaining the image optical flow field data corresponding to the current image frame and at least one object area image;

[0008] For each object area image, determining at least one motion particle point corresponding to the object area image; wherein, the motion particle point is used to characterize the motion unit corresponding to the object area image;

[0009] For each motion particle point, based on the image optical flow field data, determining the interaction power corresponding to the motion particle point; wherein, the interaction power is used to characterize the motion stability state of the motion particle point;

[0010] Based on at least one interaction power, determining whether the current image frame belongs to an image frame with a group abnormal event.

[0011] According to another aspect of the present invention, there is provided a detection device for group abnormal events, the device comprising:

[0012] An object area image acquisition module, configured to acquire image optical flow field data and at least one object area image corresponding to the current image frame for each image frame in the video data;

[0013] A moving particle point determination module, configured to determine at least one moving particle point corresponding to the object area image for each object area image; wherein, the moving particle point is used to characterize the moving unit corresponding to the object area image;

[0014] An interaction power determination module, configured to determine the interaction power corresponding to the moving particle point based on the image optical flow field data for each moving particle point; wherein, the interaction power is used to characterize the motion stability state of the moving particle point;

[0015] A group abnormal event determination module, configured to determine whether the current image frame belongs to an image frame with group abnormal events based on at least one interaction power.

[0016] According to another aspect of the present invention, there is provided an electronic device, the electronic device comprising:

[0017] At least one processor; and

[0018] A memory communicatively connected to the at least one processor;

[0019] Wherein, the memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the detection method for group abnormal events according to any embodiment of the present invention.

[0020] According to another aspect of the present invention, there is provided a computer-readable storage medium, the computer-readable storage medium stores computer instructions, and the computer instructions are used to implement the detection method for group abnormal events according to any embodiment of the present invention when executed by a processor.

[0021] The technical solution of the embodiment of the present invention, by acquiring the image optical flow field data and at least one object area image corresponding to the current image frame for each image frame in the video data, and for each object area image, determining the interaction power corresponding to at least one moving particle point in the object area image based on the image optical flow field data corresponding to the current image frame, and determining whether the current image frame belongs to an image frame with group abnormal events based on at least one interaction power, solves the problem of crowd feature extraction in scenarios with large crowd density, serious crowd occlusion, and complex scenes, and improves the detection efficiency and accuracy of the detection effect of group abnormal events.

[0022] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present invention, nor is it used to limit the scope of the present invention. Other features of the present invention will become readily understood through the following description. Description of the Drawings

[0023] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the drawings in the following description are only some embodiments of the present invention, and for those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0024] Figure 1 is a flowchart of a method for detecting group abnormal events provided in Embodiment 1 of the present invention;

[0025] Figure 2A is a schematic diagram of a foreground target area provided in Embodiment 1 of the present invention;

[0026] Figure 2B is a schematic diagram of image optical flow field data provided in Embodiment 1 of the present invention;

[0027] Figure 2C is a schematic diagram of an object area image provided in Embodiment 1 of the present invention;

[0028] Figure 2D is a schematic diagram of an actual movement speed provided in Embodiment 1 of the present invention;

[0029] Figure 2E is a schematic diagram of an interaction power provided in Embodiment 1 of the present invention;

[0030] Figure 3 is a schematic diagram of an interaction power corresponding to video data provided in Embodiment 1 of the present invention;

[0031] Figure 4 is a flowchart of a method for detecting group abnormal events provided in Embodiment 2 of the present invention;

[0032] Figure 5 is a flowchart of a method for determining an adaptive standard interaction power mean provided in Embodiment 2 of the present invention;

[0033] Figure 6 is a flowchart of a specific example of a method for detecting group abnormal events provided in Embodiment 2 of the present invention;

[0034] Figure 7It is a schematic structural diagram of a detection device for group abnormal events provided in Embodiment 3 of the present invention;

[0035] Figure 8 It is a schematic structural diagram of an electronic device provided in Embodiment 4 of the present invention. Detailed implementation manners

[0036] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without making creative efforts shall fall within the protection scope of the present invention.

[0037] It should be noted that the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device that includes a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or are inherent to these processes, methods, products or devices.

[0038] Embodiment 1

[0039] Figure 1 It is a flowchart of a method for detecting group abnormal events provided in Embodiment 1 of the present invention. This embodiment is applicable to the situation of detecting group abnormal events in video data. This method can be executed by a detection device for group abnormal events. The detection device for group abnormal events can be implemented in the form of hardware and / or software, and the detection device for group abnormal events can be configured in a terminal device. As Figure 1 shown, this method includes:

[0040] S110. For each image frame in the video data, obtain the image optical flow field data corresponding to the current image frame and at least one object area image.

[0041] Specifically, the data format of the video data can be wmv format, mp4 format, rmvb format or mov, etc. The video data can be video data collected in any scenario. Exemplarily, the scenarios include but are not limited to restaurants, parks, classrooms and office areas, etc. The data format and video scenario of the video data are not limited herein.

[0042] Among them, a frame is the smallest unit of a single image frame in video data. An image frame is a still picture, and video data is composed of multiple still pictures.

[0043] In one embodiment, optionally, obtaining the image optical flow field data and at least one object region image corresponding to the current image frame includes: determining the foreground target region in the current image frame based on a preset image processing algorithm; determining the image optical flow field data corresponding to the foreground target region based on a preset optical flow algorithm; and determining at least one object region image corresponding to the foreground target region based on a preset object detection algorithm.

[0044] Among them, by way of example, the preset image processing algorithms include but are not limited to the single Gaussian model algorithm, the mixture Gaussian model algorithm, the sliding Gaussian average algorithm, the SOBS (Self-Organizing Background Subtraction) algorithm, the SACON (Sample Consensus Modeling) algorithm, the VIBE algorithm, and so on. In one embodiment, optionally, the VIBE algorithm is used to determine the foreground target region in the current image frame. Among them, the VIBE algorithm is an algorithm for pixel-level video background modeling or foreground detection, which mainly includes three parts: the initialization of the background model, the discrimination method of foreground pixel points, and the update mechanism of background pixels.

[0045] Figure 2A It is a schematic diagram of a foreground target region provided according to Embodiment 1 of the present invention. Specifically, Figure 2A the left figure in represents the current image frame, Figure 2A the right figure in represents the foreground target region extracted based on the VIBE algorithm. The white region in the foreground target region represents the foreground image region in the current image frame, and the black region represents the background image region in the current image frame.

[0046] In space, motion can be described by a motion field. On an image plane, the motion of an object is often reflected by the different gray-scale distributions of different images in an image sequence. Therefore, the motion field in space transferred to an image is represented as an optical flow field. Specifically, the image optical flow field is a two-dimensional vector field, which reflects the change trend of the gray scale of each pixel point on the image frame, and can be regarded as an instantaneous velocity field generated by the movement of pixels with gray scale on the image plane.

[0047] Among them, by way of example, the preset optical flow algorithms include but are not limited to the differential method, the matching-based algorithm, the energy-based algorithm, the phase-based algorithm, and the neurodynamics algorithm, and so on. In one embodiment, optionally, the LDOF (Large Displacement Optical Flow) algorithm is used to determine the image optical flow field data corresponding to the foreground target region. Among them, the LDOF algorithm is an algorithm proposed by Thomas Brox et al. for estimating the optical flow field by combining a feature point matching method and a variational optical flow model.

[0048] Figure 2BIt is a schematic diagram of image optical flow field data provided according to Embodiment 1 of the present invention. Specifically, Figure 2B The arrows in it represent two-dimensional velocity vectors. The length of the arrow represents the magnitude of the velocity, and the direction of the arrow represents the moving direction of the velocity.

[0049] In one embodiment, optionally, the preset object detection algorithm is HOG (Histogram of Oriented Gradient) features and an SVM classifier. Specifically, each object region image contains one object. Exemplarily, the object can be a pedestrian, an animal, or an object, and the type of the object is not limited here.

[0050] Figure 2C It is a schematic diagram of an object region image provided according to Embodiment 1 of the present invention. Specifically, Figure 2C The image within each black square in it represents an object region image.

[0051] S120. For each object region image, determine at least one motion particle point corresponding to the object region image.

[0052] In this embodiment, the motion particle point is used to characterize the motion unit corresponding to the object region image. In one embodiment, optionally, determining at least one motion particle point corresponding to the object region image includes: taking the image center point of the object region image as the motion particle point.

[0053] In another embodiment, optionally, determining at least one motion particle point corresponding to the object region image includes: equally dividing the object region image into at least two image blocks, and taking the image block center point in each image block as the motion particle point. The advantage of such a setting is that the dimension of the set of motion particle points obtained in the current image frame is relatively low, ensuring the accuracy of the detection effect of subsequent group abnormal events.

[0054] S130. For each motion particle point, based on the image optical flow field data, determine the interaction power corresponding to the motion particle point.

[0055] In one embodiment, optionally, based on the image optical flow field data, determining the interaction power corresponding to the motion particle point includes: based on the image block corresponding to the motion particle point and the image optical flow field data corresponding to the image block, determine the actual motion speed of the motion particle point; based on the panic weight coefficient and the image optical flow field data corresponding to the motion particle point, determine the expected motion speed of the motion particle point; based on the actual motion speed and the expected motion speed, determine the interaction power corresponding to the motion particle point.

[0056] Among them, specifically, the actual motion speed v i satisfies the formula:

[0057]

[0058] Among them, O ave (x i , y i ) represents the average value of the image optical flow field data corresponding to all pixel points in the i-th image block corresponding to the i-th moving particle point. n represents the number of pixel points in the horizontal direction corresponding to the i-th image block, and m represents the number of pixel points in the vertical direction corresponding to the i-th image block. O(x i , y j ) represents the image optical flow field data corresponding to the pixel point with coordinates (x i , y j ) in the i-th image block.

[0059] Figure 2D is a schematic diagram of an actual movement speed provided according to Embodiment 1 of the present invention. Specifically, Figure 2D each of the at least one black arrow corresponding to each pedestrian in it corresponds to a moving particle point. Among them, the length of the black arrow represents the magnitude of the actual movement speed of the moving particle point, and the direction of the black arrow represents the movement direction of the actual movement speed of the moving particle point.

[0060] Among them, specifically, the expected movement speed satisfies the formula:

[0061]

[0062] Among them, p i represents the panic weight coefficient corresponding to the i-th moving particle point. O(x i , y j ) represents the optical flow field data corresponding to the center point of the image block in the i-th image block corresponding to the i-th moving particle point.

[0063] Among them, the panic weight coefficient p i can represent, for example, that when a major event occurs, individuals in the crowd will exhibit a herd behavior of following the overall crowd to flee. When p i approaches 0, pedestrians tend to move independently as individuals. When p i approaches 1, pedestrians move according to the movement trend of the surrounding crowd and no longer act independently.

[0064] Factors such as individual movement, repulsive force, and attractive force will all affect the movement of pedestrians. Since the moving particle points have the common speed of the crowd flow field, that is to say, each moving particle point has its own expected movement speed. If there is a deviation between the actual movement speed of the moving particle point and the expected movement speed, this deviation is caused by the interaction force between the moving particle point and other moving particle points or the environment around it.

[0065] In the interaction power model proposed in the embodiments of the present invention, two factors need to be considered first. One is the individual motivation of pedestrians, and the other is the influence of the environment in the scene on pedestrians. In a dense crowd scene, each pedestrian i has a mass m i , and the actual force F a received by pedestrian i satisfies the formula:

[0066]

[0067] Among them, F p represents the individual expected force generated by the pedestrian due to the personal expected movement direction, and F int represents the interaction force generated by the pedestrian being affected by the surrounding environment.

[0068] Among them, the expected force F p satisfies the formula:

[0069]

[0070] Among them, τ represents the relaxation coefficient.

[0071] For a specific scene or crowd, the individuals in the crowd can be regarded as having approximately the same size, so it can be assumed that m i = 1. The interaction power is obtained based on the interaction force F int received by each moving particle point. In one embodiment, optionally, the interaction power satisfies the formula:

[0072]

[0073] Among them, P int represents the interaction power, F int represents the interaction force, v i represents the actual movement speed of the i-th moving particle point, τ represents the relaxation coefficient, represents the expected movement speed.

[0074] In this embodiment, the interaction power is used to characterize the movement stability state of the moving particle point. When the interaction power is large, it indicates that the current movement potential energy of the moving particle point changes rapidly, then the movement state of the moving particle point is more unstable, and the crowd is more likely to show abnormalities; on the contrary, when the interaction power is small, the movement potential energy of the moving particle point changes slowly, then the movement state of the moving particle point is relatively stable, and the crowd also tends to be normal.

[0075] Figure 2EIt is a schematic diagram of interaction power provided according to Embodiment 1 of the present invention. Specifically, different gray values represent parameter values of different interaction powers. It can be understood that in practical applications, different color values can be used to represent parameter values of different interaction powers. For example, blue represents a smaller parameter value of interaction power, and red represents a larger parameter value of interaction power.

[0076] S140. Based on at least one interaction power, determine whether the current image frame belongs to an image frame with a crowd abnormal event.

[0077] In one embodiment, optionally, based on at least one interaction power, determining whether the current image frame belongs to an image frame with a crowd abnormal event includes: determining whether the number of moving particle points corresponding to the interaction power greater than a preset power threshold is greater than a preset number threshold. If so, the current image frame belongs to an image frame with a crowd abnormal event. If not, the current image frame does not belong to an image frame with a crowd abnormal event. Among them, by way of example, the preset power threshold can be 3.

[0078] Figure 3 It is a schematic diagram of interaction power corresponding to video data provided according to Embodiment 1 of the present invention. Specifically, Figure 3 The above two pictures respectively represent a certain image frame in video data A and the interaction power curve corresponding to video data A. Figure 3 The following two pictures respectively represent a certain image frame in video data B and the interaction power curve corresponding to video data B. Among them, the abscissa of the interaction power curve represents the number of image frames of the video data, and the ordinate represents the interaction power. From Figure 3 It can be seen that the crowd behavior in the image frame of video data A is stable. Correspondingly, the interaction power corresponding to video data A tends to be stable, and the parameter values of the interaction power are generally small. While the crowd in the image frame of video data B shows obvious violent movement behaviors. Correspondingly, the interaction power corresponding to video data B is unstable, and the parameter values of the interaction power are generally large. From the distribution of the interaction power of the entire video data, the interaction power corresponding to the video data of abnormal crowds (video data B) is significantly greater than the interaction power corresponding to the video data of normal crowds (video data A).

[0079] From Figure 3It can be seen that under the same scenario, the difference in the mean values of the interaction power corresponding to the video data of the normal population and the interaction power corresponding to the video data of the abnormal population is relatively obvious. Therefore, in another embodiment, optionally, based on at least one interaction power, determining whether the current image frame belongs to an image frame with a group abnormal event includes: based on at least one interaction power, determining the mean value of the interaction power corresponding to the current image frame and the power mean value threshold; based on the mean value of the interaction power and the power mean value threshold, determining whether the current image frame belongs to an image frame with a group abnormal event.

[0080] In one embodiment, optionally, based on at least one interaction power, determining the mean value of the interaction power corresponding to the current image frame and the power mean value threshold includes: averaging the interaction powers corresponding to all moving particle points to obtain the mean value of the interaction power; using the median value corresponding to at least one interaction power as the power mean value threshold.

[0081] Specifically, if the mean value of the interaction power is greater than the power mean value threshold, then the current image frame is used as an image frame with a group abnormal event, that is, an abnormal image frame; if the mean value of the interaction power is less than or equal to the power mean value threshold, then the current image frame is used as an image frame without a group abnormal event, that is, a normal image frame.

[0082] The technical solution of this embodiment, for each image frame in the video data, obtains the image optical flow field data and at least one object area image corresponding to the current image frame. For each object area image, based on the image optical flow field data corresponding to the current image frame, respectively determines the interaction power corresponding to at least one moving particle point in the object area image, and based on at least one interaction power, determines whether the current image frame belongs to an image frame with a group abnormal event, solving the problem of crowd feature extraction in scenarios with large crowd density, serious crowd occlusion, and complex scenes, and improving the detection efficiency and accuracy of the detection effect of group abnormal events.

[0083] Embodiment 2

[0084] Figure 4 is a flowchart of a method for detecting group abnormal events provided in Embodiment 2 of the present invention. This embodiment further refines the technical feature of "based on the mean value of the interaction power and the power mean value threshold, determining whether the current image frame belongs to an image frame with a group abnormal event" in the above embodiment. As Figure 4 shown, the method includes:

[0085] S210. For each image frame in the video data, obtain the image optical flow field data and at least one object area image corresponding to the current image frame.

[0086] S220. For each object region image, determine at least one moving particle point corresponding to the object region image.

[0087] S230. For each moving particle point, determine the interaction power corresponding to the moving particle point based on the image optical flow field data.

[0088] S240. Based on at least one interaction power, determine the mean interaction power and the power mean threshold corresponding to the current image frame.

[0089] Specifically, for each image frame of the video data, at least one interaction power corresponding to the current image frame constitutes an interaction power set S = {pw i}(i = 1...n), where pw i represents the interaction power corresponding to the i-th moving particle point in the current image frame, and n represents the number of moving particle points included in the current image frame. Perform a histogram statistics on the interaction power set S, find the upper limit value T of the distribution of the interaction power values exceeding more than half in the histogram, and round up or down the upper limit value T to obtain the power mean threshold T0 = [T]. At the same time, perform a mean value processing on the interaction power set S to obtain the mean interaction power corresponding to the current image frame.

[0090] S250. Judge whether the mean interaction power is greater than the power mean threshold. If so, execute S270; if not, execute S260.

[0091] Specifically, judge whether the mean interaction power pw mean is greater than the power mean threshold T0.

[0092] S260. Take the current image frame as the image frame with a population anomaly event.

[0093] S270. Based on the difference between the mean interaction power and the standard mean interaction power, determine whether the current image frame belongs to the image frame with a population anomaly event.

[0094] In one embodiment, optionally, the standard mean interaction power can be preset by the user. Specifically, if the difference is greater than the preset difference threshold, take the current image frame as the image frame with a population anomaly event; if the difference is less than or equal to the preset difference threshold, take the current image frame as the image frame without a population anomaly event.

[0095] The mean interaction power dataset is constructed based on the mean interaction power corresponding to the image frames without population anomaly events in the video data. Specifically, initialize the mean interaction power dataset based on the mean interaction power corresponding to the first K image frames without population anomaly events in the video data. Wherein, Denote the interactive power mean data set as S pw which is the interactive power mean corresponding to the j-th image frame in

[0096] Among them, specifically, the standard interactive power mean P mean satisfies the formula:

[0097]

[0098] Based on the above embodiments, the method further includes: if the image frame belongs to the image frame with a group abnormal event, the interactive power mean data set corresponding to the previous image frame is the interactive power mean data set corresponding to the current image frame.

[0099] Based on the above embodiments, optionally, before performing the histogram statistics on the interactive power set S, the method further includes: randomly sampling the interactive power set corresponding to the current image frame with a fixed size N to obtain the interactive power set S N . Among them, specifically, the interactive power set S N ={pw i}(i = 1...N).

[0100] The advantage of this setting is that it can ensure that the size of the interactive power set corresponding to the interactive power mean in the interactive power mean data set is the same, and improve the stability of the standard interactive power mean.

[0101] Figure 5 is a flowchart of a method for determining an adaptive standard interactive power mean according to Embodiment 2 of the present invention. Specifically, based on the interactive power set corresponding to the current image frame, random sampling is performed to obtain the interactive power set S N , perform histogram statistics on the interactive power set S N , find the upper limit value T of the distribution of the interactive power values exceeding more than half in the histogram, and round up or down the upper limit value T to obtain the power mean threshold T0 = [T]. At the same time, perform a mean value processing on the interactive power set S N to obtain the interactive power mean corresponding to the current image frame Judge whether the interactive power mean pw mean is greater than the power mean threshold T0. If so, regard the current image frame as the image frame with a group abnormal event. If not, judge whether the difference between the interactive power mean and the standard interactive power mean is less than the preset difference threshold ε. Among them, the standard interactive power mean is determined based on the interactive power mean data set (normal population sequence) corresponding to the current image frame. Specifically, the standard interactive power mean P mean satisfies the formula:

[0102] If the difference between the interaction power mean and the standard interaction power mean is greater than or equal to the preset difference threshold ε, the current image frame is regarded as an image frame with a group abnormal event, that is, an abnormal image frame. If the difference between the interaction power mean and the standard interaction power mean is less than the preset difference threshold ε, the current image frame is regarded as an image frame without a group abnormal event, that is, a normal image frame. Further, based on the interaction power mean corresponding to the current image frame, the interaction power mean in the interaction power mean data set S pw in is randomly updated to obtain the interaction power mean data set corresponding to the next image frame.

[0103] Figure 6 is a flowchart of a specific example of a method for detecting group abnormal events provided in Embodiment 2 of the present invention. Specifically, input video data. For each image frame in the video data, the VIBE algorithm is used to extract the foreground target area in the current image frame. Based on the LDOF algorithm, the image optical flow field data of the foreground target area is determined. Based on the HOG feature and the SVM classifier, at least one object area image in the foreground target area is extracted. For each object area image, at least one moving particle point corresponding to the object area image is determined. Based on the proposed interaction power model, the interaction power corresponding to each moving particle point is calculated, and an adaptive interaction power mean data set is constructed. Based on the power mean threshold and the interaction power mean data set, it is determined whether the current image frame is abnormal. If so, the current image frame is regarded as an image frame with a group abnormal event, that is, an image frame in an abnormal crowd scene. If not, the current image frame is regarded as an image frame without a group abnormal event, that is, an image frame in a normal crowd scene.

[0104] The technical solution of this embodiment constructs a power mean data set, and in the process of detecting group abnormal events, the interaction power mean in the interaction power mean data set is randomly replaced in real time based on the interaction power mean corresponding to the image frame without abnormal group abnormal events, so as to realize the criterion for unsupervised and adaptive determination of the standard interaction power mean, solve the problem that the fixed standard interaction power mean makes the detection result inaccurate, and is different from the traditional group abnormal detection method based on a classifier. While ensuring the accuracy of the detection result, it saves the cost and time of training the abnormal detection classifier.

[0105] Embodiment 3

[0106] Figure 7 is a schematic structural diagram of a device for detecting group abnormal events provided in Embodiment 3 of the present invention. As Figure 7 shown, the device includes: an object area image acquisition module 310, a moving particle point determination module 320, an interaction power determination module 330, and a group abnormal event determination module 340.

[0107] Among them, the object area image acquisition module 310 is used to obtain the image optical flow field data and at least one object area image corresponding to the current image frame for each image frame in the video data;

[0108] The moving particle point determination module 320 is used to determine at least one moving particle point corresponding to the object area image for each object area image; among them, the moving particle point is used to represent the moving unit corresponding to the object area image;

[0109] The interaction power determination module 330 is used to determine the interaction power corresponding to the moving particle point based on the image optical flow field data for each moving particle point; among them, the interaction power is used to represent the motion stability state of the moving particle point;

[0110] The determination module 340 of the group abnormal event is used to determine whether the current image frame belongs to the image frame with a group abnormal event based on at least one interaction power.

[0111] The technical solution of this embodiment solves the problem of crowd feature extraction in scenarios with large crowd density, serious crowd occlusion, and complex scenes by obtaining the image optical flow field data and at least one object area image corresponding to the current image frame for each image frame in the video data, determining the interaction power corresponding to at least one moving particle point in the object area image based on the image optical flow field data corresponding to the current image frame for each object area image, and determining whether the current image frame belongs to the image frame with a group abnormal event based on at least one interaction power, improving the detection efficiency of the group abnormal event and the accuracy of the detection effect.

[0112] On the basis of the above embodiment, optionally, the interaction power determination module 330 is specifically used for:

[0113] Determine the actual motion speed of the moving particle point based on the image block corresponding to the moving particle point and the image optical flow field data corresponding to the image block;

[0114] Determine the expected motion speed of the moving particle point based on the panic weight coefficient and the image optical flow field data corresponding to the moving particle point;

[0115] Determine the interaction power corresponding to the moving particle point based on the actual motion speed and the expected motion speed.

[0116] On the basis of the above embodiment, optionally, the interaction power satisfies the formula:

[0117]

[0118] Among them, P int represents the interaction power, F intrepresents the interaction force, v i represents the actual motion speed of the i-th moving particle point, τ represents the relaxation coefficient, represents the desired motion speed.

[0119] Based on the above embodiments, optionally, the group abnormal event determination module 340 includes:

[0120] A power mean threshold determination unit, configured to determine the interaction power mean corresponding to the current image frame and the power mean threshold based on at least one interaction power;

[0121] A group abnormal event determination unit, configured to determine whether the current image frame belongs to an image frame with a group abnormal event based on the interaction power mean and the power mean threshold.

[0122] Based on the above embodiments, optionally, the group abnormal event determination unit is specifically configured to:

[0123] If the interaction power mean is greater than the power mean threshold, then regard the current image frame as an image frame with a group abnormal event;

[0124] If the interaction power mean is less than or equal to the power mean threshold, then determine whether the current image frame belongs to an image frame with a group abnormal event based on the difference between the interaction power mean and the standard interaction power mean.

[0125] Based on the above embodiments, optionally, the device further includes:

[0126] A standard interaction power mean determination module, configured to obtain the interaction power

[0127] mean data set corresponding to the previous image frame; the interaction power mean data set includes at least one interaction power mean corresponding to an image frame without an abnormal group abnormal event;

[0128] If the previous image frame does not belong to an image frame with a group abnormal event, then based on the interaction power mean corresponding to the previous image frame, replace any one of the interaction power means in the interaction power mean data set to obtain the interaction power mean data set corresponding to the current image frame;

[0129] Based on the interaction power mean data set corresponding to the current image frame, determine the standard interaction power mean.

[0130] Based on the above embodiments, optionally, the object area image acquisition module 310 is specifically configured to:

[0131] Based on a preset image processing algorithm, determine the foreground target area in the current image frame;

[0132] Based on a preset optical flow algorithm, image optical flow field data corresponding to the foreground target area is determined, and based on a preset object detection algorithm, at least one object area image corresponding to the foreground target area is determined.

[0133] The detection device for group abnormal events provided by the embodiments of the present invention can execute the detection method for group abnormal events provided by any embodiment of the present invention, and has the corresponding functional modules and beneficial effects for executing the method.

[0134] Embodiment Four

[0135] Figure 8 It is a schematic structural diagram of an electronic device according to Embodiment Four of the present invention. The electronic device 10 is intended to represent various forms of digital computers, such as, laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as, personal digital processors, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present invention described and / or claimed herein.

[0136] As Figure 8 shown, the electronic device 10 includes at least one processor 11, and a memory communicatively connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc. Among them, the memory stores a computer program executable by the at least one processor. The processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 into the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. The input / output (I / O) interface 15 is also connected to the bus 14.

[0137] Multiple components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a magnetic disk, an optical disc, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0138] The processor 11 may be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as the method for detecting group abnormal events.

[0139] In some embodiments, the method for detecting group abnormal events may be implemented as a computer program tangibly embodied in a computer-readable storage medium, such as the storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed onto the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the method for detecting group abnormal events described above may be executed. Alternatively, in other embodiments, the processor 11 may be configured to execute the method for detecting group abnormal events by any other suitable means (e.g., by means of firmware).

[0140] Various embodiments of the systems and techniques described above herein may be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-chip (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include: implemented in one or more computer programs that may be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a special-purpose or general-purpose programmable processor that receives data and instructions from a storage system, at least one input device, and at least one output device, and transmits the data and instructions to the storage system, the at least one input device, and the at least one output device.

[0141] The computer program for implementing the method for detecting group abnormal events of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to the processors of general-purpose computers, special-purpose computers, or other programmable data processing devices, such that when the computer programs are executed by the processors, the functions / operations specified in the flowcharts and / or block diagrams are implemented. The computer programs may be executed entirely on the machine, partially on the machine, as a stand-alone software package partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0142] Example 5

[0143] Example 5 of the present invention also provides a computer-readable storage medium storing computer instructions for causing a processor to execute a method for segmenting medical images, the method comprising:

[0144] For each image frame in the video data, obtaining the image optical flow field data and at least one object region image corresponding to the current image frame;

[0145] For each object region image, determining at least one motion particle point corresponding to the object region image; wherein the motion particle point is used to characterize the motion unit corresponding to the object region image;

[0146] For each motion particle point, determining the interaction power corresponding to the motion particle point based on the image optical flow field data; wherein the interaction power is used to characterize the motion stability state of the motion particle point;

[0147] Based on at least one interaction power, determining whether the current image frame belongs to an image frame with a population anomaly event.

[0148] In the context of the present invention, a computer-readable storage medium may be a tangible medium that can contain or store a computer program for use by or in connection with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media would include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0149] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the electronic device. Other kinds of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0150] The systems and techniques described herein can be implemented in a computing system including backend components (e.g., as a data server), or a computing system including middleware components (e.g., an application server), or a computing system including frontend components (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described herein), or a computing system including any combination of such backend components, middleware components, or frontend components. The components of the system can be interconnected to each other by digital data communication in any form or medium (e.g., a communication network). Examples of communication networks include: local area network (LAN), wide area network (WAN), blockchain network, and the Internet.

[0151] The computing system can include a client and a server. The client and the server are generally far from each other and typically interact through a communication network. The client-server relationship is created by computer programs running on respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or a cloud host, which is a host product in the cloud computing service system and solves the defects of difficult management and weak business scalability existing in traditional physical hosts and VPS services.

[0152] It should be understood that various forms of the processes shown above can be used, with steps reordered, added, or deleted. For example, the steps recited in the present invention can be executed in parallel, sequentially, or in a different order, as long as the desired results of the technical solution of the present invention can be achieved, and no limitation is made herein.

[0153] The above specific implementation manners do not constitute a limitation on the protection scope of the present invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. A detection method for group abnormal events, characterized in that, Including: For each image frame in the video data, obtain the image optical flow field data and at least one object region image corresponding to the current image frame; For each object region image, determine at least one motion particle point corresponding to the object region image; wherein, the motion particle point is used to characterize the motion unit corresponding to the object region image; For each motion particle point, based on the image optical flow field data, determine the interaction power corresponding to the motion particle point; wherein, the interaction power is used to characterize the motion stability state of the motion particle point; Based on at least one interaction power, determine whether the current image frame belongs to an image frame with a crowd abnormal event; The determining whether the current image frame belongs to an image frame with a crowd abnormal event based on at least one interaction power includes: Based on at least one interaction power, determine the interaction power mean value and the power mean value threshold corresponding to the current image frame; If the interaction power mean value is greater than the power mean value threshold, then regard the current image frame as an image frame with a crowd abnormal event; If the interaction power mean value is less than or equal to the power mean value threshold, then based on the difference between the interaction power mean value and the standard interaction power mean value, determine whether the current image frame belongs to an image frame with a crowd abnormal event; Wherein, the standard interaction power mean value is determined based on the interaction power mean value dataset corresponding to the current image frame, and the interaction power mean value dataset is constructed according to the interaction power mean values corresponding to the image frames without crowd abnormal events in the video data.

2. The method according to claim 1, characterized in that, The determining the interaction power corresponding to the motion particle point based on the image optical flow field data includes: Based on the image block corresponding to the motion particle point and the image optical flow field data corresponding to the image block, determine the actual motion speed of the motion particle point; Based on the panic weight coefficient and the image optical flow field data corresponding to the motion particle point, determine the expected motion speed of the motion particle point; Based on the actual motion speed and the expected motion speed, determine the interaction power corresponding to the motion particle point.

3. The method according to claim 2, characterized in that, The interaction power satisfies the formula: Among them, P int represents the interaction power, F int represents the interaction force, v i represents the actual motion speed of the i-th moving particle point, and τ represents the relaxation coefficient, represents the desired motion speed.

4. The method according to claim 1, wherein The method further includes: Obtain the interaction power mean value dataset corresponding to the previous image frame; the interaction power mean value dataset includes the interaction power mean values corresponding to at least one image frame without abnormal crowd abnormal events; If the previous image frame does not belong to an image frame with a crowd abnormal event, then based on the interaction power mean value corresponding to the previous image frame, replace any one of the interaction power mean values in the interaction power mean value dataset to obtain the interaction power mean value dataset corresponding to the current image frame; Based on the interaction power mean value dataset corresponding to the current image frame, determine the standard interaction power mean value.

5. The method according to any one of claims 1-4, characterized in that, The obtaining the image optical flow field data and at least one object region image corresponding to the current image frame includes: Based on a preset image processing algorithm, determine the foreground target region in the current image frame; Based on a preset optical flow algorithm, determine the image optical flow field data corresponding to the foreground target region, and based on a preset object detection algorithm, determine at least one object region image corresponding to the foreground target region.

6. A detection device for group abnormal events, characterized in that, Including: An object region image acquisition module, configured to acquire image optical flow field data and at least one object region image corresponding to a current image frame for each image frame in video data; A moving particle point determination module, configured to determine at least one moving particle point corresponding to the object region image for each object region image; wherein, the moving particle point is used to represent a moving unit corresponding to the object region image; An interaction power determination module, configured to determine an interaction power corresponding to the moving particle point based on the image optical flow field data for each moving particle point; wherein, the interaction power is used to represent a motion stability state of the moving particle point; A determination module for group abnormal events, configured to determine whether the current image frame belongs to an image frame with a group abnormal event based on at least one interaction power; The determination module for group abnormal events includes: A power mean threshold determination unit, configured to determine an interaction power mean and a power mean threshold corresponding to the current image frame based on at least one interaction power; A determination unit for group abnormal events, specifically configured to, if the interaction power mean is greater than the power mean threshold, regard the current image frame as an image frame with a group abnormal event; If the interaction power mean is less than or equal to the power mean threshold, determine whether the current image frame belongs to an image frame with a group abnormal event based on a difference between the interaction power mean and a standard interaction power mean; wherein, the standard interaction power mean is determined based on an interaction power mean data set corresponding to the current image frame, and the interaction power mean data set is constructed according to interaction power means corresponding to image frames without group abnormal events in the video data.

7. An electronic device, characterized in that, The electronic device includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the detection method for group abnormal events according to any one of claims 1-5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions, and the computer instructions are used to implement the detection method for group abnormal events according to any one of claims 1-5 when executed by a processor.

Images