Abnormal event detection method, device, electronic device and storage medium

Abstract

The present disclosure provides an abnormal event detection method, device, electronic device and storage medium, the method comprising: acquiring an image sequence, and determining N image area sets from the image sequence; each image area set includes an image area in each frame image in the image sequence, and each image area has the same order of pedestrian density among the image areas in the frame image to which it belongs; N is a natural number greater than or equal to 1; for each image area set, a central area is selected based on the spatial distribution of the image areas in the set; for each central area, an image area at a corresponding position is extracted from each frame image of the image sequence to form a trajectory sequence; and abnormal event detection is performed using each trajectory sequence respectively.

Description

Technical Field

[0001] The present disclosure relates to the field of computer vision technology, and in particular to an abnormal event detection method, device, electronic device and storage medium. Background Art

[0002] Abnormal event detection in videos is an important issue in the field of computer vision. Currently, computer vision and deep learning techniques can be used to automatically detect abnormal events occurring in videos.

[0003] The traditional method of detecting abnormal events is to perform data enhancement or other preprocessing on the entire input video sequence and then input it into a classification model for detection, or to roughly locate the center of the event based on the dense distribution of the crowd. This detection method has low efficiency and poor accuracy. Summary of the invention

[0004] The embodiments of the present disclosure are intended to provide a method, device, electronic device and storage medium for detecting an abnormal event.

[0005] The technical solution of the embodiment of the present disclosure is implemented as follows:

[0006] The present disclosure provides a method for detecting abnormal events, the method comprising:

[0007] Acquire an image sequence, and determine N image region sets from the image sequence; each image region set includes an image region in each frame of the image sequence, and the order of pedestrian density among the image regions in the frame of the image to which each image region belongs is the same; N is a natural number greater than or equal to 1;

[0008] For each image region set, a central region is selected based on the spatial distribution of the image regions in the set;

[0009] For each central area, extract an image area at a corresponding position from each frame of the image sequence to form a trajectory sequence;

[0010] Abnormal event detection is performed using each trajectory sequence separately.

[0011] In the above method, determining N sets of image regions from the image sequence includes:

[0012] For each frame of the image sequence, N image regions are sequentially determined according to the pedestrian density from large to small to form a group of image regions;

[0013] In the image region groups corresponding to each frame of the image sequence, the image regions with the same pedestrian density ranking are divided into the same set to obtain the N image region sets.

[0014] In the above method, for each frame of the image sequence, N image regions are sequentially determined according to the pedestrian density from large to small to form a group of image regions, including:

[0015] For each frame of the image sequence, each pedestrian detection frame is obtained, and each pedestrian detection frame is expanded into an extended detection frame to form a group of extended detection frames;

[0016] For each group of extended detection frames, determining the overlap ratios between different extended detection frames included therein, and constructing a corresponding first adjacency matrix according to the obtained overlap ratios;

[0017] For each group of extended detection frames, based on the corresponding first adjacency matrix, determine the number of matches of each included extended detection frame, and select N extended detection frames in descending order of the number of matches to form a group of central detection frames;

[0018] For each frame of the image sequence, a group of image regions is determined based on a corresponding group of center detection frames.

[0019] In the above method, for each frame of the image sequence, each pedestrian detection frame is obtained and expanded into an extended detection frame respectively, forming a group of extended detection frames, including:

[0020] Obtain at least one pedestrian detection frame from the i-th frame image;

[0021] In the i-th frame image, each pedestrian detection frame in the at least one pedestrian detection frame is taken as a center, and is expanded outward according to a first preset ratio to form an extended detection frame;

[0022] The extended detection frames obtained by the outward expansion are combined into a group of extended detection frames corresponding to the i-th frame image;

[0023] The i-th frame image is any frame image in the image sequence, and i is a natural number greater than or equal to 1.

[0024] In the above method, for each frame of the image sequence, determining a set of image regions based on a corresponding set of center detection frames includes:

[0025] From the k-th frame image, obtain N pedestrian detection frames corresponding to the N extended detection frames constituting a corresponding set of central detection frames;

[0026] From the k-th frame image, taking each of the N pedestrian detection frames as a center, and expanding outwards according to a second preset ratio to form an image region;

[0027] The image regions obtained by the outward expansion are combined into a group of image regions corresponding to the k-th frame image;

[0028] The k-th frame image is any frame image in the image sequence, and k is a natural number greater than or equal to 1.

[0029] In the above method, for each image region set, selecting a central region based on the spatial distribution of the image regions in the set includes:

[0030] For each image region set, determining the overlap ratio between different image regions in the set, and constructing a corresponding second adjacency matrix according to the obtained overlap ratio, to obtain N second adjacency matrices;

[0031] For each set of image regions, determining the number of matches for each image region in the set based on the corresponding second adjacency matrix;

[0032] From each image region set, the image region with the largest number of matches is selected as the central region.

[0033] In the above method, the abnormal event detection is performed using each trajectory sequence separately, including:

[0034] Using the preset abnormal event detection model, abnormal event detection is performed on each trajectory sequence to obtain the corresponding abnormal event detection results.

[0035] The present disclosure provides an abnormal event detection device, the device comprising:

[0036] A region determination module is used to obtain an image sequence and determine N image region sets from the image sequence; each image region set includes an image region in each frame of the image sequence, and the order of pedestrian density among the image regions in the frame of the image to which each image region belongs is the same; N is a natural number greater than or equal to 1;

[0037] A region selection module is used to select a central region for each image region set based on the spatial distribution of the image regions in the set;

[0038] A sequence determination module is used to extract, for each central area, an image area at a corresponding position from each frame image of the image sequence to form a trajectory sequence;

[0039] The anomaly detection module is used to detect abnormal events using each trajectory sequence separately.

[0040] In the above-mentioned device, the area determination module is specifically used to determine, for each frame image of the image sequence, N image areas in sequence according to the pedestrian density from large to small to form a group of image areas; in the image area groups corresponding to each frame image in the image sequence, the image areas with the same pedestrian density ranking are divided into the same set to obtain the N image area sets.

[0041] In the above-mentioned device, the area determination module is specifically used to obtain each pedestrian detection frame for each frame image of the image sequence, and expand them into an extended detection frame respectively to form a group of extended detection frames; for each group of extended detection frames, determine the overlap ratio between the different extended detection frames included, and construct a corresponding first adjacency matrix based on the obtained overlap ratio; for each group of extended detection frames, determine the number of matches of each extended detection frame included based on the corresponding first adjacency matrix, and select N extended detection frames in order from large to small according to the number of matches to form a group of central detection frames; for each frame image of the image sequence, determine a group of image areas based on the corresponding group of central detection frames.

[0042] In the above-mentioned device, the area determination module is specifically used to obtain at least one pedestrian detection frame from the i-th frame image; in the i-th frame image, each pedestrian detection frame in the at least one pedestrian detection frame is taken as the center, and expanded outward according to a first preset ratio to form an extended detection frame; the extended detection frames obtained by the expansion are combined into a group of extended detection frames corresponding to the i-th frame image; wherein the i-th frame image is any frame image in the image sequence, and i is a natural number greater than or equal to 1.

[0043] In the above-mentioned device, the area determination module is specifically used to obtain N pedestrian detection frames corresponding to N extended detection frames constituting a corresponding group of central detection frames from the k-th frame image; from the k-th frame image, take each pedestrian detection frame in the N pedestrian detection frames as the center, and expand outward according to a second preset ratio to form an image area; the image areas obtained by the expansion form a group of image areas corresponding to the k-th frame image; wherein the k-th frame image is any frame image in the image sequence, and k is a natural number greater than or equal to 1.

[0044] In the above-mentioned device, the region selection module is specifically used to determine, for each image region set, the overlap ratio between different image regions in the set, and construct a corresponding second adjacency matrix based on the obtained overlap ratio; for each image region set, based on the corresponding second adjacency matrix, determine the number of matches for each image region in the set; and from each image region set, select the image region with the largest number of matches as the central region.

[0045] In the above device, the abnormality detection module is specifically used to use a preset abnormal event detection model to perform abnormal event detection on each trajectory sequence respectively to obtain a corresponding abnormal event detection result.

[0046] The present disclosure provides an electronic device, the electronic device comprising: a processor, a memory and a communication bus; wherein:

[0047] The communication bus is used to realize the connection and communication between the processor and the memory;

[0048] The processor is used to execute one or more programs stored in the memory to implement the above-mentioned abnormal event detection method.

[0049] An embodiment of the present disclosure provides a computer-readable storage medium, which stores one or more programs. The one or more programs can be executed by one or more processors to implement the above-mentioned abnormal event detection method.

[0050] The disclosed embodiments provide a method, device, electronic device and storage medium for detecting abnormal events, the method comprising: obtaining an image sequence, and determining N sets of image regions from the image sequence; each set of image regions includes an image region in each frame of the image sequence, and each image region has the same order of pedestrian density among the image regions in the frame of the image to which it belongs; N is a natural number greater than or equal to 1; for each set of image regions, a central region is selected based on the spatial distribution of the image regions in the set; for each central region, the image region at the corresponding position is extracted from each frame of the image sequence to form a trajectory sequence; and each trajectory sequence is used to perform abnormal event detection respectively. The technical solution provided by the disclosed embodiments not only reduces the amount of data for abnormal event detection and improves detection efficiency through a heuristic pedestrian dense group position estimation method based on center point consistency constraints, but also provides an effective perception field for abnormal event detection and improves detection accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 A flowchart of an abnormal event detection method provided by an embodiment of the present disclosure;

[0052] Figure 2 A schematic diagram of a process for determining an image region group provided by an embodiment of the present disclosure;

[0053] Figure 3 A schematic diagram of an exemplary trajectory sequence provided for an embodiment of the present disclosure;

[0054] Figure 4 A schematic diagram of the structure of an abnormal event detection device provided in an embodiment of the present disclosure;

[0055] Figure 5 A schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0056] The technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure.

[0057] The embodiments of the present disclosure provide an abnormal event detection method, the execution subject of which may be an abnormal event detection device, for example, the abnormal event detection method may be executed by a terminal device or a server or other electronic device, wherein the terminal device may be a user equipment (UE), a mobile device, a user terminal, a terminal, a cellular phone, a cordless phone, a personal digital assistant (PDA), a handheld device, a computing device, a vehicle-mounted device, a wearable device, etc. In some possible implementations, the abnormal event detection method may be implemented by a processor calling a computer-readable instruction stored in a memory.

[0058] Figure 1 The following is a flow chart of an abnormal event detection method provided by an embodiment of the present disclosure. Figure 1 As shown, in the embodiment of the present disclosure, the abnormal event detection method mainly includes the following steps:

[0059] S101, obtaining an image sequence, and determining N image region sets from the image sequence; each image region set includes an image region in each frame of the image sequence, and the order of pedestrian density among each image region in the frame of the image to which it belongs is the same; N is a natural number greater than or equal to 1.

[0060] In an embodiment of the present disclosure, the abnormal event detection device may first acquire an image sequence, thereby determining N image region sets from the image sequence.

[0061] It should be noted that in the embodiments of the present disclosure, the abnormal event detection device may include an image acquisition device, so as to use the image acquisition device to acquire continuous frame images of a scene to obtain an image sequence. Of course, the image sequence may also be acquired by an independent camera device, for example, a camera, and then transmitted to the abnormal event detection device. The specific method of acquiring the image sequence and the image sequence can be determined according to actual needs and application scenarios, and the embodiments of the present disclosure are not limited thereto.

[0062] It should be noted that in the embodiment of the present disclosure, the abnormal event detection device determines N image area sets, each of which includes an image area in each frame of the image sequence, and the order of pedestrian density among each image area in the frame of the image to which it belongs is the same.

[0063] Specifically, in an embodiment of the present disclosure, the abnormal event detection device determines N image area sets from an image sequence, including: for each frame image of the image sequence, N image areas are determined in sequence from large to small according to the pedestrian density to form a group of image areas; in the image area groups corresponding to each frame image in the image sequence, image areas with the same ranking of pedestrian density are divided into the same set to obtain N image area sets.

[0064] It should be noted that in the embodiments of the present disclosure, the image sequence includes multiple frames of images arranged in sequence in time. For each frame of the image, the abnormal event detection device can determine N image areas in sequence according to the pedestrian density from large to small, and then use the N image areas as a group of image areas corresponding to the frame of the image, thereby finally determining multiple groups of image areas.

[0065] It can be understood that in the embodiments of the present disclosure, the abnormal event detection device divides the image areas with the same pedestrian density ranking in different groups into the same set for multiple groups of image areas, that is, the image area with the first ranking of pedestrian density in each group of multiple groups of image areas is selected and divided into the same set, and so on, until the image with the last ranking of pedestrian density in each group is selected and divided into the same set, and since each group of image areas includes N image areas, the number of image sets finally divided is also N.

[0066] Figure 2 A schematic diagram of a process for determining an image region group provided by an embodiment of the present disclosure. Figure 2 As shown, in the embodiment of the present disclosure, the following steps are mainly included:

[0067] S201. For each frame of the image sequence, obtain each pedestrian detection frame, and expand each of them into an extended detection frame to form a group of extended detection frames.

[0068] It should be noted that in the embodiments of the present disclosure, the abnormal event detection device can perform pedestrian detection for each frame of the image sequence, thereby generating a pedestrian detection frame for each detected pedestrian, and then expanding each pedestrian detection frame to form an extended detection frame. All extended detection frames in a frame of the image constitute a set of extended detection frames corresponding to the frame of the image. Since the image sequence includes multiple frames of images, the abnormal event detection device determines a set of extended detection frames corresponding to each frame of the image in the above manner.

[0069] Specifically, in an embodiment of the present disclosure, the abnormal event detection device obtains each pedestrian detection frame for each frame image of the image sequence, and expands them into an extended detection frame respectively to form a group of extended detection frames, including: obtaining at least one pedestrian detection frame from the i-th frame image; in the i-th frame image, taking each pedestrian detection frame in the at least one pedestrian detection frame as the center, and expanding them outward according to a first preset ratio to form an extended detection frame; the extended detection frames obtained by the expansion are combined into a group of extended detection frames corresponding to the i-th frame image; wherein the i-th frame image is any frame image in the image sequence, and i is a natural number greater than or equal to 1.

[0070] It should be noted that in the embodiments of the present disclosure, for any frame of image in the image sequence, i.e., the i-th frame of image, the abnormal event detection device can obtain at least one pedestrian detection frame therefrom, and then in the i-th frame of image, each obtained pedestrian detection frame is taken as the center and expanded at a certain ratio. In this way, the obtained expanded detection frame not only includes the pedestrians included in the pedestrian detection frame, but also includes the image information around the pedestrians, for example, people or objects around the pedestrians. This facilitates the subsequent evaluation of information such as the crowd density at the corresponding position, making the subsequent image area determination more accurate. With regard to the expansion method of the pedestrian detection frame, the abnormal event detection device can also directly expand each pedestrian detection frame to a preset size, that is, the size of each obtained expanded detection frame is the same preset size. The abnormal event detection device can also expand in different ways based on the positions of different pedestrian detection frames in the corresponding frame of image, which is not limited by the embodiments of the present disclosure.

[0071] It should be noted that in the embodiments of the present disclosure, the first preset ratio can be set according to actual needs and application scenarios, and the embodiments of the present disclosure are not limited. For example, the first preset ratio can be 1.5 times, that is, with the pedestrian detection frame as the center, the detection frame is expanded 1.5 times outward in the image, and the expanded detection frame is the extended detection frame.

[0072] S202: For each group of extended detection frames, determine the overlap ratios between different extended detection frames included therein, and construct a corresponding first adjacency matrix according to the obtained overlap ratios.

[0073] In an embodiment of the present disclosure, after obtaining a group of extended detection frames for each frame image, the abnormal event detection device can determine the overlap ratio between different extended detection frames contained in each group of extended detection frames, and construct a corresponding first adjacency matrix based on the obtained overlap ratio.

[0074] It should be noted that, in the embodiments of the present disclosure, for each group of extended detection frames, the abnormal event detection device can calculate the overlap ratio between each extended detection frame and other extended detection frames in the group, thereby constructing a corresponding first adjacency matrix, which actually records the overlap between each pair of extended detection frames in the group.

[0075] It should be noted that in the embodiment of the present disclosure, the abnormal event detection device constructs a corresponding first adjacency matrix for each group of extended detection frames according to the obtained overlap ratios between different extended detection frames, which actually takes each overlap ratio as an element in the first adjacency matrix.

[0076] S203. For each group of extended detection frames, based on the corresponding first adjacency matrix, determine the number of matches of each included extended detection frame, and select N extended detection frames in descending order of the number of matches to form a group of central detection frames.

[0077] In an embodiment of the present disclosure, after the abnormal event detection device determines the corresponding first adjacency matrix for each group of extended detection frames, it can determine the number of matches of each extended detection frame in the group based on the first adjacency matrix, and select N extended detection frames in order from large to small according to the number of matches to form a group of central detection frames.

[0078] It can be understood that in the embodiments of the present disclosure, for each group of extended detection frames, the corresponding first adjacency matrix includes the overlap ratio information between different extended detection frames therein, so the abnormal event detection device can directly know from the first adjacency matrix whether each extended detection frame overlaps with other extended detection frames in the group. For example, for an extended detection frame, if the overlap ratio between it and an extended detection frame in the group included in the first adjacency matrix is ​​0, it indicates that there is no overlap with the extended detection frame. If the overlap ratio is not 0, it indicates that there is overlap with the extended detection frame. If an extended detection frame overlaps with another extended detection frame, the number of matches is recorded as one, so that the number of matches of the extended detection frame can be determined.

[0079] In an embodiment of the present disclosure, when the abnormal event detection device determines the number of matches of each extended detection frame in a group of extended detection frames, it can select extended detection frames according to the number of matches from large to small, and form the selected N extended detection frames into a group of central detection frames.

[0080] It can be understood that in the embodiments of the present disclosure, in each group of extended detection frames, since each extended detection frame is actually obtained by expanding the pedestrian detection frame, that is, it includes pedestrians, if an extended detection frame has a large number of matches, that is, it overlaps with more extended detection frames, then there will be more pedestrians distributed around the extended detection frame, and the location of the extended detection frame is actually the center of a pedestrian-dense area. Therefore, the abnormal event detection device selects it from the group of extended detection frames, so that the area with a high pedestrian density can be accurately determined from the frame image to which it belongs based on the extended detection frame.

[0081] It should be noted that in the embodiments of the present disclosure, for a group of extended detection frames, there may be a situation where some of the extended detection frames have the same number of matches. When the abnormal event detection device selects according to the number of matches from large to small, it can further select from some of the extended detection frames with the same number of matches based on other information. For example, the area of ​​each extended detection frame in the partial extended detection frames with the same number of matches can be further obtained, and the extended detection frame with a larger area can be selected therefrom.

[0082] S204 . For each frame of the image sequence, determine a group of image regions based on a corresponding group of center detection frames.

[0083] In an embodiment of the present disclosure, the abnormal event detection device may determine a set of image regions for each frame of an image sequence based on a corresponding set of center detection frames.

[0084] It should be noted that, in the embodiment of the present disclosure, as described in step S203, each group of central detection frames is actually composed of N extended detection frames in a group of extended detection frames, and each group of extended detection frames is actually included in a frame image of the image sequence. Therefore, each group of central detection frames is actually included in a frame image of the image sequence.

[0085] Specifically, in an embodiment of the present disclosure, the abnormal event detection device determines a group of image areas for each frame image of the image sequence based on a corresponding set of center detection frames, including: obtaining, from the k-th frame image, N pedestrian detection frames corresponding to the N extended detection frames that constitute the corresponding set of center detection frames; from the k-th frame image, taking each of the N pedestrian detection frames as the center and expanding outward according to a second preset ratio to form an image area; and organizing the image areas obtained by the expansion into a group of image areas corresponding to the k-th frame image; wherein the k-th frame image is any frame image in the image sequence, and k is a natural number greater than or equal to 1.

[0086] It should be noted that in the embodiments of the present disclosure, for a group of central detection frames corresponding to the k-th frame image, it is included in the k-th frame image and is composed of N extended detection frames, each of which is obtained by expanding a pedestrian detection frame. The abnormal event detection device can directly obtain the corresponding N pedestrian detection frames from the k-th frame image, and further expand outward at a certain ratio with each pedestrian detection frame as the center. In this way, the obtained image area not only includes the pedestrians included in the pedestrian detection frame, but also includes image information around the pedestrian, for example, people or objects around the pedestrian. In this way, it is convenient to perform subsequent abnormal event detection, making the abnormal event detection more accurate. For the external expansion method of the pedestrian detection frame, the abnormal event detection device can also directly expand each pedestrian detection frame to a preset size, that is, the size of each obtained extended detection frame is the same preset size. The abnormal event detection device can also expand outward using different external expansion methods based on the positions of different pedestrian detection frames in the corresponding frame image, which is not limited by the embodiments of the present disclosure.

[0087] It should be noted that in the embodiments of the present disclosure, the second preset ratio can be set according to actual needs and application scenarios, and the embodiments of the present disclosure are not limited. For example, the second preset ratio can be 2 times, that is, with the pedestrian detection frame as the center, the detection frame is expanded 2 times outward in the image, and the expanded area is the image area.

[0088] It should be noted that, in the embodiments of the present disclosure, since the selection of the extended detection frame is actually only for the basic estimation of the pedestrian center, and the image area is actually the basic area for subsequent abnormal event detection, it can be stipulated that the second preset ratio is greater than the first preset ratio, that is, when determining the image area, compared with the extended detection frame, more information is included, and of course, more pedestrians can be included, thereby providing an effective perception field for subsequent abnormal event detection.

[0089] S102: For each image region set, a central region is selected based on the spatial distribution of the image regions in the set.

[0090] In an embodiment of the present disclosure, when determining N image region sets from an image sequence, the abnormal event detection apparatus may select a central region for each image region set based on the spatial distribution of the image regions in the set.

[0091] Specifically, in an embodiment of the present disclosure, the abnormal event detection device selects a central area for each image area set based on the spatial distribution of the image areas in the set, including: for each image area set, determining the overlap ratio between different image areas in the set, and constructing a corresponding second adjacency matrix based on the obtained overlap ratio; for each image area set, determining the number of matches for each image area in the set based on the corresponding second adjacency matrix in N second adjacency matrices; and from each image area set, selecting the image area with the largest number of matches as the central area.

[0092] It should be noted that in the embodiments of the present disclosure, the abnormal event detection device can determine the overlap ratio between different image areas in each set of image areas for each image area set in the same manner as described above, thereby constructing a corresponding second adjacency matrix. The second adjacency matrix characterizes the distribution of image areas in the set in spatial positions. In order to ensure the consistency of the center point, the abnormal event detection device selects the image area with the densest pedestrians from the set as the central area, that is, the image area with the largest number of matches, and finally selects N central areas from the N image areas.

[0093] S103 , for each central area, extracting an image area at a corresponding position from each frame of the image sequence to form a trajectory sequence.

[0094] In the embodiment of the present disclosure, after obtaining each central area, the abnormal event detection device can extract the image area at the corresponding position from each frame image of the image sequence for each central area domain to form a trajectory sequence.

[0095] It should be noted that in the embodiments of the present disclosure, after the abnormal event detection device extracts an image area corresponding to a central area from each frame of the image sequence, it can resize these image areas and then splice them into a trajectory sequence based on time sequence.

[0096] Figure 3 A schematic diagram of an exemplary trajectory sequence provided in an embodiment of the present disclosure. Figure 3 As shown, if the image sequence includes 5 frames of images, then a trajectory sequence can actually include 5 image areas, wherein the 5 image areas are areas at the same position selected from the 5 frames of images, and the reference deducted from the 5 image areas, that is, the center area, can specifically be the area shown in the second image area, thereby obtaining the 5 image areas constituting the trajectory sequence shown.

[0097] It should be noted that in the embodiments of the present disclosure, the abnormal event detection device performs image area extraction based on a central area. In each frame of the image, the image area of ​​a preset size can be extracted with the position corresponding to the center point of the central area as the center. If the center is at the edge of the image, the part that cannot be extracted to the preset size can be filled with black.

[0098] It is understandable that for each frame of the image sequence, the center points determined by different frames may be inconsistent. If the image areas are directly merged, the range of abnormal event detection covered is too large, and it is easy to contain a large amount of invalid information, resulting in low detection efficiency. In the embodiment of the present disclosure, the abnormal event detection device directly determines a central area from a group of image areas, and then uniformly extracts the image area at the position of the central area for each frame of the image and combines them. In this way, the center points of the obtained trajectory sequence are consistent, which ensures the representativeness and effectiveness of the cutout area. The scope of subsequent abnormal event detection is more reasonable and the detection efficiency is higher.

[0099] S104: Utilize each trajectory sequence to perform abnormal event detection respectively.

[0100] In the embodiment of the present disclosure, when the abnormal event detection device obtains the trajectory sequence, it can use each trajectory sequence to perform abnormal event detection respectively.

[0101] Specifically, in an embodiment of the present disclosure, the abnormal event detection device uses each trajectory sequence to perform abnormal event detection separately, including: using a preset abnormal event detection model to perform abnormal event detection on each trajectory sequence in N trajectory sequences to obtain corresponding abnormal event detection results.

[0102] It should be noted that, in the embodiments of the present disclosure, a preset abnormal event detection model is used to perform abnormal event detection. The specific preset abnormal event detection model may be a specific image classification network, etc., which is not limited in the embodiments of the present disclosure.

[0103] It should be noted that, in the embodiments of the present disclosure, the abnormal event detection device can utilize a preset abnormal event detection device to perform abnormal event detection according to each trajectory sequence, thereby obtaining whether an abnormal event occurs in the real scene area corresponding to the trajectory sequence.

[0104] It can be understood that in the embodiment of the present disclosure, the abnormal event detection device repeats the process of generating the trajectory sequence N times, generates trajectory sequences corresponding to N different areas, and performs abnormal event detection respectively, thereby improving the recall of the image area while ensuring the effective receptive field of the model for the event.

[0105] It should be noted that in the embodiments of the present disclosure, it is possible for multiple trajectory sequences to represent an abnormal event. Therefore, the abnormal event detection device can also input multiple trajectory sequences into a preset abnormal event detection model at the same time. The preset abnormal event detection model can combine multiple trajectory sequences to perform abnormal event detection and analyze a corresponding abnormal event.

[0106] The disclosed embodiment provides an abnormal event detection method, including: obtaining an image sequence, and determining N image area sets from the image sequence; each image area set includes an image area in each frame of the image sequence, and each image area has the same order of pedestrian density among the image areas in the frame of the image to which it belongs; N is a natural number greater than or equal to 1; for each image area set, a central area is selected based on the spatial distribution of the image areas in the set; for each central area, the image area at the corresponding position is extracted from each frame of the image sequence to form a trajectory sequence; and each trajectory sequence is used to perform abnormal event detection separately. The abnormal event detection method provided by the disclosed embodiment not only reduces the amount of data for abnormal event detection and improves detection efficiency, but also provides an effective perception field for abnormal event detection and improves detection accuracy through a heuristic pedestrian dense group position estimation method based on center point consistency constraints.

[0107] An embodiment of the present disclosure provides an abnormal event detection device. Figure 4 The following is a schematic diagram of the structure of an abnormal event detection device provided by an embodiment of the present disclosure. Figure 4 As shown, in an embodiment of the present disclosure, the abnormal event detection device includes:

[0108] The region determination module 401 is used to obtain an image sequence and determine N image region sets from the image sequence; each image region set includes an image region in each frame of the image sequence, and the order of pedestrian density among the image regions in the frame of the image to which each image region belongs is the same; N is a natural number greater than or equal to 1;

[0109] A region selection module 402 is used to select a central region for each image region set based on the spatial distribution of the image regions in the set;

[0110] A sequence determination module 403 is used to extract, for each central area, an image area at a corresponding position from each frame of the image sequence to form a trajectory sequence;

[0111] The anomaly detection module 404 is used to perform abnormal event detection using each trajectory sequence.

[0112] In one embodiment of the present disclosure, the area determination module 401 is specifically used to determine, for each frame image of the image sequence, N image areas in sequence according to the pedestrian density from large to small to form a group of image areas; in the image area groups corresponding to each frame image in the image sequence, the image areas with the same pedestrian density ranking are divided into the same set to obtain the N image area sets.

[0113] In one embodiment of the present disclosure, the region determination module 401 is specifically used to obtain each pedestrian detection frame for each frame image of the image sequence, and expand each of them into an extended detection frame to form a group of extended detection frames; for each group of extended detection frames, determine the overlap ratio between different extended detection frames included, and construct a corresponding first adjacency matrix based on the obtained overlap ratio; for each group of extended detection frames, determine the number of matches of each extended detection frame included based on the corresponding first adjacency matrix, and select N extended detection frames in descending order according to the number of matches to form a group of central detection frames; for each frame image of the image sequence, determine a group of image regions based on the corresponding group of central detection frames.

[0114] In one embodiment of the present disclosure, the area determination module 401 is specifically used to obtain at least one pedestrian detection frame from the i-th frame image; in the i-th frame image, each pedestrian detection frame in the at least one pedestrian detection frame is taken as the center, and expanded outward according to a first preset ratio to form an extended detection frame; the extended detection frames obtained by the expansion are combined into a group of extended detection frames corresponding to the i-th frame image; wherein the i-th frame image is any frame image in the image sequence, and i is a natural number greater than or equal to 1.

[0115] In one embodiment of the present disclosure, the area determination module 401 is specifically used to obtain, from the k-th frame image, N pedestrian detection frames corresponding to N extended detection frames constituting a corresponding group of central detection frames; from the k-th frame image, take each of the N pedestrian detection frames as the center, and expand outward according to a second preset ratio to form an image area; and form the image areas obtained by the expansion into a group of image areas corresponding to the k-th frame image; wherein the k-th frame image is any frame image in the image sequence, and k is a natural number greater than or equal to 1.

[0116] In one embodiment of the present disclosure, the region selection module 402 is specifically used to determine, for each image region set, the overlap ratio between different image regions in the set, and construct a corresponding second adjacency matrix based on the obtained overlap ratio; for each image region set, based on the corresponding second adjacency matrix, determine the number of matches for each image region in the set; and from each image region set, select the image region with the largest number of matches as the central region.

[0117] In an embodiment of the present disclosure, the anomaly detection module 404 is specifically configured to perform an abnormal event detection on each trajectory sequence using a preset abnormal event detection model to obtain a corresponding abnormal event detection result.

[0118] The disclosed embodiment provides a method for detecting abnormal events, including: obtaining an image sequence, and determining N image area sets from the image sequence; each image area set includes an image area in each frame of the image sequence, and each image area has the same order of pedestrian density among the image areas in the frame of the image to which it belongs; N is a natural number greater than or equal to 1; for each image area set, a central area is selected based on the spatial distribution of the image areas in the set; for each central area, the image area at the corresponding position is extracted from each frame of the image sequence to form a trajectory sequence; and each trajectory sequence is used to perform abnormal event detection separately. The abnormal event detection device provided by the disclosed embodiment not only reduces the amount of data for abnormal event detection and improves detection efficiency through a heuristic pedestrian dense group position estimation method based on center point consistency constraints, but also provides an effective perception field for abnormal event detection and improves detection accuracy.

[0119] An embodiment of the present disclosure provides an electronic device. Figure 5 The structure diagram of an electronic device provided by the embodiment of the present disclosure is shown in FIG. Figure 5 As shown, in an embodiment of the present disclosure, the electronic device includes: a processor 501, a memory 502 and a communication bus 503; wherein,

[0120] The communication bus 503 is used to realize the connection and communication between the processor 501 and the memory 502;

[0121] The processor 501 is used to execute one or more programs stored in the memory 502 to implement the above-mentioned abnormal event detection method.

[0122] The present disclosure embodiment provides a computer-readable storage medium, wherein the computer-readable storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the above-mentioned abnormal event detection method. The computer-readable storage medium can be a volatile memory (volatile memory), such as a random access memory (Random-Access Memory, RAM); or a non-volatile memory (non-volatile memory), such as a read-only memory (Read-Only Memory, ROM), a flash memory (flash memory), a hard disk (Hard Disk Drive, HDD) or a solid-state drive (Solid-State Drive, SSD); or it can be a respective device including one or any combination of the above-mentioned memories, such as a mobile phone, a computer, a tablet device, a personal digital assistant, etc.

[0123] Those skilled in the art will appreciate that the embodiments of the present disclosure may be provided as methods, systems, or computer program products. Therefore, the present disclosure may take the form of hardware embodiments, software embodiments, or embodiments combining software and hardware. Furthermore, the present disclosure may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage and optical storage, etc.) containing computer-usable program code.

[0124] The present disclosure is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present disclosure. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable signal processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable signal processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0125] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable signal processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0126] These computer program instructions may also be loaded onto a computer or other programmable signal processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0127] The above description is only a preferred embodiment of the present disclosure and is not intended to limit the protection scope of the present disclosure.

Claims

1. A method for detecting abnormal events, characterized in that: The method comprises: Acquire an image sequence, and determine N image region sets from the image sequence; each image region set includes an image region in each frame of the image sequence, and the order of pedestrian density among the image regions in the frame of the image to which each image region belongs is the same; N is a natural number greater than or equal to 1; For each image region set, selecting a central region based on the spatial distribution of image regions in the set; For each of the central areas, extracting an image area at a corresponding position from each frame of the image sequence to form a trajectory sequence; Using each of the trajectory sequences to perform abnormal event detection respectively; The step of selecting a central region for each image region set based on the spatial distribution of image regions in the set includes: For each image region set, determining an overlap ratio between different image regions in the set, and constructing a corresponding second adjacency matrix according to the obtained overlap ratio; For each set of image regions, determining the number of matches of each image region in the set based on the corresponding second adjacency matrix; From each image region set, the image region with the largest number of matches is selected as the central region.

2. The method according to claim 1, characterized in that Determining N sets of image regions from the image sequence includes: For each frame of the image sequence, N image regions are sequentially determined according to the pedestrian density from large to small to form a group of image regions; In the image region groups corresponding to each frame of the image sequence, the image regions with the same pedestrian density ranking are divided into the same set to obtain the N image region sets.

3. The method according to claim 2, characterized in that For each frame of the image sequence, N image regions are sequentially determined according to the pedestrian density from large to small to form a group of image regions, including: For each frame of the image sequence, each pedestrian detection frame is obtained, and each pedestrian detection frame is expanded into an extended detection frame to form a group of extended detection frames; For each group of the extended detection frames, determining the overlap ratios between the different extended detection frames included therein, and constructing a corresponding first adjacency matrix according to the obtained overlap ratios; For each group of extended detection frames, based on the corresponding first adjacency matrix, determine the number of matches of each included extended detection frame, and select N extended detection frames in descending order of the number of matches to form a group of central detection frames; For each frame of the image sequence, the group of image regions is determined based on the corresponding group of center detection frames.

4. The method according to claim 3, characterized in that For each frame of the image sequence, each pedestrian detection frame is obtained and expanded into an extended detection frame respectively to form a group of extended detection frames, including: Obtain at least one pedestrian detection frame from the i-th frame image; In the i-th frame image, each pedestrian detection frame in the at least one pedestrian detection frame is taken as a center, and is expanded outward according to a first preset ratio to form an extended detection frame; The extended detection frames obtained by the outward expansion are combined into a group of extended detection frames corresponding to the i-th frame image; The i-th frame image is any frame image in the image sequence, and i is a natural number greater than or equal to 1.

5. The method according to claim 3, characterized in that: The step of determining the set of image regions for each frame of the image sequence based on the corresponding set of center detection frames includes: From the k-th frame image, obtain N pedestrian detection frames corresponding to the N extended detection frames constituting a corresponding set of central detection frames; From the k-th frame image, taking each of the N pedestrian detection frames as a center, and expanding outwards according to a second preset ratio to form an image region; The image regions obtained by the outward expansion are combined into the group of image regions corresponding to the k-th frame image; The k-th frame image is any frame image in the image sequence, and k is a natural number greater than or equal to 1.

6. The method according to claim 1, characterized in that The method of using each trajectory sequence to perform abnormal event detection includes: The abnormal event detection is performed on each trajectory sequence by using a preset abnormal event detection model to obtain a corresponding abnormal event detection result.

7. An abnormal event detection device, characterized in that: The device comprises: A region determination module is used to obtain an image sequence and determine N image region sets from the image sequence; each image region set includes an image region in each frame of the image sequence, and the order of pedestrian density among the image regions in the frame of the image to which each image region belongs is the same; N is a natural number greater than or equal to 1; A region selection module is used to determine, for each of the image region sets, the overlap ratio between different image regions in the set, and construct a corresponding second adjacency matrix according to the obtained overlap ratio; for each of the image region sets, based on the corresponding second adjacency matrix, determine the number of matches of each image region in the set; and from each of the image region sets, select the image region with the largest number of matches as the central region; A sequence determination module, for extracting, for each of the central regions, an image region at a corresponding position from each frame of the image sequence to form a trajectory sequence; The anomaly detection module is used to perform abnormal event detection using each of the trajectory sequences.

8. An electronic device, characterized in that: The electronic device comprises: a processor, a memory and a communication bus; wherein, The communication bus is used to realize the connection and communication between the processor and the memory; The processor is used to execute one or more programs stored in the memory to implement the abnormal event detection method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the abnormal event detection method according to any one of claims 1 to 6.

Images