Behavior recognition method, device, electronic device and computer-readable storage medium

Abstract

The disclosed embodiment discloses a behavior recognition method, device, equipment and computer-readable storage medium. The method includes: classifying and identifying important areas of each image frame in at least one image frame of the obtained image frame sequence, and obtaining a recognition result corresponding to at least one image frame; the important area is determined according to the detection frame area where the target object in at least one image frame is located; when the recognition result indicates the existence of a preset abnormal event, the activation center of the class activation map corresponding to the recognition result of each image frame is determined to obtain at least one activation center of at least one image frame; an activation center represents an abnormal position in the class activation map of a corresponding image frame; based on at least one activation center, the abnormal target object of each image frame is identified. Through the present disclosure, the recognition accuracy can be improved.

Description

Technical Field

[0001] The present disclosure relates to terminal technology, and in particular to a behavior recognition method, device, electronic device, and computer-readable storage medium. Background Art

[0002] Anomaly detection in videos is an important problem in the field of computer vision and has a wide range of applications in the field of intelligent recognition, such as detecting illegal behavior, traffic accidents, and some unusual events. Thousands of cameras are deployed around the world. However, most cameras only record the dynamics of each moment and do not have the ability to automatically identify (often special personnel are required to be responsible for manual inspection). Due to the huge number of videos, it is obviously unrealistic to filter the content in the video by manpower alone. We need to use computer vision and deep learning technology to automatically detect abnormal events occurring in the video.

[0003] Identifying abnormal events in videos is extremely difficult. Possible challenges include scarcity of annotated data due to low-probability events, large variance between / within classes, subjective differences in the definition of abnormal events, low resolution of video images, etc. As humans, we use common sense to identify abnormalities. For example, if a crowd gathers on a street that usually has no traffic, it may be an abnormality; for example, a violent incident such as a fight. For machines, they have no common sense, only visual features. Generally speaking, the stronger the visual features, the better the expected anomaly detection performance.

[0004] There are some methods for identifying abnormal events in the related art, but when the identification methods in the related art are used for abnormality identification, the identification accuracy is low. Summary of the invention

[0005] The embodiments of the present disclosure provide a behavior recognition method, device, electronic device, and computer-readable storage medium, which can improve recognition accuracy.

[0006] The technical solution of the embodiment of the present disclosure is implemented as follows:

[0007] The present disclosure provides a behavior recognition method, including:

[0008] In at least one image frame of the acquired image frame sequence, important regions of each image frame are classified and identified to obtain an identification result corresponding to the at least one image frame; the important region is determined according to a detection frame region where a target object is located in the at least one image frame;

[0009] In the case where the recognition result indicates the presence of a preset abnormal event, determining the activation center of the class activation map corresponding to the recognition result of each image frame to obtain at least one activation center of the at least one image frame; one activation center represents an abnormal position in the class activation map of a corresponding image frame;

[0010] Based on the at least one activation center, abnormal target objects of the respective image frames are identified.

[0011] The present disclosure provides a behavior recognition device, comprising: a recognition unit, configured to classify and recognize important regions of at least one image frame in an acquired image frame sequence, and obtain a recognition result corresponding to the at least one image frame; the important region is determined according to a detection frame region where a target object is located in the at least one image frame;

[0012] A determination unit, configured to determine, when the recognition result indicates the presence of a preset abnormal event, an activation center of a class activation map corresponding to the recognition result of each image frame, to obtain at least one activation center of the at least one image frame; one activation center indicates an abnormal position in the class activation map of the corresponding image frame;

[0013] The identification unit is further used to identify abnormal target objects in each image frame based on the at least one activation center.

[0014] An embodiment of the present disclosure provides an electronic device, comprising: a memory for storing an executable computer program; and a processor for implementing the above-mentioned behavior recognition method when executing the executable computer program stored in the memory.

[0015] The embodiment of the present disclosure provides a computer-readable storage medium storing a computer program for causing a processor to execute the above-mentioned behavior recognition method.

[0016] The behavior recognition method, apparatus, device and computer-readable storage medium provided by the embodiments of the present disclosure classify and identify important areas of each image frame in at least one image frame of the obtained image frame sequence to obtain a recognition result corresponding to the at least one image frame; the important area is determined according to the detection frame area where the target object in the at least one image frame is located; when the recognition result indicates the existence of a preset abnormal event, the activation center of the class activation map corresponding to the recognition result of each image frame is determined to obtain at least one activation center of at least one image frame, and one activation center indicates an abnormal position in the class activation map of the corresponding image frame; based on the at least one activation center, the abnormal target object of each image frame is identified. By adopting the present technical solution, on the one hand, since the important areas of the determined image frames are classified and identified, compared with identifying based on the data of the entire image of each image frame, the search range during identification is effectively narrowed, and the interference to the identification is reduced, thereby improving the recognition accuracy; on the other hand, since the abnormal center (activation center) of each image frame is located by determining the abnormal position in the class activation map corresponding to the recognition result of each image frame, and the abnormal target object in the corresponding image frame is identified based on the obtained abnormal center, and the class activation map corresponding to the recognition result can characterize the target object in the image frame related to the recognition result, the abnormal target object of each image frame is identified more accurately, thereby improving the recognition accuracy of the abnormal target object; therefore, the behavior recognition method provided by the embodiment of the present disclosure can improve the recognition accuracy.

[0017] It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The drawings herein are incorporated into the specification and constitute a part of the specification. These drawings illustrate embodiments consistent with the present disclosure and are used to illustrate the technical solutions of the present disclosure together with the specification.

[0019] Figure 1 An optional flow chart of a behavior recognition method provided in an embodiment of the present disclosure;

[0020] Figure 2 A schematic diagram showing the display effect of an exemplary class activation diagram and an activation center provided in an embodiment of the present disclosure;

[0021] Figure 3 A schematic diagram of the display effect of an abnormal target object in an exemplary image frame provided by an embodiment of the present disclosure;

[0022] Figure 4 An optional flow chart of a behavior recognition method provided in an embodiment of the present disclosure;

[0023] Figure 5 An optional flow chart of a behavior recognition method provided in an embodiment of the present disclosure;

[0024] Figure 6 Another schematic diagram showing the effect of displaying an exemplary class activation diagram provided by an embodiment of the present disclosure;

[0025] Fig. 7A An optional flow chart of a behavior recognition method provided in an embodiment of the present disclosure;

[0026] Figure 7B An optional flow chart of a behavior recognition method provided in an embodiment of the present disclosure;

[0027] Figure 8 A schematic diagram showing the display effect of a class activation map and a target activation center corresponding to an exemplary image frame provided in an embodiment of the present disclosure;

[0028] Fig. 9 An optional flow chart of a behavior recognition method provided in an embodiment of the present disclosure;

[0029] Fig.10 An optional flow chart of a behavior recognition method provided in an embodiment of the present disclosure;

[0030] Fig.11 An optional flow chart of a behavior recognition method provided in an embodiment of the present disclosure;

[0031] Fig.12 A schematic diagram of areas corresponding to the generalized area positions of two different second areas in an image frame provided by an exemplary embodiment of the present disclosure;

[0032] Fig.13 A schematic diagram of the structure of a behavior recognition device provided in an embodiment of the present disclosure;

[0033] Fig.14 A schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0034] In order to make the objectives, technical solutions and advantages of the present disclosure clearer, the present disclosure will be further described in detail below in conjunction with the accompanying drawings. The described embodiments should not be regarded as limiting the present disclosure. All other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present disclosure.

[0035] The inventor of the present disclosure has found through research that traditional behavior recognition methods usually perform full-image data enhancement or other preprocessing on the input video sequence before feeding it into the behavior recognition model for prediction. However, this method is only applicable to human-centered video behavior recognition, and such data is commonly found in public video academic data sets. For videos shot by surveillance cameras, they often contain more information and cover a larger field of view; at the same time, the location of the target event and the scale of the human body are also random; therefore, it is obviously unreasonable to simply use the full image as the model input. There is a lot of irrelevant information in the picture, which will interfere with behavior recognition, thereby affecting the accuracy of behavior recognition and the accuracy of identifying the person who performs the behavior.

[0036] In this regard, the inventors of the present disclosure believe that local areas can be used instead of the entire image to be input into the behavior recognition model for behavior classification. In this way, the interference of most irrelevant information in the picture can be reduced and the accuracy of behavior recognition can be improved. However, on this basis, it is also necessary to consider how to accurately locate the position of the specific behavior executor after narrowing the search range of the behavior recognition model.

[0037] To this end, the present disclosure provides a behavior recognition method that can improve the recognition accuracy of abnormal behavior recognition and abnormal target objects (behavior performers) associated with abnormal behavior recognition. Before further describing the present disclosure in detail, the nouns and terms involved in the present disclosure are explained, and the nouns and terms involved in the present disclosure are applicable to the following explanations.

[0038] 1) CNN (Convolutional Neural Networks) refers to convolutional neural networks, which are deep neural networks with a convolutional structure. It is essentially an input-to-output mapping that can learn a large number of mapping relationships between inputs and outputs. By training the convolutional network with known patterns, the network has the ability to map between input and output pairs. It is usually used in the field of image recognition.

[0039] 2) CAM (Class Activation Map), which refers to the class activation heat map, is a two-dimensional graph, which is essentially a two-dimensional feature score network related to a specific output category. Each position of the grid represents its importance to the category. For a picture input to the CNN model and classified as "dog", the similarity of each position in the picture to the "dog" class can be presented in the form of a heat map. The class activation map helps to understand which part of an image makes the convolutional neural network make the final decision.

[0040] The following describes exemplary applications of the electronic device provided by the embodiments of the present disclosure. The electronic device provided by the embodiments of the present disclosure can be implemented as various types of user terminals (hereinafter referred to as terminals) such as AR glasses, laptop computers, tablet computers, desktop computers, set-top boxes, mobile devices (for example, mobile phones, portable music players, personal digital assistants, dedicated messaging devices, portable gaming devices), and can also be implemented as servers.

[0041] Next, an exemplary application when the electronic device is implemented as a terminal will be described. Figure 1 is an optional flow chart of the behavior recognition method provided by the embodiment of the present disclosure, which will be combined with Figure 1 The steps shown are explained.

[0042] S101. Classify and identify important areas of at least one image frame in the acquired image frame sequence to obtain an identification result corresponding to at least one image frame; the important area is determined according to a detection frame area where a target object is located in at least one image frame.

[0043] When the electronic device obtains the important areas of each image frame in at least one image frame of an image frame sequence, the electronic device can simultaneously input the important areas of each image frame in the at least one image frame into a behavior recognition model for classification and recognition, thereby obtaining a recognition result corresponding to the at least one image frame; the important areas of each image frame are determined based on the detection frame area where all target objects in the at least one image frame are located; for example, when the electronic device obtains the important areas of each image frame in 8 image frames of an image frame sequence, the electronic device can simultaneously input the important areas of each image frame in the 8 image frames into the behavior recognition model for classification and recognition, thereby obtaining a recognition result corresponding to the 8 image frames.

[0044] Here, the electronic device may crop out important areas of each image frame in the at least one image frame, and classify and identify the cropped important areas.

[0045] In the embodiment of the present disclosure, the detection frame is used to mark the position of the target object in the image frame, and each target object corresponds to a detection frame, such as the following Figure 3 There are two detection boxes in that correspond one-to-one to the two target objects.

[0046] In some embodiments, the electronic device can directly obtain the at least one image frame in the image frame sequence and the important area of ​​each image frame from other devices, and perform classification and identification based on the important areas of each image frame in the at least one image frame, so as to obtain a recognition result corresponding to the at least one image frame.

[0047] In the disclosed embodiment, the behavior recognition model is a pre-trained neural network model; for example, a Two-Stream model, a 3D (3-dimensional) ConvNets model, and a 2D (2-dimensional) ConvNets model, etc., which are not limited in the disclosed embodiment. The electronic device can determine whether there are abnormal events such as fighting, gathering to cause trouble, and traffic accidents in the at least one image frame by classifying and identifying important areas of each image frame in the at least one image frame.

[0048] In some embodiments, the image frame sequence may be a video stream captured by a camera or other acquisition device set in a specific scene, and the at least one image frame may be a continuous frame in the video stream; the specific scene may be a street, a shopping mall, a scenic spot, etc., which is not limited to the embodiments of the present disclosure.

[0049] S102. When the recognition result indicates the presence of a preset abnormal event, determine the activation center of the class activation map corresponding to the recognition result of each image frame to obtain at least one activation center of at least one image frame; one activation center represents the abnormal position in the class activation map of the corresponding image frame.

[0050] When the electronic device recognizes that there is a preset abnormal event in the at least one image frame (i.e., recognizes the category of the preset abnormal event), it can determine the activation center of the class activation map corresponding to each image frame in the at least one image frame, thereby obtaining at least one activation center corresponding to the at least one image frame, wherein the class activation map is a class activation map related to the category corresponding to the recognition result, and an activation center represents an abnormal position in the class activation map of the corresponding image frame. For example, Figure 2 As shown, when the preset abnormal event of fighting is identified according to the eight consecutive image frames y1, y2, y3, y4, y5, y6, y7 and y8, eight class activation maps w1, w2, w3, w4, w5, w6, w7 and w8 corresponding to the eight consecutive image frames y1, y2, y3, y4, y5, y6, y7 and y8 and corresponding to the fighting event can be generated. In , regions r1, r2, r3, r4, r5, r6, r7 and r8 are marked, respectively representing the regions in their respective image frames that provide the most information for the model to identify the category of the preset abnormal event. For example, region r1 in the class activation map w1 represents the region in the image frame y1 that provides the most information for the model to identify the category of the fight event; and the activation centers of the eight class activation maps w1, w2, w3, w4, w5, w6, w7 and w8 are also determined, as shown in Figure 2j1, j2, j3, j4, j5, j6, j7 and j8 in (since the sizes of these 8 class activation maps are consistent with the sizes of the corresponding 8 image frames, Figure 2 The activation center of the class activation map is directly marked in the corresponding image frame, and Figure 2 As shown, the activation center is the position marked with a pentagonal heart).

[0051] In the embodiments of the present disclosure, the preset abnormal events may be, for example, fighting, gathering to cause trouble, traffic accidents, etc., which are not limited in the embodiments of the present disclosure.

[0052] S103: Identify abnormal target objects in each image frame based on at least one activation center.

[0053] In an embodiment of the present disclosure, when the electronic device determines the activation center of each image frame in the at least one image frame, for each image frame, it can determine whether there is an abnormal target object in the image frame based on the activation center of the image frame, and identify the abnormal target object if there is an abnormal target object.

[0054] In some embodiments, since each activation center corresponds to an image frame, for each activation center, the electronic device can determine the position of the activation center in the corresponding image frame, and when the position is located in any detection frame area in the image frame, the target object corresponding to any detection frame area is determined as an abnormal target object in the image frame; thus, for the at least one activation center, the abnormal target objects of each image frame in the at least one image frame corresponding to the at least one activation center can be obtained. For example, Figure 3 As shown, for image frame a, when the activation center 31 is located in two different human detection frames 32 and 33, the two pedestrians corresponding to the human detection frames 32 and 33 can be determined as abnormal target objects in image frame a.

[0055] Here, for each activation center, when the position of the activation center in a corresponding image frame does not belong to any detection frame area in the image frame, it can be determined that there is no abnormal target object in the image frame.

[0056] In the embodiment of the present disclosure, since the important areas of the determined image frames are classified and identified, compared with the identification based on the data of the whole image of each image frame, the search range during identification is effectively narrowed, and the interference to the identification is reduced, thereby improving the recognition accuracy; on the other hand, since the abnormal center (activation center) of each image frame is located by determining the abnormal position in the class activation map corresponding to the recognition result of each image frame, and the abnormal target object in the corresponding image frame is identified according to the obtained abnormal center, and the class activation map corresponding to the recognition result can characterize the target object related to the recognition result in the image frame, the abnormal target object of each image frame identified subsequently can be made more accurate, thereby improving the recognition accuracy of the abnormal target object, and at the same time, narrowing the search range of the model during identification will not affect the subsequent identification of abnormal target objects; therefore, the technical solution of the embodiment of the present disclosure can improve the recognition accuracy.

[0057] Figure 4 is an optional flow chart of the behavior recognition method provided by the embodiment of the present disclosure, such as Figure 4 As shown, the above S102 can be implemented by S1021-S1023, combining Figure 4 Provide explanation.

[0058] S1021. When the recognition result indicates the presence of a preset abnormal event, determine a set of activation values ​​corresponding to the recognition result for each image frame; the set of activation values ​​is used to generate a class activation map of the image frame, and each pixel position in the class activation map corresponds one-to-one to an activation value in the set of activation values.

[0059] When the electronic device recognizes through the behavior recognition model that there is a preset abnormal event in at least one image frame, for each image frame, the electronic device can determine a set of activation values ​​corresponding to the preset abnormal event of the image frame through a class activation mapping method, wherein the set of activation values ​​is used to draw a class activation map corresponding to the preset abnormal event of the image frame, and each pixel position in the class activation map corresponds one-to-one to a pixel position in the set of activation values; in the process of drawing the class activation map of the image frame according to the set of activation values, for example, when an activation value represents that the color of a pixel point corresponding to the activation value is red, red is generated at the pixel position corresponding to the activation value, and the class activation map corresponding to the set of activation values ​​is drawn by this method, that is, different shades of colors are used in the class activation map to represent the contribution of different pixels to the recognition result.

[0060] In the disclosed embodiment, the size of the class activation map of the image frame drawn according to the group of activation values ​​is consistent with the size of the image frame, which facilitates the subsequent accurate positioning of the abnormal target object in the original image frame according to the abnormal position in the class activation image of the image frame.

[0061] In some embodiments, Figure 5 is an optional flow chart of the behavior recognition method provided by the embodiment of the present disclosure, such as Figure 5 As shown, the above S1021 can be implemented by S201-S203, combining Figure 5 Provide explanation.

[0062] S201: Generate a set of initial class activation map values ​​corresponding to the recognition results for each image frame.

[0063] For each image frame, when obtaining a recognition result of the image frame, the electronic device may generate a set of initial class activation mapping values ​​of the image frame corresponding to the recognition result.

[0064] In some embodiments, the above S201 can be implemented in the following manner: for each image frame, based on the weight of the classification layer corresponding to the recognition result and the feature map output by the convolution layer in the behavior recognition model, a set of initial class activation mapping values ​​corresponding to the recognition result of the image frame is generated; wherein the recognition result is obtained by classifying and identifying important areas using the behavior recognition model; the behavior recognition model includes: a classification layer and multiple convolution layers. In some embodiments, the electronic device can use a behavior recognition model including a classification layer and multiple convolution layers to classify and identify important areas of each image frame in the at least one image frame to identify the event category of a preset abnormal event. For each image frame, the last convolution layer in the behavior recognition model can generate y feature maps of important areas of the image frame, each feature map mainly extracts features related to one of the y categories, for example, the y feature maps can be expressed as A 1 ,A 2 ,…,A y ; The classification layer includes y neurons, one neuron corresponds to one category, and each neuron has y weight values, for example, w 1 ,w 2 ,…,w y ; When the event category of the preset abnormal event is class C, a set of initial class activation mapping values ​​corresponding to the image frame can be calculated using the following formula (1):

[0065]

[0066] Among them, i represents the i-th neuron, A represents the weight value of class c corresponding to the i-th neuron, i represents the feature map corresponding to the i-th neuron, Represents a set of initial class activation map values.

[0067] S202, upsampling the initial class activation map values ​​to obtain a set of intermediate activation values; the set of intermediate activation values ​​is used to generate a class activation map corresponding to the important area of ​​the image frame.

[0068] S203: Fusing the intermediate activation value with the original pixel value of the image frame to obtain a set of activation values.

[0069] Here, for an image frame, the class activation map that can be drawn by the obtained set of initial class activation map values ​​is consistent with the size of the feature map generated by the last convolutional layer, so the electronic device can perform upsampling operation on this set of initial class activation map values ​​to generate a set of intermediate activation values, and the size of the class activation map that can be drawn by this set of intermediate activation values ​​is consistent with the size of the important area of ​​the image frame, that is, the class activation map corresponding to the important area of ​​the image frame is obtained; on this basis, the electronic device can also fuse this set of intermediate activation values ​​with a set of original pixel values ​​of the image frame to obtain a set of activation values ​​corresponding to the image frame, and the size of the class activation map that can be drawn by this set of activation values ​​is consistent with the size of the original image of the image frame, thereby realizing the superposition of the original image of the image frame and the class activation map corresponding to the important area of ​​the image frame, and finally obtaining a class activation map corresponding to the image frame and consistent with the size of the image frame. For example, Figure 6 It shows the relationship between y10, y20, y30, y40, y50, y60, y70 and y80 ( Figure 6 When a fight event is identified in the eight consecutive image frames (not shown in the figure), eight class activation maps w10, w20, w30, w40, w50, w60, w70 and w80 are generated, which correspond to the eight consecutive image frames y10, y20, y30, y40, y50, y60, y70 and y80 and correspond to the fight event. Among them, in the class activation maps w10, w20, w30, w40, w50, w60, w70 and w80, regions r10, r20, r30, r40, r50, r60, r70 and r80 are marked respectively, representing the regions in their respective image frames that provide the most information when the model identifies the category of the preset abnormal event. For example, region r10 in the class activation map w10 represents the region in the image frame y10 that provides the most information when the model identifies the category of the fight event.

[0070] The following continues the above Figure 4 S1022~S1023 in the explanation are as follows:

[0071] S1022. Determine the maximum activation value among a group of activation values ​​of each image frame, and determine the pixel position corresponding to the maximum activation value as the abnormal position.

[0072] For each image frame, the electronic device can determine the maximum activation value in a set of activation values ​​corresponding to the image frame, and determine the pixel position corresponding to the maximum activation value as the abnormal position of the image frame. Since the class activation map is a two-dimensional image, the abnormal position (pixel position) can be represented by two-dimensional coordinates, for example, X(i, j), to characterize the position of the abnormal position in the class activation map or in the image frame.

[0073] S1023. Based on the abnormal position of the image frame, obtain the activation center of the image frame. For at least one image frame, obtain at least one activation center.

[0074] For each image frame, the electronic device may determine the abnormal position in the class activation map of the image frame as the activation center of the class activation map of the image frame, so that at least one activation center may be obtained correspondingly for the at least one image frame.

[0075] Fig. 7A is an optional flow chart of the behavior recognition method provided by the embodiment of the present disclosure, such as Fig. 7A As shown, the above S103 can be implemented by S1031-S1032. Figure 1 For example, combined with Fig. 7A Provide explanation.

[0076] S1031. Determine a target activation center according to at least one activation center.

[0077] Since the positions of activation centers in different types of activation maps may be offset, the electronic device can determine a target activation center based on the obtained activation center after determining all activation centers corresponding to the at least one image frame; in this way, the drift phenomenon of the activation center can be alleviated, thereby making the abnormal target object of each image frame in the at least one image frame subsequently determined by using the target activation center more accurate, which is beneficial to improving the recognition accuracy of abnormal target objects.

[0078] In some embodiments, the electronic device may calculate the average value of all activation centers corresponding to the at least one image frame, and use the average value as the target activation center. Here, since each activation center is actually the two-dimensional coordinate of a pixel point, the electronic device may obtain the target activation center by calculating the coordinate average value of these two-dimensional coordinates, for example, the average value of the x coordinate and the average value of the y coordinate may be calculated respectively.

[0079] S1032. Identify abnormal target objects in each image frame according to the target activation center.

[0080] When the electronic device determines a target activation center corresponding to at least one image frame, for each image frame, it can determine whether there is an abnormal target object in the image frame based on the target activation center, and when it is determined that there is an abnormal target object, it can identify the existing abnormal target object.

[0081] Here, for each image frame, the electronic device can determine the position of the target activation center in the image frame, and when the position is located in any detection frame area in the image frame, the target object corresponding to any detection frame area is determined as an abnormal target object in the image frame; thus, for the at least one image frame, the abnormal target objects of each image frame in the at least one image frame can be obtained. For example, as mentioned above Figure 3 32 and 33 are abnormal target objects in image frame a.

[0082] Figure 7B is an optional flow chart of the behavior recognition method provided by the embodiment of the present disclosure, such as Figure 7B As shown, the above S103 can also be implemented through S1033-S1034. Figure 1 For example, combined with Figure 7B Provide explanation.

[0083] S1033. Cluster at least one activation center by an unsupervised clustering method to obtain a target activation center.

[0084] S1034. Identify abnormal target objects in each image frame according to the target activation center.

[0085] When the electronic device determines at least one activation center that corresponds one-to-one to at least one image frame, it can use an unsupervised clustering method to determine a target activation center based on the at least one activation center obtained; in this way, the drift phenomenon of the activation center can be alleviated, so that the abnormal target object of each image frame in the at least one image frame subsequently determined using the target activation center is more accurate, which is beneficial to improve the accuracy of identifying abnormal target objects.

[0086] In some embodiments, the electronic device can use the K-Means clustering method to cluster all activation centers corresponding to the at least one image frame. In this way, drift points can be eliminated through unsupervised clustering, so that the abnormal target objects in each image frame of the at least one image frame subsequently determined by the target activation center are more accurate, which is conducive to improving the accuracy of identifying abnormal target objects.

[0087] Here, the unsupervised clustering method may also be other clustering methods, which are not specifically limited in the embodiments of the present disclosure.

[0088] In some embodiments, the above S1031 may also be implemented through the above S1033.

[0089] In some embodiments, the above S1033 can be implemented through S301-S302:

[0090] S301. Cluster at least one activation center by an unsupervised clustering method to obtain a center cluster containing the largest number of activation centers.

[0091] The electronic device can cluster all activation centers corresponding to the at least one image frame by an unsupervised clustering method, thereby obtaining multiple clusters, each of which contains at least one activation center. Afterwards, the cluster containing the largest number of activation centers can be selected from the multiple clusters, thereby obtaining a central cluster.

[0092] S302: Determine the average value of the activation centers included in the center cluster, and determine the average value as the target activation center.

[0093] When the electronic device determines the center cluster, it can calculate the average value of all activation centers contained in the center cluster and use the average value as the target activation center. Similarly, since each activation center is actually the two-dimensional coordinate of a pixel point, the electronic device can obtain the target activation center by calculating the coordinate average value of these two-dimensional coordinates, for example, obtaining the target activation center X(i, j). Figure 8 It is shown that when the preset abnormal event of fighting is identified according to the eight consecutive image frames y11, y22, y33, y44, y55, y66, y77 and y88, eight class activation maps w11, w22, w33, w44, w55, w66, w77 and w88 corresponding to the eight consecutive image frames y11, y22, y33, y44, y55, y66, y77 and y88 and corresponding to the fighting event can be generated. 7 and w88, regions r11, r22, r33, r44, r55, r66, r77 and r88 are marked respectively, representing the regions in their respective image frames that provide the most information for the model to recognize the category of the preset abnormal event. For example, region r11 in the class activation map w11 represents the region in the image frame y11 that provides the most information for the model to recognize the category of the fight event; and the target activation centers of the eight class activation maps w11, w22, w33, w44, w55, w66, w77 and w88 are also determined, as shown in FIG. Figure 8As shown in j11, j22, j33, j44, j55, j66, j77 and j88 in (since the sizes of these 8 class activation maps are consistent with the sizes of the corresponding 8 image frames, Figure 8 The target activation center of the class activation map is directly marked in the corresponding image frame, and Figure 8 The target activation center is shown as the location marked with a pentagonal heart).

[0094] In some embodiments, the electronic device may also obtain the at least one image frame in S101 and the important area of ​​each image frame in the at least one image by itself through some methods, such as Fig. 9 The method shown. Fig. 9 is an optional flow chart of the behavior recognition method provided by the embodiment of the present disclosure, such as Fig. 9 As shown, before the above S101, S001-S003 can also be executed to Figure 1 Take this as an example Fig. 9 Provide explanation.

[0095] S001. Perform target detection on a current image frame of an image frame sequence to determine at least one detection frame area corresponding to at least one existing target object.

[0096] The electronic device can obtain a real-time captured video stream of a specific scene from a capture device such as a camera set in a specific scene, and perform real-time target detection on the real-time captured video stream in units of image frames. In the target detection process, for the current image frame, the electronic device can use a pre-trained target detection network to detect whether there is a target object in the current image frame, and if there is a target object, mark all target objects in the current image frame with a detection frame, thereby obtaining the detection frame area corresponding to all target objects in the current image frame; for example, the above Figure 3 In the image frame a, there are two detection frame areas 32 and 33 corresponding to the two target objects.

[0097] In the embodiments of the present disclosure, the target detection network may be a CNN network or other networks, etc., and the embodiments of the present disclosure are not limited to this.

[0098] For an image frame, there may or may not be a target object. In the embodiments of the present disclosure, description is given for the case where there is a target object in the image frame.

[0099] S002. Determine a central detection frame area in at least one detection frame area.

[0100] S003. When at least one image frame has been detected, determine important areas of each image frame in the at least one image frame based on at least one central detection frame area corresponding to the at least one image frame.

[0101] For each image frame, when the electronic device obtains all the detection frame areas in the image frame, it can determine a detection frame area as the central detection frame area. Thus, when the electronic device obtains at least one image frame, it can obtain at least one central detection frame area and determine the important areas of each image frame in the at least one image frame based on the at least one central detection frame area.

[0102] In the embodiment of the present disclosure, the electronic device can determine that at least one image frame has been detected when it is determined that the preset condition is met. Exemplarily, the electronic device can determine that the preset condition is met when it is determined that the detected time has reached the preset time, and obtain the image frame that has been detected when the preset time has been reached, and the preset time can be set according to actual needs; for example, the preset time can be 6 seconds, and the electronic device can use the image frame detected within every 6 seconds as the detected image frame; that is, the electronic device will obtain at least one image frame every 6 seconds, and will determine the important area of ​​each image frame in the at least one image frame; Exemplarily, the electronic device can also determine that the preset condition is met and obtain the detected image frame when it is determined that the number of detected image frames reaches the preset number, and the preset number can be set according to actual needs; for example, when the preset number is 8, the electronic device detects 8 image frames every time, and takes these 8 image frames as the detected image frames; that is, the electronic device detects 8 image frames every time, and will determine the important area of ​​each image frame in these 8 image frames.

[0103] In some embodiments, Fig.10 is an optional flow chart of the behavior recognition method provided by the embodiment of the present disclosure, such as Fig.10 As shown, the above S002 can be implemented by S401-S404, combining Fig.10 Provide explanation.

[0104] S401: Enlarge at least one detection frame area by a first preset ratio to obtain at least one first area corresponding to the at least one detection frame area.

[0105] For all detection frame areas in each image frame, the electronic device may enlarge the areas of all detection frame areas by a first preset ratio, thereby obtaining first areas corresponding to all detection frame areas; for example, when there are N detection frame areas in each image frame, the electronic device may enlarge the areas of the N detection frame areas by a first preset ratio, thereby obtaining N first areas, where N is an integer greater than 0. It should be noted that enlarging the area of ​​a detection frame area by a first preset ratio means enlarging the range included in the detection frame area by the first preset ratio, thereby obtaining a first area with a larger range.

[0106] In the embodiment of the present disclosure, the first preset ratio can be set according to actual needs. For example, the first preset ratio can be 1.5, which is not limited in the embodiment of the present disclosure.

[0107] S402: Determine a matching result between any two first regions based on an area intersection-and-union ratio between any two first regions in at least one first region; the matching result indicates whether any two first regions match.

[0108] For the first areas corresponding to all detection frame areas in each image frame, the electronic device can calculate the area intersection and union ratio between any two first areas, and determine whether the two first areas match based on the size relationship between the obtained area intersection and union ratio and a preset threshold. For example, in the case where an image frame corresponds to three first areas (first area 1, first area 2, and first area 3), the electronic device can calculate the area intersection and union ratio between first area 1 and first area 2, the area intersection and union ratio between first area 1 and first area 3, and the area intersection and union ratio between first area 2 and first area 3, and determine that first area 1 and first area 2 do not match when the area intersection and union ratio between first area 1 and first area 2 is less than the preset threshold, and determine that first area 1 and first area 2 match when the area intersection and union ratio between first area 1 and first area 3 is greater than or equal to the preset threshold, and determine that first area 2 and first area 3 match when the area intersection and union ratio between first area 2 and first area 3 is greater than or equal to the preset threshold.

[0109] In the embodiment of the present disclosure, the preset threshold can be set according to actual needs, for example, it can be 0, which is not limited in the embodiment of the present disclosure.

[0110] S403: Determine the number of matches of each first region based on the matching result, and determine the first region with the largest number of matches.

[0111] For each image frame, when the electronic device determines the matching results between all the first regions of the image frame, it can determine the number of other first regions matched by each first region based on these matching results, and use this number as the number of matches, and arrange the number of matches of all the first regions in descending or ascending order, and determine one or more first regions with the largest number of matches based on the sorting results. Continuing with the above example, since it is determined through the area intersection and union ratio that the first region 1 does not match the first region 2, the first region 1 matches the first region 3, and the first region 2 matches the first region 3, it can be determined that the other first regions matched by the first region 1 are the first region 3, the other first regions matched by the first region 2 are the first region 3, and the other first regions matched by the first region 3 are the first region 1 and the first region 2; thus, it can be obtained that the number of matches of the first region 1 is 1, the number of matches of the first region 2 is 1, the number of matches of the first region 3 is 2, and the first region 3 is the first region with the largest number of matches.

[0112] S404: Determine a dense center according to the first region with the largest number of matches, and determine a detection frame region corresponding to the dense center as a central detection frame region.

[0113] For each image frame, the electronic device can determine the dense center of the image frame based on the first area with the largest number of matches, and determine the detection frame area corresponding to the dense center as the central detection frame area of ​​the image frame. Figure 3 For example, when the detection frame area corresponding to the dense center is 33, the detection frame area 33 can be used as the central detection frame area of ​​the image frame a.

[0114] In some embodiments, determining the dense center based on the first region with the largest number of matches in the above S404 can be achieved in the following way: when there is one first region with the largest number of matches, the first region with the largest number of matches is determined as the dense center; when there are two or more first regions with the largest number of matches, the first region with the largest area among the two or more first regions with the largest number of matches is determined as the dense center; in this way, the accuracy of the obtained dense center can be improved.

[0115] In some embodiments, Fig.11 is an optional flow chart of the behavior recognition method provided by the embodiment of the present disclosure, such as Fig.11 As shown, the above S003 can be implemented by S501-S503, combining Fig.11 Provide explanation.

[0116] S501. When at least one image frame has been detected, enlarge at least one central detection frame area corresponding to the at least one image frame by a second preset ratio to obtain at least one second area corresponding to the at least one detection frame area.

[0117] For all detection frame areas of each image frame in the at least one image frame, the electronic device may enlarge the areas of all detection frame areas by a second preset ratio, thereby obtaining second areas corresponding to all detection frame areas; for example, in the case where there are N detection frame areas in each image frame, the electronic device may enlarge the areas of the N detection frame areas by a second preset ratio, thereby obtaining N second areas, where N is an integer greater than 0. It should be noted that enlarging the area of ​​a detection frame area by a second preset ratio means enlarging the range included in the detection frame area by the second preset ratio, thereby obtaining a second area with a larger range.

[0118] In the embodiment of the present disclosure, the second preset ratio can be set according to actual needs. For example, the second preset ratio can be 2, and the embodiment of the present disclosure is not limited to this.

[0119] S502: Determine a summarized region position according to a position of each second region in the at least one second region in the corresponding image frame.

[0120] In some embodiments, for all the second areas corresponding to the at least one image frame obtained, the electronic device can determine a generalized area position corresponding to all the second areas corresponding to the at least one image frame based on the position information of each second area in the corresponding image frame, and make the area corresponding to the generalized area position include any second area among all the second areas corresponding to the at least one image frame.

[0121] Here, the generalized region position can be region coordinates. For all the second regions corresponding to at least one image frame, the generalized region position can be the union of the region coordinates of all the second regions corresponding to at least one image frame. For example, it can be the smallest union. For illustration, when two second regions corresponding to two image frames are obtained, and the region coordinates of the first second region m11 in the first image frame are (x11, y11), (x12, y12), and the region coordinates of the second second region m12 in the second image frame are (x21, y21), (x22, y22), and x11 < x12 < x21 < x22, y21 < y22 < y11 < y12, the smallest coordinate union of the region coordinates of these two second regions is: (x11, y21), (x22, y12); where the region corresponding to the smallest coordinate union in the first image frame contains the first second region m11, and the region corresponding to the smallest coordinate union in the second image frame contains the second second region m12. For example, Fig.12 shows the region coordinates (x11, y11)(x12, y12) of the second region m11 in the first image frame, the region coordinates (x21, y21), (x22, y22) of the second region m12 in the second image frame, and the region 111 corresponding to the smallest coordinate union between these two second regions, where the region 111 contains the region 112 corresponding to the region coordinates (x11, y11)(x12, y12), and contains the region 113 corresponding to the region coordinates (x21, y21), (x22, y22).

[0122] S503. For each image frame, determine the region corresponding to the generalized region position in this image frame as the important region of this image frame; the important region of each image frame contains the second region corresponding to this image frame.

[0123] When a generalized region position commonly corresponding to all the regions of interest of at least one image frame is obtained, for each of at least one image frame, the electronic device can determine the region of the generalized region position in this image frame, and use the determined region as the important region of this image frame, where the important region of this image frame contains the second region of this image frame.

[0124] The present disclosure also provides a behavior recognition device, Fig.13 which is a schematic structural diagram of the behavior recognition device provided by the embodiments of the present disclosure; as Fig.13As shown, the behavior recognition device 1 includes: a recognition unit 11, which is used to classify and recognize important areas of each image frame in at least one image frame of the obtained image frame sequence, and obtain a recognition result corresponding to the at least one image frame; the important area is determined according to the detection frame area where the target object in the at least one image frame is located; a determination unit 12, which is used to determine the activation center of the class activation map corresponding to the recognition result of each image frame when the recognition result represents the existence of a preset abnormal event, and obtain at least one activation center of the at least one image frame; an activation center represents an abnormal position in the class activation map of the corresponding image frame; the recognition unit 11 is also used to identify the abnormal target objects of the respective image frames based on the at least one activation center.

[0125] In some embodiments of the present disclosure, the determination unit 12 is further used to determine a set of activation values ​​corresponding to the recognition result for each image frame; the set of activation values ​​is used to generate a class activation map for each image frame, and each pixel position in the class activation map corresponds one-to-one to an activation value in the set of activation values; the maximum activation value in the set of activation values ​​for each image frame is determined, and the pixel position corresponding to the maximum activation value is determined as the abnormal position; based on the abnormal position of each image frame, the activation center of each image frame is obtained, and for the at least one image frame, the at least one activation center is obtained accordingly.

[0126] In some embodiments of the present disclosure, the identification unit 11 is also used to determine the position of each activation center in a corresponding image frame; when the position is located in any detection frame area in the image frame, the target object corresponding to any detection frame area is determined as an abnormal target object in the image frame, and for the at least one activation center, the abnormal target object of each image frame is obtained.

[0127] In some embodiments of the present disclosure, the recognition unit 11 is further used to determine a target activation center based on the at least one activation center; and to identify abnormal target objects in each image frame based on the target activation center.

[0128] In some embodiments of the present disclosure, the recognition unit 11 is further used to cluster the at least one activation center by an unsupervised clustering method to obtain a target activation center; and identify abnormal target objects in each image frame based on the target activation center.

[0129] In some embodiments of the present disclosure, the identification unit 11 is also used to cluster the at least one activation center by an unsupervised clustering method to obtain a center cluster containing the largest number of activation centers; determine the average value of the activation centers contained in the center cluster, and determine the average value as the target activation center.

[0130] In some embodiments of the present disclosure, the determination unit 12 is further used to generate a set of initial class activation mapping values ​​corresponding to the recognition results for each image frame; upsample the initial class activation mapping values ​​to obtain a set of intermediate activation values; the set of intermediate activation values ​​is used to generate a class activation map corresponding to the important areas of each image frame; and the set of intermediate activation values ​​is fused with the original pixel values ​​of each image frame to obtain the set of activation values.

[0131] In some embodiments of the present disclosure, the behavior recognition device 1 also includes a detection unit 13, and the detection unit 13 is used to classify and identify important areas of each image frame in at least one image frame of the acquired image frame sequence, and before obtaining a recognition result corresponding to at least one image frame, perform target detection on the current image frame in the image frame sequence to determine at least one detection frame area corresponding to at least one existing target object; the determination unit 12 is also used to determine a central detection frame area in the at least one detection frame area; when the at least one image frame has been detected, determine the important areas of each image frame in the at least one image frame based on the at least one central detection frame area corresponding to the at least one image frame.

[0132] In some embodiments of the present disclosure, the determination unit 12 is further used to expand the at least one detection frame area by a first preset proportion to obtain at least one first area corresponding to the at least one detection frame area; based on the area intersection and union ratio between any two first areas in the at least one first area, determine the matching result between the any two first areas; the matching result characterizes whether the any two first areas match; based on the matching result, determine the number of matches for each first area, and determine the first area with the largest number of matches; based on the first area with the largest number of matches, determine the dense center, and determine the detection frame area corresponding to the dense center as the central detection frame area.

[0133] In some embodiments of the present disclosure, the determination unit 12 is further used to expand at least one central detection frame area corresponding to the at least one image frame by a second preset ratio to obtain at least one second area corresponding to the at least one detection frame area; determine the generalized area position according to the position of each second area in the image frame to which it belongs; for each image frame, determine the area corresponding to the generalized area position in each image frame as the important area of ​​each image frame; and the important area of ​​each image frame includes the second area corresponding to each image frame.

[0134] In some embodiments of the present disclosure, the determination unit 12 is further used to, when there is one first region with the largest number of matches, determine the first region with the largest number of matches as the dense center; when there are two or more first regions with the largest number of matches, determine the first region with the largest area among the two or more first regions with the largest number of matches as the dense center.

[0135] The present disclosure also provides an electronic device, Fig.14 A schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure, such as Fig.14 As shown, the electronic device 2 includes: a memory 21 and a processor 22, wherein the memory 21 and the processor 22 are connected via a bus 23; the memory 21 is used to store executable computer programs; the processor 22 is used to implement the method provided by the embodiment of the present disclosure when executing the executable computer program stored in the memory 21, for example, the behavior recognition method provided by the embodiment of the present disclosure.

[0136] The embodiment of the present disclosure provides a computer-readable storage medium storing a computer program for causing the processor 22 to execute and implement the method provided by the embodiment of the present disclosure, for example, the behavior recognition method provided by the embodiment of the present disclosure.

[0137] In some embodiments of the present disclosure, the storage medium may be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, flash memory, magnetic surface memory, optical disk, or CD-ROM; or it may be various devices including one or any combination of the above memories.

[0138] In some embodiments of the present disclosure, the computer program may be in the form of software, software modules, scripts or codes, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as a stand-alone program or as a module, component, subroutine or other unit suitable for use in a computing environment.

[0139] As an example, a computer program may, but need not, correspond to a file in a file system, may be stored as part of a file that stores other programs or data, for example, in one or more scripts in a HyperText Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple coordinated files (for example, files storing one or more modules, subroutines, or code portions).

[0140] By way of example, a computer program may be deployed to be executed on one computing device or on multiple computing devices at one site or on multiple computing devices distributed across multiple sites and interconnected by a communication network.

[0141] In summary, by adopting the technical implementation scheme, on the one hand, since the important areas of the determined image frames are classified and identified, compared with the identification based on the data enhancement of the whole image of each image frame, the search range during identification is effectively narrowed, and the interference during identification is reduced, thereby improving the recognition accuracy; on the other hand, since the abnormal center (activation center) of each image frame is located by determining the abnormal position in the class activation map corresponding to the recognition result of each image frame, and the abnormal target object in the corresponding image frame is identified according to the obtained abnormal center, and the class activation map corresponding to the recognition result can characterize the target object in the image frame related to the recognition result, the abnormal target object of each image frame is identified more accurately, thereby improving the recognition accuracy of the abnormal target object; therefore, the behavior recognition method provided by the embodiment of the present disclosure can improve the recognition accuracy.

[0142] The above description is only an embodiment of the present disclosure and is not intended to limit the protection scope of the present disclosure. Any modification, equivalent replacement and improvement made within the spirit and scope of the present disclosure are included in the protection scope of the present disclosure.

Claims

1. A behavior recognition method, characterized in that: include: Classifying and identifying important areas of each image frame in at least one image frame of the acquired image frame sequence to obtain an identification result corresponding to the at least one image frame; The important area is determined according to a detection frame area where the target object is located in the at least one image frame; In a case where the recognition result indicates the presence of a preset abnormal event, determining a set of activation values ​​corresponding to the recognition result for each image frame; The set of activation values ​​is used to generate a class activation map of each image frame, and each pixel position in the class activation map corresponds one-to-one to an activation value in the set of activation values; Determine a maximum activation value among a set of activation values ​​of each image frame, and determine a pixel position corresponding to the maximum activation value as an abnormal position; Based on the abnormal position of each image frame, an activation center of each image frame is obtained, and for the at least one image frame, at least one activation center is correspondingly obtained; an activation center represents an abnormal position in the class activation map of a corresponding image frame; Based on the at least one activation center, identifying abnormal target objects in each of the image frames; The step of identifying abnormal target objects in each image frame based on the at least one activation center includes: Determine the position of each activation center in a corresponding image frame; When the position is located in any detection frame area in the image frame, the target object corresponding to any detection frame area is determined as an abnormal target object in the image frame, and for the at least one activation center, the abnormal target object of each image frame is obtained.

2. The method according to claim 1, characterized in that The step of identifying abnormal target objects in each image frame based on the at least one activation center further includes: Determining a target activation center according to the at least one activation center; According to the target activation center, abnormal target objects in each image frame are identified.

3. The method according to claim 1, characterized in that The step of identifying abnormal target objects in each image frame based on the at least one activation center further includes: Clustering the at least one activation center by an unsupervised clustering method to obtain a target activation center; According to the target activation center, abnormal target objects in each image frame are identified.

4. The method according to claim 3, characterized in that The step of clustering the at least one activation center by an unsupervised clustering method to obtain a target activation center includes: Clustering the at least one activation center by an unsupervised clustering method to obtain a center cluster containing the largest number of activation centers; An average value of the activation centers included in the center cluster is determined, and the average value is determined as the target activation center.

5. The method according to claim 1, characterized in that The determining of a set of activation values ​​corresponding to the recognition result for each image frame comprises: Generating a set of initial class activation map values ​​corresponding to the recognition result for each image frame; Upsampling the initial class activation map values ​​to obtain a set of intermediate activation values; the set of intermediate activation values ​​is used to generate a class activation map corresponding to the important area of ​​each image frame; The set of intermediate activation values ​​is fused with the original pixel value of each image frame to obtain the set of activation values.

6. The method according to any one of claims 1 to 5, characterized in that: Before classifying and identifying the important areas of at least one image frame in the obtained image frame sequence and obtaining the identification result corresponding to the at least one image frame, the method further includes: Performing target detection on a current image frame in the image frame sequence to determine at least one detection frame area corresponding to at least one existing target object; Determine a central detection frame area among the at least one detection frame area; In a case where the at least one image frame has been detected, the important area of ​​each image frame in the at least one image frame is determined based on at least one central detection frame area corresponding to the at least one image frame.

7. The method according to claim 6, characterized in that The determining a central detection frame area in the at least one detection frame area comprises: Enlarging the at least one detection frame area by a first preset ratio to obtain at least one first area corresponding to the at least one detection frame area; Determine a matching result between any two first regions based on an area intersection-and-union ratio between any two first regions in the at least one first region; the matching result indicates whether the any two first regions match; Based on the matching results, determining the number of matches for each first region, and determining the first region with the largest number of matches; A dense center is determined according to the first area with the largest number of matches, and a detection frame area corresponding to the dense center is determined as the central detection frame area.

8. The method according to claim 6, characterized in that The determining the important area of ​​each image frame in the at least one image frame based on at least one central detection frame area corresponding to the at least one image frame includes: Enlarging at least one central detection frame area corresponding to the at least one image frame by a second preset ratio to obtain at least one second area corresponding to the at least one detection frame area; Determine a generalized region position according to a position of each second region in the image frame to which it belongs; For each image frame, the area corresponding to the position of the generalized area in each image frame is determined as the important area of ​​each image frame; the important area of ​​each image frame includes the second area corresponding to each image frame.

9. The method according to claim 7, characterized in that: Determining the dense center according to the first area with the largest number of matches includes: In the case where there is a first region with the largest number of matches, determining the first region with the largest number of matches as the dense center; In the case that there are two or more first regions with the largest number of matches, the first region with the largest area among the two or more first regions with the largest number of matches is determined as the dense center.

10. A behavior recognition device, characterized in that: include: an identification unit, configured to classify and identify important regions of each image frame in at least one image frame of the acquired image frame sequence, and obtain an identification result corresponding to the at least one image frame; The important area is determined according to a detection frame area where the target object is located in the at least one image frame; A determination unit, configured to determine, in a case where the recognition result indicates the presence of a preset abnormal event, a set of activation values ​​corresponding to the recognition result of each image frame; The set of activation values ​​is used to generate a class activation map of each image frame, and each pixel position in the class activation map corresponds one-to-one to an activation value in the set of activation values; Determine a maximum activation value among a set of activation values ​​of each image frame, and determine a pixel position corresponding to the maximum activation value as an abnormal position; Based on the abnormal position of each image frame, an activation center of each image frame is obtained, and for the at least one image frame, at least one activation center is correspondingly obtained; an activation center represents an abnormal position in the class activation map of a corresponding image frame; The identification unit is further configured to identify abnormal target objects in each image frame based on the at least one activation center; The recognition unit is also used to determine the position of each activation center in a corresponding image frame; when the position is located in any detection frame area in the image frame, the target object corresponding to any detection frame area is determined as an abnormal target object in the image frame, and for the at least one activation center, the abnormal target object of each image frame is obtained.

11. An electronic device, characterized in that: include: a memory for storing executable computer programs; A processor, configured to implement the method according to any one of claims 1 to 9 when executing the executable computer program stored in the memory.

12. A computer-readable storage medium, characterized in that: A computer program is stored, which is used to cause a processor to execute and implement the method according to any one of claims 1 to 9.

Images