Abnormal behavior detection method and device based on video stream

Abstract

The invention relates to the technical field of video recovery, and discloses an abnormal behavior detection method and device based on a video stream, and the method comprises the steps: obtaining a target image frame from the video stream; when a target area is detected from the target image frame by using the target detection network and the cumulative number of times of the target area identifier is greater than a number-of-times threshold, it is determined that an abnormal behavior occurs, the target area is an area in which the probability of the abnormal behavior is greater than or equal to a probability threshold, and the target area identifier is a tracking identifier of the target area; the cumulative number of times indicates the number of the detected target areas with the target area identifiers. According to the method, the detection precision of the abnormal behavior and the robustness of the system can be improved by means of the target area identifier, the cumulative number of times, the probability threshold and the like, and meanwhile, the method has high efficiency and real-time response capability.

Description

Technical Field

[0001] The present invention relates to the technical field of performance testing, and in particular to a method and device for detecting abnormal behavior based on video streams. Background Art

[0002] Existing security monitoring systems usually rely on manual real-time monitoring of abnormal situations, but this manual monitoring method has the problems of long response time, low efficiency and high dependence on human resources. Therefore, developing an abnormal behavior recognition method based on video streams that can detect in real time and automatically alarm has become an important means to improve public safety and protect people's lives and property.

[0003] Most of the current abnormal behavior detection technologies rely on pedestrian posture analysis or distance changes between pedestrians to identify abnormal behavior. However, pedestrian postures vary, and some occlusions may occur during monitoring, resulting in inaccurate posture estimation; and the relative distance between pedestrians is greatly affected by the camera's viewing angle and scene layout, which limits the accuracy of distance calculation. Therefore, traditional detection methods based on posture and distance have certain deficiencies in accuracy and stability, and are difficult to cope with complex and dynamic monitoring environments. Summary of the invention

[0004] In view of this, the present invention provides a method, apparatus, computer equipment, and storage medium for detecting abnormal behavior based on video streams to solve the problem of how to accurately detect abnormal behavior in a monitoring environment.

[0005] In a first aspect, the present invention provides a method for detecting abnormal behavior based on a video stream, the method comprising:

[0006] Get the target image frame from the video stream;

[0007] When a target area is detected from a target image frame using a target detection network and the cumulative number of target area identifications is greater than a number threshold, it is determined that abnormal behavior has occurred, wherein the target area is an area where the probability of the existence of abnormal behavior is greater than or equal to a probability threshold, the target area identification is a tracking identification of the target area, and the cumulative number indicates the number of target areas with target area identifications that have been detected.

[0008] The present invention provides a method for detecting abnormal behavior based on video stream, which has the following advantages:

[0009] By acquiring the target image frame from the video stream; when the target area is detected from the target image frame using the target detection network and the cumulative number of target area identifications is greater than the number threshold, it is determined that abnormal behavior has occurred, wherein the target area is an area where the probability of abnormal behavior is greater than or equal to the probability threshold, the target area identification is the tracking identification of the target area, and the cumulative number indicates the number of target areas with target area identifications that have been detected. The area is tracked in subsequent video frames to determine whether abnormal behavior continues to occur, thereby filtering out false alarms and occasional anomalies, and enhancing the accuracy and stability of the system in identifying real abnormal behavior.

[0010] In an optional embodiment, the method further includes:

[0011] Acquire a pending area in the target image frame, where the pending area is an area where the probability of the existence of abnormal behavior is less than a probability threshold;

[0012] Input the undetermined area into the classification network to obtain the classification result;

[0013] When the classification result shows that the pending area has abnormal behavior, the cumulative number of tracking marks corresponding to the pending area is updated.

[0014] In an optional implementation, detecting a target region from a target image frame using a target detection network includes:

[0015] Input the target image frame into the backbone network of the target detection network to obtain feature data of the target area of ​​the image frame;

[0016] Input the feature data into the global attention mechanism module to obtain the adjusted feature data;

[0017] Inputting the adjusted feature data into a detection head of a target detection network to obtain a probability that at least one region of the target image frame has abnormal behavior;

[0018] An area in which the probability of abnormal behavior existing in at least one area is greater than a probability threshold is determined as a target area.

[0019] In an optional implementation, the feature data is input into a global attention mechanism module to obtain adjusted feature data, including:

[0020] After converting the dimension of the feature data, the feature data is input into a two-layer perceptron to obtain the first updated feature data;

[0021] After normalizing the first updated feature data, the first updated feature data is input into the first convolution layer to reduce the number of channels, and then input into the second convolution layer to restore the number of channels to obtain the second updated feature data;

[0022] After the second updated feature data is normalized, adjusted feature data is obtained.

[0023] In an optional embodiment, the method further includes:

[0024] When it is determined that abnormal behavior has occurred, an alarm message is sent to the monitoring terminal, and an alarm mark of the target area is set, the alarm mark indicating that the abnormal behavior of the target area has been alarmed.

[0025] In an optional embodiment, the method further includes:

[0026] When abnormal behavior is determined, detect whether there is an alarm mark in the target area;

[0027] If not, an alarm message is sent to the monitoring terminal and an alarm mark of the target area is set.

[0028] In a second aspect, the present invention provides an abnormal behavior detection device based on video stream, the device comprising:

[0029] An acquisition module is used to acquire a target image frame from a video stream;

[0030] A determination module is used to determine that abnormal behavior has occurred when a target area is detected from a target image frame using a target detection network and the cumulative number of target area identifications is greater than a number threshold, wherein the target area is an area where the probability of the existence of abnormal behavior is greater than or equal to the probability threshold, the target area identification is a tracking identification of the target area, and the cumulative number indicates the number of target areas with target area identifications that have been detected.

[0031] In a third aspect, the present invention provides a computer device, comprising: a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the abnormal behavior detection method based on video stream of the above-mentioned first aspect or any corresponding embodiment thereof by executing the computer instructions.

[0032] In a fourth aspect, the present invention provides a computer-readable storage medium having computer instructions stored thereon, the computer instructions being used to enable a computer to execute the abnormal behavior detection method based on video stream according to the first aspect or any corresponding embodiment thereof.

[0033] In a fifth aspect, the present invention provides a computer program product, comprising computer instructions, wherein the computer instructions are used to enable a computer to execute the hardware monitoring method based on the Redfish interface of the above-mentioned first aspect or any corresponding embodiment thereof. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0035] Figure 1 is one of the flow charts of the abnormal behavior detection method based on video stream according to an embodiment of the present invention;

[0036] Figure 2 is a schematic diagram of a target detection network according to an embodiment of the present invention;

[0037] Figure 3 is a schematic diagram of a global attention mechanism module according to an embodiment of the present invention;

[0038] Figure 4 2 is a flow chart of an abnormal behavior detection method based on video stream according to an embodiment of the present invention;

[0039] Figure 5 is a schematic diagram of an abnormal behavior detection device based on video stream according to an embodiment of the present invention;

[0040] Figure 6 It is a schematic diagram of the hardware structure of a computer device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0041] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0042] According to an embodiment of the present invention, an embodiment of an abnormal behavior detection method based on video stream is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0043] In this embodiment, a method for detecting abnormal behavior based on video stream is provided, which can be used in the above-mentioned computer. Figure 1 FIG. 1 is a flow chart of a method for detecting abnormal behavior based on video stream according to an embodiment of the present invention. The method includes:

[0044] Step S101, obtaining a target image frame from a video stream.

[0045] In this embodiment, the video stream is composed of continuous image frames, and the target image frame can be any image frame in the video stream. Specifically, the target image frame can be quickly extracted from the video stream through hardware acceleration or streaming media decoding.

[0046] Step S102, when the target area is detected from the target image frame using the target detection network and the cumulative number of target area identifications is greater than the number threshold, it is determined that abnormal behavior occurs.

[0047] Among them, the above-mentioned target area is an area where the probability of the above-mentioned abnormal behavior existing is greater than or equal to the probability threshold, the above-mentioned target area identifier is the tracking identifier of the above-mentioned target area, and the above-mentioned cumulative number indicates the number of target areas with the above-mentioned target area identifier that have been detected.

[0048] In this embodiment, the target detection network can be a YOLOv7 (You Only Look One version7) model, a single-stage target detection algorithm that identifies and locates objects based on a deep neural network. The tracking identifier can be the positioning coordinates of the target area, and the cumulative number can be recorded in the memory in the form of a key-value pair corresponding to the tracking identifier, and the tracking identifier is used as a key in a string type, and the cumulative number is used as a value of the key in an integer form. The probability threshold can be a probability value used to filter out low-probability detection results. Specifically, using the target detection network to detect the target area from the target image frame can be to input the target image frame into the target detection network, obtain the positioning coordinates of the target area, and the probability that the target area has the above-mentioned abnormal behavior, and use the positioning coordinates as its tracking identifier.

[0049] The present invention provides a method for detecting abnormal behavior based on video stream, which has the following advantages:

[0050] By acquiring the target image frame from the video stream; when the target area is detected from the target image frame using the target detection network and the cumulative number of target area identifications is greater than the number threshold, it is determined that abnormal behavior has occurred, wherein the target area is an area where the probability of abnormal behavior is greater than or equal to the probability threshold, the target area identification is the tracking identification of the target area, and the cumulative number indicates the number of target areas with target area identifications that have been detected. The area is tracked in subsequent video frames to determine whether abnormal behavior continues to occur, thereby filtering out false alarms and occasional anomalies, and enhancing the accuracy and stability of the system in identifying real abnormal behavior.

[0051] Furthermore, it is possible to use ByteTrack technology in multi-target tracking, a tracking method of the tracking-by-detection paradigm. By retaining the detection box (including the above-mentioned target area) and performing identity matching, the detection box is divided into two groups: high confidence and low confidence. First, the high-confidence detection box is associated with the trajectory to complete the tracking, and then the low-confidence detection box is used to use its similarity with the unmatched object (by calculating the similarity between them to determine whether they belong to the same target) and filter out the background to complete the tracking. For multiple possible abnormal areas in the image, they are tracked separately, and the unique abnormal area ID is marked and sent to the downstream task of identifying and confirming abnormal behavior. Such as behavior recognition, event classification, etc., for more advanced analysis (such as determining whether it is an abnormal behavior, whether an alarm is required, etc.).

[0052] In some optional implementations, the above method further includes:

[0053] Obtain a pending area in the target image frame, where the probability of the existence of the abnormal behavior is less than the probability threshold. Input the pending area into the classification network to obtain a classification result. If the classification result shows that the pending area has abnormal behavior, update the cumulative number of tracking marks corresponding to the pending area.

[0054] In this embodiment, the classification network can be a binary classification task, such as using a ResNet-18 (a classic convolutional neural network architecture, which is part of the ResNet (Residual Networks) series, and its network architecture includes 18 layers) network to classify the above-mentioned located area to be determined. The feature values ​​of the identified area to be determined are used as input and passed to the classification network for further analysis to determine whether there is abnormal behavior in the area to be determined. The classification network outputs a type label (such as "normal" or "abnormal"), or a probability value associated with the abnormal behavior. After the classification network confirms that there is abnormal behavior in the area to be determined, the cumulative number of tracking identifiers corresponding to the area needs to be updated.

[0055] The abnormal behavior detection method based on video stream in this embodiment can realize accurate identification of abnormal behavior by obtaining the pending area in the target image frame and inputting it into the classification network for classification. By updating the cumulative number of tracking marks of the pending area where abnormal behavior exists, the monitoring range of abnormal behavior monitored by the target detection model is supplemented, and the status of the area can be further updated in time to accumulate the number of abnormal behavior occurrences. It helps to gradually improve the ability to identify abnormal behavior and make tracking and monitoring more accurate.

[0056] In some optional implementations, detecting a target region from a target image frame using a target detection network includes:

[0057] The target image frame is input into the backbone network of the target detection network to obtain feature data of the target region of the image frame. The feature data is input into the Global Attention Mechanism (GAM) module to obtain adjusted feature data. The adjusted feature data is input into the detection head of the target detection network to obtain the probability that at least one region of the target image frame has abnormal behavior. The region where the probability of the abnormal behavior in at least one region is greater than the probability threshold is determined as the target region.

[0058] In this embodiment, Figure 2 : is a schematic diagram of a target detection network according to an embodiment of the present invention. The target detection network includes an input (Input), a backbone network (Backbone), a detection head (Head), and an output (Output), which are respectively used to input raw data (such as a target image frame), extract features of the input data (i.e., the above-mentioned feature data), predict specific tasks based on the extracted features (such as classification, regression), and the final prediction result of the model (such as the above-mentioned probability). In the above steps, a global attention mechanism module is added between the backbone network and the detection head in the target detection network to improve performance by reducing information loss and amplifying global interaction representation. The GAM module mainly includes a channel attention module and a spatial attention module. The channel attention module makes the network pay more attention to the foreground and the area of ​​interest of the image, that is, the area where abnormal behavior exists; and the spatial attention module can make the network pay attention to the location of the entire image context information. Further, the output end of the backbone network is connected to the CBS (Cross-Batch Smoothing, a technology for optimizing the generalization ability of the model) module, and the output of the CBS module is connected to the GAM module. The CBS module consists of three modules: Conv (convolutional layer, used to extract features and perform spatial transformation), BN (batch normalization), and SiLU (Sigmoid-weighted Linear Unit, activation function). The output of the CBS module is 1024-dimensional, and the input and output of the GAM module are both 1024-dimensional.

[0059] The abnormal behavior detection method based on video stream in this embodiment, through the global attention mechanism module, can automatically focus on key areas where abnormal behavior may exist when extracting target area features, reducing interference with irrelevant areas. This can significantly improve the accuracy of abnormal behavior detection, especially in complex backgrounds or multiple target scenes. The backbone network of the target detection network first extracts the feature data of the target area, and then further optimizes these features through the adjustment of the global attention mechanism. Thereby enhancing the detection model's perception of the target area, especially when the target area is more complex or has greater changes, it can better identify abnormal behavior.

[0060] In some optional implementations, the above-mentioned feature data is input into the global attention mechanism module to obtain adjusted feature data, including:

[0061] After converting the dimension of the feature data, the feature data is input into a two-layer perceptron to obtain the first updated feature data. After normalizing the first updated feature data, the feature data is input into a first convolutional layer to reduce the number of channels, and then input into a second convolutional layer to restore the number of channels to obtain the second updated feature data. After normalizing the second updated feature data, the adjusted feature data is obtained.

[0062] In this embodiment, Figure 3 : is a schematic diagram of a global attention mechanism module according to an embodiment of the present invention. Channel attention module: The input feature data will undergo a certain dimensional conversion (for example, from C×W×H to W×H×C, with C channels, W width, and H height), and its shape or number of channels will be adjusted to a format suitable for subsequent operations, and then input into an MLP (Multi-Layer Perceptron) network to amplify the cross-dimensional channel and spatial dependencies. The network is usually composed of two fully connected layers with a nonlinear activation function, and this process is used to capture more complex feature relationships. Then the first updated feature data is processed with Sigmoid (an activation function) to obtain weighted and normalized feature data. Spatial attention module: First, the number of channels of the normalized feature data is reduced by a 7×7 convolution kernel, and then the number of channels is increased to the original number by a 7×7 convolution kernel. The spatial information is fused through two convolution layers, and at the same time, the parameters are prevented from increasing significantly. A group convolution strategy can be adopted. Finally, the adjusted feature data is output through Sigmoid processing.

[0063] The abnormal behavior detection method based on video stream in this embodiment, by introducing a two-layer perceptron, performs nonlinear mapping on the input features, so that the network can capture more complex feature relationships. This nonlinear transformation enables the network to not only learn simple linear features, but also capture higher-order feature combinations, enhancing the expressive power of the model. This reduces redundant information and reduces the amount of calculation. This process helps to reduce the complexity of the model, avoid overfitting, and speed up the training process. It helps to reconstruct important feature information. Through the channel recovery operation, the model can retain the features of interest and enhance the integrity of feature expression.

[0064] Furthermore, the training set of the target detection network can be derived from online data sets, crawling, and purchasing, and then the data set can be expanded through data enhancement strategies. The fighting area is manually annotated as a training set and sent to the target detection network for training.

[0065] In some optional implementations, the above method further includes:

[0066] When it is determined that abnormal behavior has occurred, an alarm message is sent to the monitoring terminal, and an alarm mark of the target area is set, wherein the alarm mark indicates that the abnormal behavior of the target area has been alarmed.

[0067] In this embodiment, the monitoring end may be a mobile terminal for receiving alarm information. The alarm information may be information such as the time and location of abnormal behavior. The alarm identifier may be the above-mentioned cumulative number or another value that is a key-value pair with the tracking identifier. Specifically, a map with LRU (Least Recently Used) characteristics may be created, referred to as L. Its key is a tracking identifier of string type, and its value is a combination of two data, including an integer representing the cumulative number and a Boolean type (alarm identifier) ​​representing whether an alarm has been issued.

[0068] The abnormal behavior detection method based on video stream in this embodiment can detect abnormal behavior in video stream in real time and accurately judge whether similar abnormal events have occurred by dynamically setting and updating alarm flags. This can avoid repeated detection and false alarms, and improve the accuracy and timeliness of abnormal behavior detection.

[0069] In some optional implementations, the above method further includes:

[0070] When it is determined that an abnormal behavior occurs, it is detected whether the target area has the warning mark. If not, the warning information is sent to the monitoring terminal, and the warning mark of the target area is set.

[0071] In this embodiment, for each target image frame, it is determined whether the target area has abnormal behavior. For the same tracking mark, if an alarm has been issued, the tracking mark is skipped to avoid repeated alarms; otherwise, the cumulative number is increased by one. When the cumulative number reaches the number threshold, an alarm is issued and an alarm is set.

[0072] The abnormal behavior detection method based on video stream in this embodiment checks whether the target area has been set with an alarm mark, and sends an alarm message when no abnormality is marked. This avoids multiple alarms for the same abnormal event, reduces invalid alarm messages, and ensures the effectiveness and real-time nature of the alarm message.

[0073] Figure 4 FIG2 is a second flow chart of a method for detecting abnormal behavior based on video stream according to an embodiment of the present invention, the method comprising:

[0074] Step S401, obtaining a target image frame from a video stream.

[0075] Step S402: locate the area to be tested in the target image frame.

[0076] Step S403: Tracking the area to be measured.

[0077] Step S404, determine whether the probability of abnormal behavior in the detected area is greater than or equal to the probability threshold. If yes, execute step S406; if not, execute step 405.

[0078] Step S405: output the area to be tested to the classification model to determine whether there is abnormal behavior.

[0079] Step S406: filter the area to be tested after sending the alarm information.

[0080] The present invention provides an abnormal behavior detection device based on video stream. Figure 5 is a schematic diagram of an abnormal behavior detection device based on video stream according to an embodiment of the present invention, the device comprising:

[0081] The acquisition module 501 is used to acquire a target image frame from a video stream.

[0082] The determination module 502 is used to determine that abnormal behavior has occurred when a target area is detected from a target image frame using a target detection network and the cumulative number of target area identifications is greater than a number threshold, wherein the above-mentioned target area is an area where the probability of the above-mentioned abnormal behavior existing is greater than or equal to the probability threshold, the above-mentioned target area identification is a tracking identification of the above-mentioned target area, and the above-mentioned cumulative number indicates the number of target areas with the above-mentioned target area identification that have been detected.

[0083] The abnormal behavior detection device based on video stream provided by the present invention has the following advantages:

[0084] By acquiring the target image frame from the video stream; when the target area is detected from the target image frame using the target detection network and the cumulative number of target area identifications is greater than the number threshold, it is determined that abnormal behavior has occurred, wherein the target area is an area where the probability of abnormal behavior is greater than or equal to the probability threshold, the target area identification is the tracking identification of the target area, and the cumulative number indicates the number of target areas with target area identifications that have been detected. The area is tracked in subsequent video frames to determine whether abnormal behavior continues to occur, thereby filtering out false alarms and occasional anomalies, and enhancing the accuracy and stability of the system in identifying real abnormal behavior.

[0085] The further functional description of each of the above modules and units is the same as that of the above corresponding embodiments and will not be repeated here.

[0086] The video stream-based abnormal behavior detection device in this embodiment is presented in the form of a functional unit, where the unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that executes one or more software or fixed programs, and / or other devices that can provide the above functions.

[0087] The embodiment of the present invention also provides a computer device having the above Figure 5 The abnormal behavior detection device based on video stream is shown.

[0088] See also Figure 6 , Figure 6 is a schematic diagram of the structure of a computer device provided by an optional embodiment of the present invention, such as Figure 6 As shown, the computer device includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. Various components are connected to each other using different buses for communication, and can be installed on a common mainboard or installed in other ways as needed. The processor can process the instructions executed in the computer device, including instructions stored in or on the memory to display the graphical information of the GUI on an external input / output device (such as, a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple computer devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 6 A processor 10 is taken as an example.

[0089] The processor 10 may be a central processing unit, a network processor or a combination thereof. The processor 10 may further include an integrated circuit. The integrated circuit may be a dedicated integrated circuit, a programmable logic device or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic or any combination thereof.

[0090] The memory 20 stores instructions executable by at least one processor 10, so that the at least one processor 10 executes the method shown in the above embodiment.

[0091] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely arranged relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0092] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid state drive; the memory 20 may also include a combination of the above types of memory.

[0093] The computer device further comprises a communication interface 30 for the computer device to communicate with other devices or a communication network.

[0094] The embodiment of the present invention also provides a computer-readable storage medium. The method according to the embodiment of the present invention can be implemented in hardware, firmware, or can be implemented as a computer code that can be recorded in a storage medium, or can be implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and will be stored in a local storage medium through a network download, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state hard disk, etc.; further, the storage medium can also include a combination of the above types of memories. It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor, or hardware, the method shown in the above embodiment is implemented.

[0095] A part of the present invention may be applied as a computer program product, such as a computer program instruction, which, when executed by a computer, can call or provide the method and / or technical solution according to the present invention through the operation of the computer. Those skilled in the art should understand that the existence of the computer program instruction in a computer-readable medium includes, but is not limited to, a source file, an executable file, an installation package file, etc., and accordingly, the way in which the computer program instruction is executed by the computer includes, but is not limited to: the computer directly executes the instruction, or the computer compiles the instruction and then executes the corresponding compiled program, or the computer reads and executes the instruction, or the computer reads and installs the instruction and then executes the corresponding installed program. Here, the computer-readable medium may be any available computer-readable storage medium or communication medium accessible to the computer.

[0096] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A method for detecting abnormal behavior based on video stream, characterized in that: The method comprises: Get the target image frame from the video stream; When a target area is detected from a target image frame using a target detection network and the cumulative number of target area identifications is greater than a number threshold, it is determined that abnormal behavior has occurred, wherein the target area is an area where the probability of the existence of the abnormal behavior is greater than or equal to a probability threshold, the target area identification is a tracking identification of the target area, and the cumulative number indicates the number of target areas with the target area identification that have been detected.

2. The method according to claim 1, characterized in that The method further comprises: Acquire a pending area in the target image frame, where the pending area is an area where the probability of the abnormal behavior existing is less than a probability threshold; Inputting the undetermined area into a classification network to obtain a classification result; When the classification result is that the pending area has abnormal behavior, the accumulated number of tracking marks corresponding to the pending area is updated.

3. The method according to claim 1, characterized in that The target detection network is used to detect the target area from the target image frame, including: Inputting the target image frame into the backbone network of the target detection network to obtain feature data of the target area of ​​the image frame; Inputting the feature data into a global attention mechanism module to obtain adjusted feature data; Inputting the adjusted feature data into a detection head of the target detection network to obtain a probability that at least one region of the target image frame has abnormal behavior; An area in the at least one area where the probability of the abnormal behavior existing is greater than the probability threshold is determined as the target area.

4. The method according to claim 3, characterized in that The step of inputting the feature data into a global attention mechanism module to obtain adjusted feature data includes: After converting the dimension of the feature data, the feature data is input into a two-layer perceptron to obtain first updated feature data; After normalizing the first updated feature data, the first updated feature data is input into a first convolutional layer to reduce the number of channels, and then input into a second convolutional layer to restore the number of channels, thereby obtaining second updated feature data; After the second updated feature data is normalized, adjusted feature data is obtained.

5. The method according to claim 1, characterized in that: The method further comprises: When it is determined that abnormal behavior occurs, an alarm message is sent to the monitoring terminal, and an alarm mark of the target area is set, wherein the alarm mark indicates that the abnormal behavior of the target area has been alarmed.

6. The method according to claim 5, characterized in that The method further comprises: In the case where it is determined that abnormal behavior occurs, detecting whether the warning mark exists in the target area; If not, the alarm information is sent to the monitoring terminal, and the alarm flag of the target area is set.

7. An abnormal behavior detection device based on video stream, characterized in that: The device comprises: An acquisition module is used to acquire a target image frame from a video stream; A determination module is used to determine that abnormal behavior has occurred when a target area is detected from a target image frame using a target detection network and the cumulative number of target area identifications is greater than a number threshold, wherein the target area is an area where the probability of the existence of the abnormal behavior is greater than or equal to a probability threshold, the target area identification is a tracking identification of the target area, and the cumulative number indicates the number of target areas with the target area identification that have been detected.

8. A computer device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the abnormal behavior detection method based on video stream according to any one of claims 1 to 6 by executing the computer instructions.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the abnormal behavior detection method based on video stream according to any one of claims 1 to 6.

10. A computer program product, characterized in that The method comprises computer instructions, wherein the computer instructions are used to cause a computer to execute the abnormal behavior detection method based on video stream according to any one of claims 1 to 6.

Images