Regional abnormal behavior monitoring method, device, equipment and storage medium

Abstract

The present invention relates to artificial intelligence technology, and discloses a method for monitoring regional abnormal behavior, including: classifying monitoring images according to preset scenes and deleting invalid scenes to obtain target scene monitoring images; when using a pre-trained abnormal event recognition model to determine that there is an abnormal event in the target scene monitoring image, marking the corresponding target scene monitoring image as an abnormal evidence picture, extracting identification information and annotation box position information from the abnormal evidence picture; according to the identification information and the annotation box position information, using preset rules to delete repeated abnormal events in the cache, obtain the retained abnormal events and send them to the preset handler. The present invention also proposes a regional abnormal behavior monitoring device, an electronic device, and a storage medium. The present invention can improve the accuracy and low efficiency of abnormal behavior analysis.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a method, device, electronic device and computer-readable storage medium for monitoring regional abnormal behavior. Background Art

[0002] With the development of projects such as smart cities, surveillance equipment in the region is becoming more and more common, and people are increasingly identifying abnormal behaviors in videos captured by surveillance equipment.

[0003] However, currently, when using the video captured by the monitoring equipment to identify abnormal behavior, manual identification is usually used. This manual method has the following three main disadvantages: first, the video volume is huge, requiring a large amount of manpower to carry out daily manual verification; second, it is often necessary to replay to confirm whether the abnormal behavior is valid, which is inefficient; third, it depends on the personal ability and experience of the relevant staff, and there is a possibility of inaccurate identification due to human factors.

[0004] In summary, the accuracy and efficiency of current regional monitoring for abnormal behavior analysis are low. Summary of the invention

[0005] The present invention provides a method, a device and a computer-readable storage medium for monitoring abnormal behavior in a region, the main purpose of which is to solve the problem of low accuracy and efficiency in abnormal behavior analysis in regional monitoring.

[0006] To achieve the above object, the present invention provides a method for monitoring abnormal behavior in a region, comprising:

[0007] Obtaining a real-time surveillance video stream and de-framing the video stream to obtain a surveillance image;

[0008] Classifying the monitoring image according to a preset scene to obtain a scene monitoring image;

[0009] Identifying invalid scene images from the scene monitoring images, and deleting the invalid scene images to obtain a target scene monitoring image;

[0010] When it is determined that an abnormal event exists in the target scene surveillance image using a pre-trained abnormal event recognition model, the corresponding target scene surveillance image is marked as an abnormal forensic picture, identification information and annotation box position information of the abnormal event are extracted from the abnormal forensic picture, and the abnormal forensic picture, identification information and annotation box position information corresponding to the abnormal event are stored in a preset cache;

[0011] According to the identification information and the mark box position information, using a preset rule, the abnormal evidence pictures, identification information and mark box position information corresponding to the repeated abnormal events in the cache are deleted to obtain the abnormal evidence pictures, identification information and mark box position information corresponding to the retained abnormal events;

[0012] The abnormal evidence pictures, identification information and marked box position information corresponding to the retained abnormal events are stored in a preset abnormal library, and the abnormal evidence pictures, identification information and marked box position information corresponding to the abnormal events are sent to a preset handler.

[0013] Optionally, the monitoring image is classified according to a preset scene to obtain a scene monitoring image;

[0014] Performing grayscale and normalization processing on the monitoring image to obtain a standardized monitoring image;

[0015] Performing convolution operations, maximum pooling, and full connection operations on the standardized monitoring image at multiple scales to extract features of the standardized monitoring image;

[0016] The features are classified according to the preset scenes through the pre-built classification model to obtain the scene monitoring image.

[0017] Optionally, the identifying an invalid scene image from the scene monitoring image and deleting the invalid scene image to obtain a target scene monitoring image includes:

[0018] Extracting image features of the scene monitoring image;

[0019] Using preset invalid scene rules corresponding to different scene types to match the image features to obtain matching similarity;

[0020] According to the matching similarity, invalid scene images are screened out from the scene monitoring images, and the invalid scene images are deleted to obtain a target scene monitoring image.

[0021] Optionally, before using the pre-trained abnormal event recognition model to determine whether an abnormal event exists in the target scene surveillance image, the method further includes:

[0022] Extracting an initial feature map of the target scene monitoring image;

[0023] Using a preset attention mechanism module to adjust the weight of the initial feature map to obtain an attention feature map;

[0024] Performing convolution and average pooling operations on the attention feature map to obtain image features of the target scene monitoring image;

[0025] Calculating the probability value of the image feature of the target scene monitoring image belonging to a preset abnormal event using a preset activation function;

[0026] According to the probability value, it is determined whether there is any abnormal event in the target scene monitoring image.

[0027] Optionally, the deleting the abnormal forensic pictures, identification information and the annotation box position information corresponding to the repeated abnormal events in the cache using a preset rule according to the identification information and the annotation box position information, and obtaining the abnormal forensic pictures, identification information and the annotation box position information corresponding to the retained abnormal events, includes:

[0028] querying the cache according to the identification information to determine whether there is a target abnormal event identical to the identification information in the cache;

[0029] When there is no target abnormal event with the same identification information in the cache, the abnormal evidence picture, identification information and marked box position information corresponding to the abnormal event corresponding to the identification information are retained;

[0030] When there is a target abnormal event with the same identification information in the cache, judging whether the abnormal event is a duplicate of the target abnormal event based on the annotation box position information and the annotation box position information corresponding to the target abnormal event;

[0031] When the abnormal event is repeated with the target abnormal event, the abnormal evidence picture, identification information and marked box position information corresponding to the repeated abnormal event in the cache are deleted;

[0032] When the abnormal event does not overlap with the target abnormal event, the abnormal evidence picture, identification information and annotation box position information corresponding to the abnormal event are retained.

[0033] Optionally, querying the cache according to the identification information to determine whether there is a target abnormal event identical to the identification information in the cache, the method further includes:

[0034] Calculating the similarity between the identification information and the identification information in the cache;

[0035] It is determined whether there is a target abnormal event with the same identification information in the cache according to the similarity.

[0036] Optionally, judging whether the abnormal event is a duplicate of a target abnormal event based on the annotation box position information and the annotation box position information corresponding to the target abnormal event includes:

[0037] Matching the annotation frame position information with the position information of the annotation frame corresponding to the target abnormal event to obtain an overlap rate between the annotation frame position information and the position information of the annotation frame corresponding to the target abnormal event;

[0038] According to the overlap rate, it is determined whether the abnormal event overlaps with the target abnormal event.

[0039] In order to solve the above problems, the present invention further provides a regional abnormal behavior monitoring device, the device comprising:

[0040] A monitoring image acquisition module is used to acquire a real-time monitoring video stream and deframe the video stream to obtain a monitoring image;

[0041] An invalid scene deletion module is used to classify the monitoring images according to preset scenes to obtain scene monitoring images; identify invalid scene images from the scene monitoring images, delete the invalid scene images, and obtain a target scene monitoring image;

[0042] An abnormal event judgment module is used to use a pre-trained abnormal event recognition model to judge that there is an abnormal event in the target scene monitoring image, mark the corresponding target scene monitoring image as an abnormal forensic picture, extract the identification information and the marked box position information of the abnormal event from the abnormal forensic picture, and store the abnormal forensic picture, identification information and marked box position information corresponding to the abnormal event in a preset cache;

[0043] An abnormal event deduplication module is used to delete the abnormal evidence pictures, identification information and annotation box position information corresponding to the repeated abnormal events in the cache according to the identification information and the annotation box position information, using preset rules, and obtain the abnormal evidence pictures, identification information and annotation box position information corresponding to the retained abnormal events;

[0044] The abnormal event processing module is used to store the abnormal evidence pictures, identification information and marked box position information corresponding to the retained abnormal events into a preset abnormal library, and send the abnormal evidence pictures, identification information and marked box position information corresponding to the abnormal events to a preset handler.

[0045] In order to solve the above problem, the present invention further provides an electronic device, the electronic device comprising:

[0046] at least one processor; and,

[0047] a memory communicatively connected to the at least one processor; wherein,

[0048] The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the above-mentioned regional abnormal behavior monitoring method.

[0049] In order to solve the above problems, the present invention also provides a computer-readable storage medium, in which at least one computer program is stored. The at least one computer program is executed by a processor in an electronic device to implement the above-mentioned method for monitoring abnormal regional behavior.

[0050] The embodiment of the present invention obtains the target scene monitoring image by classifying the monitoring images according to the preset scenes and deleting the invalid scenes. Classification for different scenes is conducive to improving the recognition accuracy of abnormal behavior, deleting the invalid scenes reduces the images of abnormal event recognition, and improving the efficiency of abnormal event recognition; using the pre-trained abnormal event recognition model to determine whether there is an abnormal event in the target scene monitoring image, improve the feature extraction ability of the abnormal event recognition model, thereby improving the accuracy of abnormal event recognition; using the preset rules to delete the abnormal forensic pictures, identification information and annotation box position information corresponding to the repeated abnormal events in the cache, reduce the number of abnormal events, reduce the storage pressure of abnormal events, and improve the efficiency of abnormal behavior analysis. Therefore, the regional abnormal behavior monitoring method, device, electronic device and computer-readable storage medium proposed by the present invention can solve the problem of low accuracy and efficiency of regional monitoring for abnormal behavior analysis. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 A schematic diagram of a flow chart of a method for monitoring regional abnormal behavior provided by an embodiment of the present invention;

[0052] Figure 2 for Figure 1 A schematic diagram of a detailed implementation process of one of the steps in the method for monitoring abnormal behavior in the area shown;

[0053] Figure 3 for Figure 1 A schematic diagram of a detailed implementation process of one of the steps in the method for monitoring abnormal behavior in the area shown;

[0054] Figure 4 for Figure 1 A schematic diagram of a detailed implementation process of one of the steps in the method for monitoring abnormal behavior in the area shown;

[0055] Figure 5 A functional module diagram of a regional abnormal behavior monitoring device provided by an embodiment of the present invention;

[0056] Figure 6A schematic diagram of the structure of an electronic device for implementing the method for monitoring regional abnormal behavior provided by an embodiment of the present invention.

[0057] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0058] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0059] The embodiment of the present application provides a method for monitoring regional abnormal behavior. The execution subject of the method for monitoring regional abnormal behavior includes but is not limited to at least one of the electronic devices such as a server, a terminal, etc. that can be configured to execute the method provided by the embodiment of the present application. In other words, the method for monitoring regional abnormal behavior can be executed by software or hardware installed on a terminal device or a server device, and the software can be a blockchain platform. The server includes but is not limited to: a single server, a server cluster, a cloud server or a cloud server cluster, etc. The server can be an independent server, or it can be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (Content Delivery Network, CDN), and big data and artificial intelligence platforms.

[0060] Reference Figure 1 FIG. 1 is a flow chart of a method for monitoring regional abnormal behavior provided by an embodiment of the present invention. In this embodiment, the method for monitoring regional abnormal behavior includes:

[0061] S1. Obtain a real-time monitoring video stream, and deframe the video stream to obtain a monitoring image;

[0062] In the embodiment of the present invention, the monitoring video stream can be obtained by a device with monitoring and capturing function such as a camera or a video recorder installed in the monitored area, such as lamp posts on both sides of XX, XXXX entrance, City X, and key areas of Small X.

[0063] In one embodiment of the present invention, currently commonly used de-framing software may be used, for example, animation GIF production software, Jianying, etc. may be used to de-frame the video stream.

[0064] S2. Classify the monitoring image according to preset scenes to obtain a scene monitoring image;

[0065] In one embodiment of the present invention, the preset scenes include consumption XXX, horse racing XXX, community XXX, market XXX and other scenes.

[0066] In the embodiment of the present invention, a convolutional neural network with an image feature extraction function can be used to construct a classification model, and the monitoring image can be classified using the classification model. The convolutional neural network includes, but is not limited to, AlexNet network, ZFNet network, VGGNet network, ResNet50 and GoogLeNet network, etc.

[0067] In one embodiment of the present invention, the classification model is constructed using a GoogLeNet network, and the GoogLeNet network may be composed of a convolutional layer, a pooling layer, and a fully connected layer.

[0068] For details, see Figure 2 As shown, the S2 comprises:

[0069] S21, graying and normalizing the monitoring image to obtain a standardized monitoring image;

[0070] The embodiment of the present invention can obtain a standardized monitoring image of equal size and the same grayscale value by graying and normalizing the monitoring image. The preferred value of the grayscale value is between 0 and 1. The standardized monitoring image can reduce a large amount of calculations in the convolution calculation process and improve the classification efficiency of the monitoring image.

[0071] S22, performing convolution operations, maximum pooling, and full connection operations on the standardized monitoring image at multiple scales to extract features of the standardized monitoring image;

[0072] In one embodiment of the present invention, 1×1 convolution, 3×3 convolution and 5×5 convolution are used to extract features of the standardized monitoring image from multiple scales, so that the obtained features are richer and more accurate in classification judgment.

[0073] S23. Classify the features according to preset scenarios using a pre-built classification model to obtain a scene monitoring image.

[0074] Since the monitoring image contains multiple scenes, the abnormal behavior in each scene is different. Classifying the monitoring image according to preset scenes is conducive to improving the accuracy of identifying abnormal behaviors.

[0075] S3, identifying invalid scene images from the scene monitoring images, and deleting the invalid scene images to obtain a target scene monitoring image;

[0076] In the embodiment of the present invention, the invalid scenes include: scene monitoring images that do not include shopping XXX, do not include road X, do not include store X, do not include a place where XX is not allowed to be placed in the market X, etc.

[0077] For details, see Figure 3As shown, the S3 includes:

[0078] S31, extracting image features of the scene monitoring image;

[0079] S32, using preset invalid scene rules corresponding to different scene types to match the image features to obtain matching similarity;

[0080] S33. Filter out invalid scene images from the scene monitoring images according to the matching similarity, and delete the invalid scene images to obtain a target scene monitoring image.

[0081] In the embodiment of the present invention, the invalid scene rules are rules corresponding to various invalid scenes, for example, in the image features, image features such as elimination XXX and elimination XXX operation site do not appear.

[0082] Specifically, in an embodiment of the present invention, when the matching similarity is greater than or equal to a preset first similarity threshold, the corresponding scene surveillance image is retained as a target scene surveillance image; when the matching similarity is less than the preset first similarity threshold, the corresponding scene surveillance image is deleted as an invalid scene surveillance image.

[0083] In the embodiment of the present invention, invalid scene images are deleted to obtain target scene monitoring images, which can reduce the images for abnormal event recognition and improve the efficiency of abnormal event recognition.

[0084] S4, using a pre-trained abnormal event recognition model to determine whether there is an abnormal event in the target scene monitoring image;

[0085] The abnormal events described in the embodiment of the present invention include, but are not limited to, occupying fire XXX, occupying fire XXX operation site, horse XXX, community XXX store XX, city XXXXX.

[0086] In one embodiment of the present invention, the abnormal event recognition model can be a model of a convolution layer (convolution) between each block of a densely connected convolutional neural network (DenseNet) plus an attention mechanism module, and the model of the attention mechanism module includes a channel attention module and a spatial attention module.

[0087] In detail, the method of using a pre-trained abnormal event recognition model to determine whether there is an abnormal event in the target scene monitoring image includes:

[0088] Extracting an initial feature map of the target scene monitoring image;

[0089] Using a preset attention mechanism module to adjust the weight of the initial feature map to obtain an attention feature map;

[0090] Performing convolution and average pooling operations on the attention feature map to obtain image features of the target scene monitoring image;

[0091] Calculating the probability value of the image feature of the target scene monitoring image belonging to a preset abnormal event using a preset activation function;

[0092] According to the probability value, it is determined whether there is any abnormal event in the target scene monitoring image.

[0093] Specifically, when the probability value is greater than or equal to a preset probability threshold, it is determined that an abnormal event exists in the target scene monitoring image; when the probability value is less than the preset probability threshold, it is determined that no abnormal event exists in the target scene monitoring image.

[0094] Preferably, in one embodiment of the present invention, the kernel size of the convolution layer of the abnormal event recognition model is 7×7, and the step size is 2; the kernel size of the average pooling layer is 3×3, and the step size is 2. This setting can not merge the depth information prematurely, but also can reduce the number of parameters of the network structure and enhance the robustness of the network structure.

[0095] In an embodiment of the present invention, the attention mechanism module is used to adjust and optimize the abnormal event recognition model, thereby improving the feature extraction capability of the abnormal event recognition model and thus improving the accuracy of abnormal event recognition.

[0096] When there is no abnormal event in the target scene monitoring image, return to step S1;

[0097] When an abnormal event exists in the target scene monitoring image, S5, marking the corresponding target scene monitoring image as an abnormal forensic picture, extracting identification information and annotation box position information of the abnormal event from the abnormal forensic picture, and storing the abnormal forensic picture, identification information and annotation box position information corresponding to the abnormal event in a preset cache;

[0098] In one of the embodiments of the present invention, the identification information of the abnormal event is used to identify a certain abnormal event at a preset point of a certain monitoring point, and can be: point number, preset point, abnormal event number, wherein the point number is used to identify the monitoring point; wherein the preset point is used to identify the monitoring area of ​​the monitoring point; and the abnormal event number is used to identify an abnormal event.

[0099] In one of the embodiments of the present invention, the annotation box position information identifies the position information of the annotation box corresponding to the specific abnormal behavior of the abnormal forensic image.

[0100] S6. According to the identification information and the mark box position information, using a preset rule, delete the abnormal evidence pictures, identification information and mark box position information corresponding to the repeated abnormal events in the cache, and obtain the abnormal evidence pictures, identification information and mark box position information corresponding to the retained abnormal events;

[0101] For details, see Figure 4 As shown, the S6 comprises:

[0102] S61, querying the cache according to the identification information, and determining whether there is a target abnormal event identical to the identification information in the cache;

[0103] When there is no target abnormal event with the same identification information in the cache, executing S62, retaining the abnormal evidence picture, identification information and marked box position information corresponding to the abnormal event corresponding to the identification information;

[0104] When the target abnormal event with the same identification information exists in the cache, S63 is executed to determine whether the abnormal event is a duplicate of the target abnormal event based on the annotation box position information and the annotation box position information corresponding to the target abnormal event;

[0105] When the abnormal event is repeated with the target abnormal event, S64 is executed to delete the abnormal evidence picture, identification information and marked box position information corresponding to the repeated abnormal event in the cache;

[0106] When the abnormal event does not overlap with the target abnormal event, S65 is executed to retain the abnormal evidence picture, identification information and annotation box position information corresponding to the abnormal event.

[0107] In detail, querying the cache according to the identification information to determine whether there is a target abnormal event identical to the identification information in the cache includes:

[0108] Calculating the similarity between the identification information and the identification information in the cache;

[0109] It is determined whether there is a target abnormal event with the same identification information in the cache according to the similarity.

[0110] Specifically, when the similarity of the identification information is greater than or equal to the preset second similarity threshold, it is determined that there is a target abnormal event identical to the identification information in the cache; when the similarity of the identification information is less than the preset second similarity threshold, there is no target abnormal event identical to the identification information in the cache.

[0111] Further, judging whether the abnormal event is a duplicate of the target abnormal event based on the annotation box position information and the annotation box position information corresponding to the target abnormal event includes:

[0112] Matching the annotation frame position information with the position information of the annotation frame corresponding to the target abnormal event to obtain an overlap rate between the annotation frame position information and the position information of the annotation frame corresponding to the target abnormal event;

[0113] According to the overlap rate, it is determined whether the abnormal event overlaps with the target abnormal event.

[0114] Specifically, when the overlap rate is greater than or equal to the preset overlap rate threshold, it is determined that the abnormal event and the target abnormal event are repeated; when the overlap rate is less than the preset overlap rate threshold, it is determined that the abnormal event and the target abnormal event are not repeated.

[0115] Abnormal events are a continuous process. The embodiment of the present invention deletes the abnormal forensic images, identification information and annotation box position information corresponding to repeated abnormal events in the cache according to the identification information and the annotation box position information, thereby reducing the storage pressure of abnormal events, reducing the number of abnormal events, and improving the efficiency of abnormal behavior analysis.

[0116] S7. The abnormal evidence pictures, identification information and marked box position information corresponding to the retained abnormal events are stored in a preset abnormal library, and the abnormal evidence pictures, identification information and marked box position information corresponding to the abnormal events are sent to a preset handler.

[0117] In the embodiment of the present invention, the management personnel may be an administrator in the XX system of smart XX or other relevant administrators.

[0118] In the embodiment of the present invention, the abnormal evidence pictures, identification information and marked box position information corresponding to the retained abnormal events are stored in a preset abnormal library, which is conducive to subsequent query, verification and analysis of the abnormal events.

[0119] In one implementation of the present invention, after the abnormal evidence picture, identification information and marked box position information corresponding to the abnormal event are sent to the preset handler in S7, when the time difference between the sending time and the current time is greater than the first time threshold, and the manager has not handled it, the embodiment of the present invention can send the abnormal evidence picture, identification information and marked box position information corresponding to the abnormal event to the manager again, prompting the manager to handle it and preventing the manager from missing the information. Furthermore, when the time difference between the sending time and the current time is greater than the second time threshold, and the manager still has not handled it, the embodiment of the present invention can send the abnormal evidence picture, identification information and marked box position information corresponding to the abnormal event to other managers to ensure that at least one manager receives and handles it, which is conducive to taking corresponding measures against abnormal behavior.

[0120] like Figure 5 , which is a functional module diagram of a regional abnormal behavior monitoring device provided by an embodiment of the present invention.

[0121] The regional abnormal behavior monitoring device 100 of the present invention can be installed in an electronic device. According to the functions to be implemented, the regional abnormal behavior monitoring device 100 can include a monitoring image acquisition module 101, an invalid scene deletion module 102, an abnormal event judgment module 103, an abnormal event deduplication module 104 and an abnormal event processing module 105. The module of the present invention can also be called a unit, which refers to a series of computer program segments that can be executed by an electronic device processor and can complete fixed functions, which are stored in the memory of the electronic device.

[0122] In this embodiment, the functions of each module / unit are as follows:

[0123] The monitoring image acquisition module 101 is used to acquire a real-time monitoring video stream and deframe the video stream to obtain a monitoring image;

[0124] The invalid scene deletion module 102 is used to classify the monitoring images according to preset scenes to obtain scene monitoring images; identify invalid scene images from the scene monitoring images, delete the invalid scene images, and obtain target scene monitoring images;

[0125] The abnormal event judgment module 103 is used to use the pre-trained abnormal event recognition model to judge that there is an abnormal event in the target scene monitoring image, mark the corresponding target scene monitoring image as an abnormal forensic picture, extract the identification information and the marked box position information of the abnormal event from the abnormal forensic picture, and store the abnormal forensic picture, identification information and marked box position information corresponding to the abnormal event into a preset cache;

[0126] The abnormal event deduplication module 104 is used to delete the abnormal evidence pictures, identification information and annotation box position information corresponding to the repeated abnormal events in the cache according to the identification information and the annotation box position information, using a preset rule, and obtain the abnormal evidence pictures, identification information and annotation box position information corresponding to the retained abnormal events;

[0127] The abnormal event processing module 105 is used to store the abnormal evidence pictures, identification information and marked box position information corresponding to the retained abnormal events into a preset abnormal library, and send the abnormal evidence pictures, identification information and marked box position information corresponding to the abnormal events to a preset handler.

[0128] In detail, each module described in the regional abnormal behavior monitoring device 100 in the embodiment of the present invention is used in the same manner as described above. Figures 1 to 4 The technical means are the same as the regional abnormal behavior monitoring method described in, and can produce the same technical effects, so they will not be repeated here.

[0129] like Figure 6 FIG. 1 is a schematic diagram of the structure of an electronic device for implementing a method for monitoring abnormal regional behavior according to an embodiment of the present invention.

[0130] The electronic device 1 may include a processor 10, a memory 11, a communication bus 12, and a communication interface 13, and may also include a computer program stored in the memory 11 and executable on the processor 10, such as a regional abnormal behavior monitoring program.

[0131] In some embodiments, the processor 10 may be composed of an integrated circuit, for example, a single packaged integrated circuit, or a plurality of packaged integrated circuits with the same or different functions, including one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and combinations of various control chips. The processor 10 is the control core (ControlUnit) of the electronic device, and uses various interfaces and lines to connect various components of the entire electronic device, and executes or executes programs or modules stored in the memory 11 (for example, executing a regional abnormal behavior monitoring program, etc.), and calls data stored in the memory 11 to execute various functions of the electronic device and process data.

[0132] The memory 11 includes at least one type of readable storage medium, and the readable storage medium includes flash memory, mobile hard disk, multimedia card, card-type memory (for example: SD or DX memory, etc.), magnetic memory, disk, optical disk, etc. The memory 11 may be an internal storage unit of an electronic device in some embodiments, such as a mobile hard disk of the electronic device. The memory 11 may also be an external storage device of an electronic device in other embodiments, such as a plug-in mobile hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the electronic device. Further, the memory 11 may also include both an internal storage unit of the electronic device and an external storage device. The memory 11 may not only be used to store application software and various types of data installed in the electronic device, such as the code of the regional abnormal behavior monitoring program, but also be used to temporarily store data that has been output or is to be output.

[0133] The communication bus 12 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. The bus is configured to realize connection and communication between the memory 11 and at least one processor 10, etc.

[0134] The communication interface 13 is used for communication between the above-mentioned electronic device and other devices, including a network interface and a user interface. Optionally, the network interface may include a wired interface and / or a wireless interface (such as a WI-FI interface, a Bluetooth interface, etc.), which is generally used to establish a communication connection between the electronic device and other electronic devices. The user interface may be a display (Display), an input unit (such as a keyboard (Keyboard)), and optionally, the user interface may also be a standard wired interface, a wireless interface. Optionally, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, and an OLED (Organic Light-Emitting Diode, organic light-emitting diode) touch device, etc. Among them, the display may also be appropriately referred to as a display screen or a display unit, which is used to display information processed in the electronic device and to display a visual user interface.

[0135] Figure 6 Only an electronic device with components is shown, and those skilled in the art will understand that Figure 6The structure shown does not constitute a limitation on the electronic device 1, and may include fewer or more components than shown in the figure, or combine certain components, or arrange the components differently.

[0136] For example, although not shown, the electronic device may also include a power source (such as a battery) for supplying power to each component. Preferably, the power source may be logically connected to the at least one processor 10 through a power management device, so that the power management device can realize functions such as charging management, discharging management, and power consumption management. The power source may also include one or more DC or AC power sources, recharging devices, power failure detection circuits, power converters or inverters, power status indicators, and other arbitrary components. The electronic device may also include a variety of sensors, Bluetooth modules, Wi-Fi modules, etc., which will not be repeated here.

[0137] It should be understood that the embodiment is for illustration only and the scope of the patent application is not limited to this structure.

[0138] The regional abnormal behavior monitoring program stored in the memory 11 of the electronic device 1 is a combination of multiple instructions. When running in the processor 10, it can achieve:

[0139] Obtaining a real-time surveillance video stream and de-framing the video stream to obtain a surveillance image;

[0140] Classifying the monitoring image according to a preset scene to obtain a scene monitoring image;

[0141] Identifying invalid scene images from the scene monitoring images, and deleting the invalid scene images to obtain a target scene monitoring image;

[0142] When it is determined that an abnormal event exists in the target scene surveillance image using a pre-trained abnormal event recognition model, the corresponding target scene surveillance image is marked as an abnormal forensic picture, identification information and annotation box position information of the abnormal event are extracted from the abnormal forensic picture, and the abnormal forensic picture, identification information and annotation box position information corresponding to the abnormal event are stored in a preset cache;

[0143] According to the identification information and the mark box position information, using a preset rule, the abnormal evidence pictures, identification information and mark box position information corresponding to the repeated abnormal events in the cache are deleted to obtain the abnormal evidence pictures, identification information and mark box position information corresponding to the retained abnormal events;

[0144] The abnormal evidence pictures, identification information and marked box position information corresponding to the retained abnormal events are stored in a preset abnormal library, and the abnormal evidence pictures, identification information and marked box position information corresponding to the abnormal events are sent to a preset handler.

[0145] Specifically, the specific implementation method of the processor 10 for the above instructions can refer to the description of the relevant steps in the corresponding embodiment of the accompanying drawings, which will not be repeated here.

[0146] Furthermore, if the module / unit integrated in the electronic device 1 is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. The computer-readable storage medium can be volatile or non-volatile. For example, the computer-readable medium can include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a mobile hard disk, a magnetic disk, an optical disk, a computer memory, and a read-only memory (ROM).

[0147] The present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor of an electronic device, the computer program can implement:

[0148] Obtaining a real-time surveillance video stream and de-framing the video stream to obtain a surveillance image;

[0149] Classifying the monitoring image according to a preset scene to obtain a scene monitoring image;

[0150] Identifying invalid scene images from the scene monitoring images, and deleting the invalid scene images to obtain a target scene monitoring image;

[0151] When it is determined that an abnormal event exists in the target scene surveillance image using a pre-trained abnormal event recognition model, the corresponding target scene surveillance image is marked as an abnormal forensic picture, identification information and annotation box position information of the abnormal event are extracted from the abnormal forensic picture, and the abnormal forensic picture, identification information and annotation box position information corresponding to the abnormal event are stored in a preset cache;

[0152] According to the identification information and the mark box position information, using a preset rule, the abnormal evidence pictures, identification information and mark box position information corresponding to the repeated abnormal events in the cache are deleted to obtain the abnormal evidence pictures, identification information and mark box position information corresponding to the retained abnormal events;

[0153] The abnormal evidence pictures, identification information and marked box position information corresponding to the retained abnormal events are stored in a preset abnormal library, and the abnormal evidence pictures, identification information and marked box position information corresponding to the abnormal events are sent to a preset handler.

[0154] In the several embodiments provided by the present invention, it should be understood that the disclosed devices, apparatuses and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of the modules is only a logical function division, and there may be other division methods in actual implementation.

[0155] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0156] In addition, each functional module in each embodiment of the present invention may be integrated into one processing unit, each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of hardware plus software functional modules.

[0157] It is obvious to those skilled in the art that the present invention is not limited to the details of the above exemplary embodiments, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.

[0158] Therefore, no matter from which point of view, the embodiments should be regarded as illustrative and non-restrictive, and the scope of the present invention is limited by the appended claims rather than the above description, so it is intended that all changes falling within the meaning and scope of the equivalent elements of the claims are included in the present invention. Any attached figure mark in the claims should not be regarded as limiting the claims involved.

[0159] The blockchain referred to in this invention is a new application mode of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanism, encryption algorithm, etc. Blockchain is essentially a decentralized database, a string of data blocks generated by cryptographic methods. Each data block contains a batch of network transaction information, which is used to verify the validity of its information (anti-counterfeiting) and generate the next block. Blockchain can include the underlying blockchain platform, platform product service layer, and application service layer.

[0160] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Among them, artificial intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0161] In addition, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices stated in the system claim can also be implemented by one unit or device through software or hardware. The words first, second, etc. are used to indicate names, and do not indicate any particular order.

[0162] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the spirit and scope of the technical solution of the present invention.

Claims

1. A method for monitoring regional abnormal behavior, characterized in that: The method comprises: Obtaining a real-time surveillance video stream and de-framing the video stream to obtain a surveillance image; Classifying the monitoring image according to a preset scene to obtain a scene monitoring image; Identifying invalid scene images from the scene monitoring images, and deleting the invalid scene images to obtain a target scene monitoring image; When it is determined that an abnormal event exists in the target scene surveillance image using a pre-trained abnormal event recognition model, the corresponding target scene surveillance image is marked as an abnormal forensic picture, identification information and annotation box position information of the abnormal event are extracted from the abnormal forensic picture, and the abnormal forensic picture, identification information and annotation box position information corresponding to the abnormal event are stored in a preset cache; According to the identification information and the mark box position information, using a preset rule, the abnormal evidence pictures, identification information and mark box position information corresponding to the repeated abnormal events in the cache are deleted to obtain the abnormal evidence pictures, identification information and mark box position information corresponding to the retained abnormal events; The abnormal evidence pictures, identification information and marked box position information corresponding to the retained abnormal events are stored in a preset abnormal library, and the abnormal evidence pictures, identification information and marked box position information corresponding to the abnormal events are sent to a preset handler.

2. The method for monitoring regional abnormal behavior according to claim 1, characterized in that: The monitoring image is classified according to the preset scene to obtain a scene monitoring image; Performing grayscale and normalization processing on the monitoring image to obtain a standardized monitoring image; Performing convolution operations, maximum pooling, and full connection operations on the standardized monitoring image at multiple scales to extract features of the standardized monitoring image; The features are classified according to the preset scenes through the pre-built classification model to obtain the scene monitoring image.

3. The method for monitoring regional abnormal behavior according to claim 1, characterized in that: The step of identifying an invalid scene image from the scene monitoring image and deleting the invalid scene image to obtain a target scene monitoring image includes: Extracting image features of the scene monitoring image; Using preset invalid scene rules corresponding to different scene types to match the image features to obtain matching similarity; According to the matching similarity, invalid scene images are screened out from the scene monitoring images, and the invalid scene images are deleted to obtain a target scene monitoring image.

4. The method for monitoring regional abnormal behavior according to claim 1, characterized in that: The using of the pre-trained abnormal event recognition model to determine whether an abnormal event exists in the target scene monitoring image includes: Extracting an initial feature map of the target scene monitoring image; Using a preset attention mechanism module to adjust the weight of the initial feature map to obtain an attention feature map; Performing convolution and average pooling operations on the attention feature map to obtain image features of the target scene monitoring image; Calculating the probability value of the image feature of the target scene monitoring image belonging to a preset abnormal event using a preset activation function; According to the probability value, it is determined whether there is any abnormal event in the target scene monitoring image.

5. The method for monitoring regional abnormal behavior according to claim 1, characterized in that: The method of deleting the abnormal evidence pictures, identification information and the annotation box position information corresponding to the repeated abnormal events in the cache according to the identification information and the annotation box position information, and obtaining the abnormal evidence pictures, identification information and the annotation box position information corresponding to the retained abnormal events includes: querying the cache according to the identification information to determine whether there is a target abnormal event identical to the identification information in the cache; When there is no target abnormal event with the same identification information in the cache, the abnormal evidence picture, identification information and marked box position information corresponding to the abnormal event corresponding to the identification information are retained; When there is a target abnormal event with the same identification information in the cache, judging whether the abnormal event is a duplicate of the target abnormal event based on the annotation box position information and the annotation box position information corresponding to the target abnormal event; When the abnormal event is repeated with the target abnormal event, the abnormal evidence picture, identification information and marked box position information corresponding to the repeated abnormal event in the cache are deleted; When the abnormal event does not overlap with the target abnormal event, the abnormal evidence picture, identification information and annotation box position information corresponding to the abnormal event are retained.

6. The method for monitoring regional abnormal behavior as claimed in claim 5, characterized in that: The querying the cache according to the identification information to determine whether there is a target abnormal event identical to the identification information in the cache includes: Calculating the similarity between the identification information and the identification information in the cache; It is determined whether there is a target abnormal event with the same identification information in the cache according to the similarity.

7. The method for monitoring regional abnormal behavior as claimed in claim 5, characterized in that: The determining whether the abnormal event is a duplicate of the target abnormal event based on the annotation box position information and the annotation box position information corresponding to the target abnormal event includes: Matching the annotation frame position information with the position information of the annotation frame corresponding to the target abnormal event to obtain an overlap rate between the annotation frame position information and the position information of the annotation frame corresponding to the target abnormal event; According to the overlap rate, it is determined whether the abnormal event overlaps with the target abnormal event.

8. A regional abnormal behavior monitoring device, characterized in that: The device comprises: A monitoring image acquisition module is used to acquire a real-time monitoring video stream and deframe the video stream to obtain a monitoring image; An invalid scene deletion module is used to classify the monitoring images according to preset scenes to obtain scene monitoring images; identify invalid scene images from the scene monitoring images, delete the invalid scene images, and obtain a target scene monitoring image; An abnormal event judgment module is used to use a pre-trained abnormal event recognition model to judge that there is an abnormal event in the target scene monitoring image, mark the corresponding target scene monitoring image as an abnormal forensic picture, extract the identification information and the marked box position information of the abnormal event from the abnormal forensic picture, and store the abnormal forensic picture, identification information and marked box position information corresponding to the abnormal event in a preset cache; An abnormal event deduplication module is used to delete the abnormal evidence pictures, identification information and annotation box position information corresponding to the repeated abnormal events in the cache according to the identification information and the annotation box position information, using preset rules, and obtain the abnormal evidence pictures, identification information and annotation box position information corresponding to the retained abnormal events; The abnormal event processing module is used to store the abnormal evidence pictures, identification information and marked box position information corresponding to the retained abnormal events into a preset abnormal library, and send the abnormal evidence pictures, identification information and marked box position information corresponding to the abnormal events to a preset handler.

9. An electronic device, characterized in that: The electronic device comprises: at least one processor; and, a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the regional abnormal behavior monitoring method as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method for monitoring regional abnormal behavior as described in any one of claims 1 to 7 is implemented.

Images