Remote intelligent monitoring method and system based on Internet of Things

Abstract

The invention provides a remote intelligent monitoring system and method based on the Internet of Things, and the method comprises a data collection module which is used for collecting a public area monitoring video through a video monitoring camera; the behavior analysis module is used for detecting and recognizing an abnormal behavior type existing in a target in the public area monitoring video through a human body behavior recognition model, acquiring face feature information of the abnormal behavior target, and sending the face feature information to the server; remotely uploading the target abnormal behavior type, the target abnormal behavior monitoring video and the target face feature information to a safety early warning platform; and the safety early warning platform is used for judging a target safety risk level according to the abnormal behavior type and the target abnormal behavior monitoring video, and when the target safety risk level reaches a preset level, acquiring a same-time-period associated monitoring video according to the target face feature information, and generating an early warning prompt. According to the scheme, the monitoring data processing efficiency is effectively improved, real-time early warning is achieved, and the accuracy and reliability of early warning can be guaranteed.

Description

Technical Field

[0001] The present invention belongs to the technical field of video monitoring, and in particular relates to a remote intelligent monitoring method and system based on the Internet of Things. Background Art

[0002] In daily life, surveillance cameras can provide security for public area activities and play an important role in preventing illegal activities. Surveillance cameras can not only collect live images of public areas in a timely manner, but also provide early warnings for some potential security risks.

[0003] At present, for video surveillance in public areas, the collected surveillance videos are usually sent directly to the monitoring center platform, which conducts intelligent analysis of the surveillance images and generates security warnings. However, this method will cause the monitoring center platform to be overloaded due to the large amount of surveillance data processing, which will lead to delayed warning processing. On the other hand, the current intelligent surveillance camera technology is becoming more and more mature, and it can process surveillance data on site and generate security warnings. However, in the recognition of abnormal target behavior, the recognition accuracy of intelligent cameras is not high due to factors such as crowd occlusion and long target distance, resulting in a high false alarm rate. Summary of the invention

[0004] In view of this, an embodiment of the present invention provides a remote intelligent monitoring method and system based on the Internet of Things, which is used to solve the problems of delayed early warning processing of existing monitoring platforms and high false alarm rate of intelligent cameras.

[0005] In a first aspect of an embodiment of the present invention, a remote intelligent monitoring system based on the Internet of Things is provided, comprising: A data acquisition module, used to collect surveillance videos of public areas through video surveillance cameras; A behavior recognition module is used to detect and identify the type of abnormal behavior of the target in the public area surveillance video through a human behavior recognition model, and obtain facial feature information of the abnormal behavior target, and remotely upload the target abnormal behavior type, target abnormal behavior surveillance video and target facial feature information to the security warning platform; The human behavior recognition model is a residual network including an attention module and a time shift module, the attention module is used to extract channel attention features and spatial attention features, and the time shift module is used to fuse the attention features in the time dimension; The security warning platform is used to determine the target security risk level based on the abnormal behavior type and the target abnormal behavior surveillance video. When the target security risk level reaches the preset level, the associated surveillance video of the same period is obtained based on the target facial feature information, and a warning prompt is generated.

[0006] In a second aspect of an embodiment of the present invention, a remote intelligent monitoring method based on the Internet of Things is provided, comprising: Collect surveillance videos of public areas through video surveillance cameras; Through the human behavior recognition model, the abnormal behavior type of the target in the public area surveillance video is detected and identified, and the facial feature information of the abnormal behavior target is obtained, and the target abnormal behavior type, target abnormal behavior surveillance video and target facial feature information are remotely uploaded to the security warning platform; The human behavior recognition model is a residual network including an attention module and a time shift module, the attention module is used to extract channel attention features and spatial attention features, and the time shift module is used to fuse the attention features in the time dimension; The target security risk level is determined based on the abnormal behavior type and the target abnormal behavior surveillance video. When the target security risk level reaches the preset level, the associated surveillance video of the same period is obtained based on the target facial feature information, and an early warning prompt is generated.

[0007] In a third aspect of an embodiment of the present invention, an electronic device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the method described in the second aspect of the embodiment of the present invention when executing the computer program.

[0008] In a fourth aspect of an embodiment of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method provided in the second aspect of the embodiment of the present invention are implemented.

[0009] In the embodiment of the present invention, the abnormal behavior target is identified through the video surveillance camera, and the corresponding surveillance video is sent to the monitoring platform. The security warning platform determines the target security risk level according to the target abnormal behavior and the surveillance video, and issues an early warning, thereby not only reducing the load of the early warning platform, realizing timely processing of monitoring data, avoiding the problem of delayed early warning information, but also ensuring the accuracy of abnormal behavior identification and avoiding false alarms. At the same time, by adopting a residual network including an attention module and a time shift module, the recognition accuracy of the smart camera can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0011] Figure 1 A schematic diagram of a flow chart of a remote intelligent monitoring system based on the Internet of Things provided by one embodiment of the present invention; Figure 2 A schematic diagram of a remote intelligent monitoring method based on the Internet of Things provided by one embodiment of the present invention; Figure 3 A schematic structural diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0012] In order to make the purpose, features and advantages of the present invention more obvious and easy to understand, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the embodiments described below are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0013] It should be understood that the term "including" and other similar expressions in the specification or claims of the present invention and the above-mentioned drawings are intended to cover non-exclusive inclusions, such as a process, method, system, or device including a series of steps or units is not limited to the listed steps or units. In addition, "first" and "second" are used to distinguish different objects, not to describe a specific order.

[0014] See also Figure 1 , a flow chart of a remote intelligent monitoring system based on the Internet of Things provided by an embodiment of the present invention includes: The data acquisition module 110 is used to collect public area surveillance videos through video surveillance cameras; The public area generally refers to a specific access area for the public, such as parks, stations, shopping malls, etc. The surveillance video is composed of a series of video frames. After the surveillance camera collects the surveillance video, it will perform behavior recognition on the target of the video frame through a deep learning model.

[0015] The behavior recognition module 120 is used to detect and identify the abnormal behavior type of the target in the public area surveillance video through the human behavior recognition model, and obtain the facial feature information of the abnormal behavior target, and remotely upload the target abnormal behavior type, target abnormal behavior surveillance video and target facial feature information to the security warning platform; Among them, the human behavior recognition model is a residual network including an attention module and a time shift module. The attention module is used to extract channel attention features and spatial attention features, and the time shift module is used to fuse the attention features in the time dimension.

[0016] The attention module can optimize the intermediate features extracted by the residual block. After the features optimized by the attention module are modeled and fused in the time dimension, the new feature information is input into the fully connected layer. The fully connected layer multiplies the weight matrix with the input vector and adds a bias to output the prediction confidence of different behavior categories. The category with the highest probability is the behavior category predicted by the model.

[0017] In some embodiments, a period of video is captured at regular intervals, and a human behavior recognition model is used to identify and determine whether there is abnormal behavior and the type of abnormal behavior in the captured surveillance video.

[0018] Abnormal behavior types refer to behaviors that may endanger public safety, such as running, beating, stalking, using dangerous equipment, and wandering for a long time. When identifying abnormal behaviors, the human behavior recognition model will make behavioral judgments on identifiable targets in the video screen, and will not detect unidentifiable or obscured targets.

[0019] Among them, monitoring videos of abnormal human behavior are collected, and the abnormal behavior types in the monitoring video frames are annotated to form an abnormal behavior data set; a human behavior recognition model is constructed, and the human behavior recognition model is trained and tested through the abnormal behavior data set.

[0020] Preferably, the time shift module replaces partial channels of the features of the current frame image with partial channels of the features of the two frames before and after the current frame image in the time dimension.

[0021] In the time dimension, partial channels of the image features of the two frames before and after a certain frame can be used to replace partial channels of the image features of the current frame. For example, the time shift module can replace 1 / 4 of the channels of the current frame with 1 / 8 of the channels of the two frames before and after in the time dimension T.

[0022] The security warning platform 130 is used to determine the target security risk level based on the abnormal behavior type and the target abnormal behavior monitoring video. When the target security risk level reaches a preset level, the associated monitoring video of the same period is obtained based on the target facial feature information, and a warning prompt is generated.

[0023] Optionally, based on the target abnormal behavior type, other target behaviors in the target surveillance video frame are detected, the current target abnormal behavior type is verified based on other target behaviors, and the target security risk level is calibrated.

[0024] After obtaining the abnormal behavior of the target, the security warning platform will detect the abnormal behavior of other targets in the video frame to verify the abnormal behavior type of the target and determine the target security risk level. When detecting other targets, the security warning platform can use image enhancement and other technologies to improve the clarity of the video frame so as to detect and identify the behavior of other targets. The security warning platform can use a separately trained target detection model to detect the behavioral characteristics of other targets, such as detecting the moving direction, moving speed and target posture of other targets. The current target behavior can be verified based on the behavior detection results of other targets. For example, the pursuit behavior of the current target can be verified based on the movement behavior of other targets.

[0025] Among them, the video source of the target abnormal behavior surveillance video is obtained, and the video surveillance camera of the video source is located; the surveillance videos collected by all video surveillance cameras within the preset distance range of the video surveillance camera at the same time are obtained, and the related surveillance videos are screened out through matching of the target facial feature information.

[0026] Get the location of the camera that detects abnormal behavior. Based on the camera location, you can get the surveillance video of the surrounding cameras. Through face matching, filter out all the surveillance images of the detected target, which is convenient for real-time tracking of the surveillance target.

[0027] In this embodiment, after making a preliminary abnormal behavior judgment on the monitored target in the monitoring camera section, the abnormal behavior is further judged and verified on the early warning platform, which can not only reduce the data processing volume of the early warning platform, reduce the platform load, and ensure that the platform can issue real-time warnings, but also improve the accuracy of abnormal behavior judgment. With the help of the higher computing power of the early warning platform, the reliability of abnormal behavior recognition can be further guaranteed. At the same time, the use of a residual network including an attention module and a time shift module can effectively improve the accuracy of abnormal recognition.

[0028] It should be understood that the serial numbers of the steps in the above embodiments do not imply a sequence of execution, and the execution sequence of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0029] Figure 2 A schematic diagram of a remote intelligent monitoring method based on the Internet of Things provided by an embodiment of the present invention, the method comprising: S201, collecting surveillance videos of public areas through video surveillance cameras; S202, detecting and identifying the abnormal behavior type of the target in the public area surveillance video through a human behavior recognition model, and obtaining facial feature information of the abnormal behavior target, and remotely uploading the target abnormal behavior type, target abnormal behavior surveillance video, and target facial feature information to a security warning platform; The human behavior recognition model is a residual network including an attention module and a time shift module, the attention module is used to extract channel attention features and spatial attention features, and the time shift module is used to fuse the attention features in the time dimension; Among them, monitoring videos of abnormal human behavior are collected, and the abnormal behavior types in the monitoring video frames are annotated to form an abnormal behavior data set; a human behavior recognition model is constructed, and the human behavior recognition model is trained and tested through the abnormal behavior data set.

[0030] Preferably, the time shift module replaces partial channels of the features of the current frame image with partial channels of the features of the two frames before and after the current frame image in the time dimension.

[0031] S203. Determine the target security risk level according to the abnormal behavior type and the target abnormal behavior surveillance video. When the target security risk level reaches a preset level, obtain the associated surveillance video of the same period according to the target facial feature information, and generate an early warning prompt.

[0032] Optionally, based on the target abnormal behavior type, other target behaviors in the target surveillance video frame are detected, the current target abnormal behavior type is verified based on other target behaviors, and the target security risk level is calibrated.

[0033] Among them, the video source of the target abnormal behavior surveillance video is obtained, and the video surveillance camera of the video source is located; the surveillance videos collected by all video surveillance cameras within the preset distance range of the video surveillance camera at the same time are obtained, and the related surveillance videos are screened out through matching of the target facial feature information.

[0034] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described systems and modules can refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here.

[0035] Figure 3 1 is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. The electronic device is used for target behavior detection. Figure 3 As shown, the electronic device 3 of this embodiment includes: a memory 310, a processor 320 and a system bus 330, and the memory 310 includes an executable program 3101 stored thereon. Those skilled in the art can understand that Figure 3 The electronic device structure shown in the figure does not constitute a limitation of the electronic device, and may include more or less components than shown in the figure, or combine certain components, or arrange the components differently.

[0036] Combine the following Figure 3 A detailed introduction to the various components of electronic equipment: The memory 310 can be used to store software programs and modules. The processor 320 executes various functional applications and data processing of the electronic device by running the software programs and modules stored in the memory 310. The memory 310 can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area can store data created according to the use of the electronic device (such as cache data), etc. In addition, the memory 310 can include a high-speed random access memory, and can also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0037] The memory 310 includes an executable program 3101 of a network request method, and the executable program 3101 can be divided into one or more modules / units, which are stored in the memory 310 and executed by the processor 320 to detect and identify targets in surveillance videos, etc. The one or more modules / units can be a series of computer program instruction segments that can complete specific functions, and the instruction segments are used to describe the execution process of the executable program 3101 in the electronic device 3. For example, the executable program 3101 can be a data acquisition module, a behavior recognition module, a data upload module, and other functional modules.

[0038] The processor 320 is the control center of the electronic device. It uses various interfaces and lines to connect various parts of the entire electronic device. By running or executing software programs and / or modules stored in the memory 310, and calling data stored in the memory 310, it performs various functions of the electronic device and processes data, thereby monitoring the overall status of the electronic device. Optionally, the processor 320 may include one or more processing units; preferably, the processor 320 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, application programs, etc., and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 320.

[0039] The system bus 330 is used to connect the various functional components inside the computer, and can transmit data information, address information, and control information. Its types can be, for example, PCI bus, ISA bus, CAN bus, etc. The instructions of the processor 320 are transmitted to the memory 310 through the bus, and the memory 310 feeds back data to the processor 320. The system bus 330 is responsible for the data and instruction exchange between the processor 320 and the memory 310. Of course, the system bus 330 can also be connected to other devices, such as network interfaces, display devices, etc.

[0040] In the embodiment of the present invention, the executable program executed by the processor 320 included in the electronic device includes: Collect surveillance videos of public areas through video surveillance cameras; Through the human behavior recognition model, the abnormal behavior type of the target in the public area surveillance video is detected and identified, and the facial feature information of the abnormal behavior target is obtained, and the target abnormal behavior type, target abnormal behavior surveillance video and target facial feature information are remotely uploaded to the security warning platform; The human behavior recognition model is a residual network including an attention module and a time shift module, the attention module is used to extract channel attention features and spatial attention features, and the time shift module is used to fuse the attention features in the time dimension; The target security risk level is determined based on the abnormal behavior type and the target abnormal behavior surveillance video. When the target security risk level reaches the preset level, the associated surveillance video of the same period is obtained based on the target facial feature information, and an early warning prompt is generated.

[0041] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and modules described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0042] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0043] As described above, the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features thereof may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A remote intelligent monitoring system based on the Internet of Things, characterized in that: include: A data acquisition module, used to collect surveillance videos of public areas through video surveillance cameras; A behavior recognition module is used to detect and identify the type of abnormal behavior of the target in the public area surveillance video through a human behavior recognition model, and obtain facial feature information of the abnormal behavior target, and remotely upload the target abnormal behavior type, target abnormal behavior surveillance video and target facial feature information to the security warning platform; The human behavior recognition model is a residual network including an attention module and a time shift module. The attention module is used to extract channel attention features and spatial attention features, and the time shift module is used to fuse the attention features in the time dimension. The security warning platform is used to determine the target security risk level based on the abnormal behavior type and the target abnormal behavior monitoring video. When the target security risk level reaches the preset level, the associated monitoring video of the same period is obtained based on the target facial feature information, and a warning prompt is generated.

2. The system according to claim 1, characterized in that Before detecting and identifying the abnormal behavior type of the target in the public area surveillance video by using the human behavior recognition model, the method further includes: Collect monitoring videos of abnormal human behavior, annotate the abnormal behavior types in the monitoring video frames, and form an abnormal behavior dataset; Build a human behavior recognition model and train and test it using abnormal behavior datasets.

3. The system according to claim 1, characterized in that The time shift module replaces partial channels of the features of the current frame image with partial channels of the features of the two frames before and after the current frame image in the time dimension.

4. The system according to claim 1, characterized in that Determining the target security risk level based on the abnormal behavior type and the target abnormal behavior monitoring video includes: According to the target abnormal behavior type, other target behaviors in the target surveillance video frame are detected, the current target abnormal behavior type is verified based on other target behaviors, and the target security risk level is calibrated.

5. The system according to claim 1, characterized in that The acquiring of the associated surveillance video of the same period according to the target face feature information comprises: Obtain the video source of the target abnormal behavior surveillance video and locate the video surveillance camera of the video source; Obtain surveillance videos collected by all video surveillance cameras in the preset distance range of the video surveillance camera at the same time, and filter out related surveillance videos by matching the target facial feature information.

6. A remote intelligent monitoring method based on the Internet of Things, characterized in that: include: Collect surveillance videos of public areas through video surveillance cameras; Through the human behavior recognition model, the abnormal behavior type of the target in the public area surveillance video is detected and identified, and the facial feature information of the abnormal behavior target is obtained, and the target abnormal behavior type, target abnormal behavior surveillance video and target facial feature information are remotely uploaded to the security warning platform; The human behavior recognition model is a residual network including an attention module and a time shift module. The attention module is used to extract channel attention features and spatial attention features, and the time shift module is used to fuse the attention features in the time dimension. The target security risk level is determined based on the abnormal behavior type and the target abnormal behavior surveillance video. When the target security risk level reaches the preset level, the associated surveillance video of the same period is obtained based on the target facial feature information, and an early warning prompt is generated.

7. The method according to claim 6, characterized in that Before detecting and identifying the abnormal behavior type of the target in the public area surveillance video by using the human behavior recognition model, the method further includes: Collect monitoring videos of abnormal human behavior, annotate the abnormal behavior types in the monitoring video frames, and form an abnormal behavior dataset; Build a human behavior recognition model and train and test it using abnormal behavior datasets.

8. The method according to claim 6, characterized in that The time shift module replaces partial channels of the current frame image feature with partial channels of the two frames of image features before and after any frame of image feature in the time dimension.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of a remote intelligent monitoring method based on the Internet of Things as described in any one of claims 6 to 8 are implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed, the steps of a remote intelligent monitoring method based on the Internet of Things as described in any one of claims 6 to 8 are implemented.

Images