Video surveillance warning method, device, equipment and storage medium

Abstract

The present invention relates to the field of artificial intelligence, and discloses a video monitoring warning method, device, equipment and storage medium. The video monitoring warning method includes: reading a video monitoring address from a preset data table; sending a connection request according to the video monitoring address to obtain a response result; performing video acquisition and frame extraction according to the successfully connected video monitoring address to obtain an original video stream and at least one video key frame; identifying at least one video key frame based on a deep learning target detection algorithm to obtain the coordinate point position of abnormal information; using the coordinate point position of the abnormal information to perform image annotation on at least one video key frame to obtain annotated video key frames; transcoding the original video stream corresponding to the annotated video key frame according to a preset resolution, a preset video bit rate and a preset video transcoding format and sending a warning message. The present invention expands the number of video monitoring by dynamically configuring the video monitoring address and reduces the cost of video acquisition monitoring.

Description

Technical Field

[0001] The present invention relates to the field of region extraction, and in particular to a video surveillance warning method, device, equipment and storage medium. Background Art

[0002] As my country's economy continues to develop, environmental pollution has received increasing attention, and environmental protection has become a basic policy in my country's modernization drive. One of the difficult problems in the field of environmental protection is monitoring and tracing. By introducing artificial intelligence and collecting and comparing real-time monitoring data on equipment, we can provide solutions to discover suspicious persons, suspicious objects, changes in water quality and water bodies, and causes of air pollution, and avoid missing major abnormal accidents.

[0003] The traditional video surveillance acquisition method is to use expensive video acquisition systems and video acquisition equipment, such as video recorders. This method is highly dependent, complex to install, expensive, and lacks flexibility. At the same time, commercial systems limit the number of videos collected and charge additional fees for video sources that exceed the preset number, resulting in high costs for enterprises to use video acquisition monitoring. Summary of the invention

[0004] The main purpose of the present invention is to solve the technical problems of limited expansion of existing video acquisition quantity and high cost of video acquisition monitoring.

[0005] To achieve the above-mentioned purpose, the first aspect of the present invention provides a video surveillance warning method, comprising: reading a video surveillance address from a preset data table, wherein the connection state corresponding to the video surveillance address is a waiting state; sending a connection request according to the video surveillance address to obtain a response result, wherein the response result is used to indicate whether the video surveillance address is successfully connected; performing video acquisition and frame extraction according to the successfully connected video surveillance address to obtain an original video stream and at least one frame of video key frame; identifying the at least one frame of video key frame based on a deep learning target detection algorithm to obtain a recognition result, wherein the recognition result includes the coordinate point position of abnormal information; using the coordinate point position of the abnormal information to perform image annotation on the at least one frame of video key frame to obtain an annotated video key frame; transcoding the original video stream corresponding to the annotated video key frame according to a preset resolution, a preset video bit rate and a preset video transcoding format and sending a warning message, and sending the annotated video key frame and the corresponding original video stream to a target terminal.

[0006] Optionally, in a first implementation method of the first aspect of the present invention, the connection request is sent according to the video surveillance address to obtain a response result, and the response result is used to indicate whether the video surveillance address is successfully connected, including: sending the connection request according to the video surveillance address, and updating the connection status corresponding to the video surveillance address to a connecting state according to the structured query language SQL syntax rules; receiving a returned response result, and the response result is used to indicate whether the video surveillance address is successfully connected; parsing the response result to obtain a status code; updating the connection status corresponding to the video surveillance address to a connection success state or a connection failure state according to the status code and the SQL syntax rules, and recording the connection time.

[0007] Optionally, in a second implementation of the first aspect of the present invention, the video is captured and framed according to the successfully connected video surveillance address to obtain the original video stream and at least one frame of video key frame, including: reading the successfully connected video surveillance address from the preset data table according to the SQL grammar rules and the successful connection status; video is captured according to the multimedia video processing tool ffmpeg and the successfully connected video surveillance address to obtain the original video stream, wherein the original video stream is a video segment generated by segmentation according to a preset duration; the original video stream is named according to a preset video naming rule and stored in a first directory, wherein the first directory is used to store original monitoring data; the original video stream is framed using the ffmpeg at a preset interval to obtain at least one frame of video key frame, and the at least one frame of video key frame is named according to a preset image naming rule and stored in the first directory.

[0008] Optionally, in a third implementation manner of the first aspect of the present invention, the deep learning-based target detection algorithm identifies the at least one frame of video key frame to obtain a recognition result, and the recognition result includes the coordinate point position of the abnormal information, including: performing multiple convolution and pooling processes on the at least one frame of video key frame through the feature pyramid network FPN algorithm to obtain a first feature map; approximating the feature map corresponding to the at least one frame of video key frame after scaling through the image feature pyramid rule to obtain a second feature map; respectively passing the first feature map and the second feature map through the region proposal network RPN to generate candidate windows; determining the candidate windows according to the preset abnormal target size distribution and summarizing them; processing the summarized candidate windows through a preset classifier to obtain a confidence level; comparing the confidence level with a preset threshold to obtain a recognition result, and the recognition result includes the coordinate point position of the abnormal information.

[0009] Optionally, in a fourth implementation method of the first aspect of the present invention, the image annotation of the at least one frame of video key frame is performed using the coordinate point position of the abnormal information to obtain annotated video key frame, including: reading the at least one frame of video key frame from the first directory; annotating the at least one frame of video key frame according to a preset border color, a preset border line width, and the coordinate point position of the abnormal information to obtain annotated video key frame.

[0010] Optionally, in a fifth implementation method of the first aspect of the present invention, the original video stream corresponding to the marked video key frame is transcoded according to a preset resolution, a preset video bit rate and a preset video transcoding format, a warning message is sent, and the marked video key frame and the corresponding original video stream are sent to a target terminal, including: obtaining a preset resolution, a preset video bit rate and a preset video transcoding format; performing ffmpeg transcoding and compression on the original video stream corresponding to the marked video key frame according to the preset resolution, the preset video bit rate and the preset video transcoding format to obtain a target video stream; storing the marked video key frame and the target video stream in a second directory to obtain multiple storage addresses, and sending a warning message, the second directory being used to store abnormal monitoring data; sending the multiple storage addresses to a target terminal, the target terminal being used to instruct to click and play the target video stream according to the marked video key frame; and regularly cleaning at least one frame of the video key frame and the original video stream in the first directory.

[0011] Optionally, in a sixth implementation of the first aspect of the present invention, after updating the connection status corresponding to the video surveillance address to a successful connection status or a failed connection status according to the status code and the SQL grammar rule, and recording the connection time, the video surveillance warning method also includes: periodically querying the video surveillance address in the connection failure status through a first preset task; and resending the connection request according to the video surveillance address in the connection failure status until a successful connection is made.

[0012] The second aspect of the present invention provides a video surveillance warning device, comprising: a reading unit, used to read a video surveillance address from a preset data table, the connection state corresponding to the video surveillance address is a waiting state; a connection unit, used to send a connection request according to the video surveillance address, and obtain a response result, and the response result is used to indicate whether the video surveillance address is successfully connected; a frame extraction unit, used to perform video acquisition and frame extraction according to the successfully connected video surveillance address, to obtain an original video stream and at least one frame of video key frame; an identification unit, used to identify the at least one frame of video key frame based on a deep learning target detection algorithm, and obtain an identification result, and the identification result includes the coordinate point position of abnormal information; a labeling unit, used to use the coordinate point position of the abnormal information to perform image labeling on the at least one frame of video key frame, and obtain a labeled video key frame; a transcoding warning unit, used to transcode the original video stream corresponding to the labeled video key frame according to a preset resolution, a preset video bit rate and a preset video transcoding format and send a warning message, and send the labeled video key frame and the corresponding original video stream to a target terminal.

[0013] Optionally, in a first implementation method of the second aspect of the present invention, the connection unit also includes: a first connection subunit, used to send the connection request according to the video surveillance address, and update the connection status corresponding to the video surveillance address to a connecting state according to a structured query language SQL syntax rule; a receiving subunit, used to receive a returned response result, wherein the response result is used to indicate whether the video surveillance address is connected successfully; a parsing subunit, used to parse the response result to obtain a status code; an updating subunit, used to update the connection status corresponding to the video surveillance address to a connection success state or a connection failure state according to the status code and the SQL syntax rule, and record the connection time.

[0014] Optionally, in a second implementation method of the second aspect of the present invention, the frame extraction unit is specifically used to: read the successfully connected video surveillance address from the preset data table according to the SQL syntax rules and the successful connection status; perform video capture according to the multimedia video processing tool ffmpeg and the successfully connected video surveillance address to obtain an original video stream, wherein the original video stream is a video segment generated by segmentation according to a preset time length; name the original video stream according to a preset video naming rule and store it in a first directory, wherein the first directory is used to store original monitoring data; use the ffmpeg to extract frames from the original video stream at a preset interval to obtain at least one frame of video key frame, and name the at least one frame of video key frame according to a preset image naming rule and store it in the first directory.

[0015] Optionally, in a third implementation manner of the second aspect of the present invention, the recognition unit is specifically used to: perform multiple convolution and pooling processes on the at least one frame of video key frame through a feature pyramid network FPN algorithm to obtain a first feature map; approximate the feature map corresponding to the at least one frame of video key frame after scaling through an image feature pyramid rule to obtain a second feature map; respectively generate candidate windows for the first feature map and the second feature map through a region proposal network RPN; determine and summarize the candidate proposal windows according to a preset abnormal target size distribution; process the summarized candidate windows through a preset classifier to obtain a confidence level; compare the confidence level with a preset threshold to obtain a recognition result, and the recognition result includes the coordinate points of the abnormal information.

[0016] Optionally, in a fourth implementation method of the second aspect of the present invention, the labeling unit is specifically used to: read the at least one frame of video key frame from the first directory; label the at least one frame of video key frame according to a preset border color, a preset border line width, and the coordinate point position of the abnormal information to obtain a labeled video key frame.

[0017] Optionally, in the fifth implementation method of the second aspect of the present invention, the transcoding warning unit is specifically used to: obtain a preset resolution, a preset video bit rate and a preset video transcoding format; perform ffmpeg transcoding and compression on the original video stream corresponding to the marked video key frames according to the preset resolution, the preset video bit rate and the preset video transcoding format to obtain a target video stream; store the marked video key frames and the target video stream in a second directory to obtain multiple storage addresses, and send a warning message, the second directory is used to store abnormal monitoring data; send the multiple storage addresses to a target terminal, the target terminal is used to instruct to click and play the target video stream according to the marked video key frames; and regularly clean up at least one frame of video key frames and the original video stream in the first directory.

[0018] Optionally, in a sixth implementation method of the second aspect of the present invention, the connection unit further includes: a query subunit, used to periodically query the video surveillance address of the connection failure state through a first preset task; and a second connection subunit, used to resend the connection request according to the video surveillance address of the connection failure state until a successful connection is made.

[0019] The third aspect of the present invention provides a video surveillance warning device, comprising: a memory and at least one processor, wherein instructions are stored in the memory, and the memory and the at least one processor are interconnected via a line; the at least one processor calls the instructions in the memory so that the video surveillance warning device executes the video surveillance warning method described in the first aspect above.

[0020] A fourth aspect of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores instructions, which, when executed on a computer, enable the computer to execute the video surveillance warning method described in the first aspect above.

[0021] In the technical solution provided by the present invention, a video surveillance address is read from a preset data table, and the connection state corresponding to the video surveillance address is a waiting state; a connection request is sent according to the video surveillance address to obtain a response result, and the response result is used to indicate whether the video surveillance address is successfully connected; video acquisition and frame extraction are performed according to the successfully connected video surveillance address to obtain an original video stream and at least one video key frame; the target detection algorithm based on deep learning is used to identify the at least one video key frame to obtain an identification result, and the identification result includes the coordinate point position of abnormal information; the coordinate point position of the abnormal information is used to perform image annotation on the at least one video key frame to obtain an annotated video key frame; the original video stream corresponding to the annotated video key frame is transcoded according to the preset resolution, preset video bit rate and preset video transcoding format, and a warning message is sent, and the annotated video key frame and the corresponding original video stream are sent to the target terminal. In the embodiment of the present invention, the video is acquired and intercepted by directly accessing the video surveillance address, and the video identified as abnormal is marked and warned, which reduces the cost of video acquisition and monitoring, and can be dynamically configured according to the actual video acquisition quantity. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 A schematic diagram of an embodiment of a video monitoring warning method in an embodiment of the present invention;

[0023] Figure 2 A schematic diagram of another embodiment of the video monitoring warning method in an embodiment of the present invention;

[0024] Figure 3 A schematic diagram of an embodiment of a video monitoring warning device in an embodiment of the present invention;

[0025] Figure 4 It is a schematic diagram of another embodiment of the video monitoring warning device in the embodiment of the present invention;

[0026] Figure 5 The figure is a schematic diagram of an embodiment of a video surveillance warning device in an embodiment of the present invention. DETAILED DESCRIPTION

[0027] The embodiments of the present invention provide a video surveillance warning method, apparatus, device and storage medium for collecting and intercepting videos by directly accessing a video surveillance address, marking and warning videos with identified abnormalities, thereby reducing the cost of video acquisition and monitoring, and can be dynamically configured according to the actual number of videos collected.

[0028] In order to enable those skilled in the art to better understand the solutions of the present invention, the embodiments of the present invention will be described below in conjunction with the accompanying drawings in the embodiments of the present invention.

[0029] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "including" or "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0030] For ease of understanding, the specific process of the embodiment of the present invention is described below. Figure 1 , an embodiment of the video surveillance warning method in the embodiment of the present invention includes:

[0031] 101. Read the video surveillance address from the preset data table. The connection state corresponding to the video surveillance address is the waiting state.

[0032] It is understandable that the execution subject of the present invention may be a video monitoring warning device, or a terminal or a server, which is not limited here. The embodiment of the present invention is described by taking a server as the execution subject as an example.

[0033] The server reads the video surveillance address from the preset data table, and the connection state corresponding to the video surveillance address is the waiting state. The video surveillance address is pre-stored in the preset data table, wherein the fields of the preset data table include the monitoring site name, the monitoring channel name, the video surveillance address, the connection state, the connection priority and the connection time, and the connection state includes the waiting state, the connecting state, the connection success state and the connection failure state, and the connection priority is used to indicate the order of the video surveillance address connection.

[0034] It should be noted that the server can dynamically adjust the video surveillance address according to the actual business, and the dynamic adjustment includes adding, modifying and deleting the video surveillance address, so as to expand the number of video surveillance according to actual needs. Furthermore, when the connection status of the video surveillance address is in the waiting state, the server can add, modify and delete the video surveillance address.

[0035] 102. Send a connection request according to the video surveillance address and obtain a response result, where the response result is used to indicate whether the video surveillance address is successfully connected;

[0036] The server sends a connection request according to the video surveillance address and obtains a response result, and the response result is used to indicate whether the video surveillance address is successfully connected. Specifically, the server generates and sends a connection request according to the video surveillance address, and updates the connection state corresponding to the video surveillance address to a connecting state according to the structured query language SQL grammar rules; the server receives the returned response result, and the response result is used to indicate whether the video surveillance address is successfully connected; the server parses the response result to obtain a status code, for example, a status code of 200 indicates a successful connection, and a status code of 500 indicates a failed connection; if the connection is successful, the server modifies the connection state corresponding to the video surveillance address to a successful connection state according to the SQL grammar rules; if the connection fails, the server modifies the connection state corresponding to the video surveillance address to a failed connection state according to the SQL grammar rules.

[0037] 103. Capture video and extract frames according to the successfully connected video surveillance address to obtain an original video stream and at least one video key frame;

[0038] The server performs video acquisition and frame extraction according to the successfully connected video surveillance address to obtain the original video stream and at least one video key frame. Specifically, after the video surveillance address is successfully connected, the server uses the multimedia video processing tool ffmpeg to perform video acquisition according to the successfully connected video surveillance address to obtain the original video stream; the server performs multi-level storage of the original video stream according to the monitoring site name, monitoring channel name, monitoring date and monitoring timestamp. Optionally, the server divides the original video stream into 1 minute video storage duration for storage. Further, the server extracts frames from the stored original video stream to obtain at least one video key frame.

[0039] It should be noted that frame extraction means capturing images of target video stream data of preset duration, and supports multiple image capture ratios, including original resolution capture and high, medium and low resolution capture of target video stream data of preset duration, which is not limited here.

[0040] 104. Identify at least one key frame of the video using a deep learning-based target detection algorithm to obtain a recognition result, where the recognition result includes a coordinate point position of the abnormal information;

[0041] The server identifies at least one video key frame based on a deep learning target detection algorithm to obtain a recognition result, which includes the coordinate point position of the abnormal information. For the original video stream, it is impossible to directly determine whether an abnormality occurs. By identifying at least one video key frame through a preset target detection algorithm, the intrusion target appearing in the original video stream can be identified. For example, the coordinate point position of the abnormal information includes (100, 200) and (400, 600), where (100, 200) is the pixel coordinate of the upper left corner of the frame to be annotated, and (400, 600) is the pixel coordinate of the lower right corner of the frame to be annotated. The server annotates the video key frame image according to (100, 200) and (400, 600).

[0042] It can be understood that the target detection algorithms based on deep learning include the feature pyramid network FPN algorithm and the single-shot multi-box detector SSD algorithm. Suspicious persons, suspicious objects, changes in water quality and air pollution are set as training samples, and then detected and identified after being trained by the corresponding algorithms.

[0043] 105. Use the coordinate point positions of the abnormal information to perform image annotation on at least one frame of video key frame to obtain annotated video key frame;

[0044] The server uses the coordinate point position of the abnormal information to annotate at least one frame of video key frame to obtain annotated video key frame. When a picture is detected to indicate an abnormality, such as a suspicious person breaking into the monitoring area, the server obtains the coordinate point position of the abnormal information from the recognition result and annotates at least one frame of video key frame, wherein the appearance of a suspicious person in the monitoring area is abnormal, and vice versa, the absence of a suspicious person in the monitoring area is normal, and the point coordinates are the pixel values ​​of at least one frame of video key frame. Further, the server determines that the original video stream corresponding to at least one frame of video key frame is an abnormal video, and checks the abnormal behavior by playing the abnormal video.

[0045] 106. Transcode the original video stream corresponding to the marked video key frame according to the preset resolution, preset video bit rate and preset video transcoding format and send a warning message, and send the marked video key frame and the corresponding original video stream to the target terminal.

[0046] The server transcodes the original video stream corresponding to the marked video key frame according to the preset resolution, preset bit rate and preset transcoding format, sends a warning message, and sends the marked video key frame and the corresponding original video stream to the target terminal. Furthermore, the marked key frame and video data are transferred to the abnormal video storage area, and the video is transcoded to a bit rate size that can be directly played.

[0047] It is understandable that the server sets the warning information according to the preset mobile phone number or email address and the preset abnormal target type, and sends the warning information via SMS or email to promptly notify the target person for processing.

[0048] In the embodiment of the present invention, the video is collected and intercepted by directly accessing the video monitoring address, and the video with abnormality is marked and warned, which reduces the cost of video collection and monitoring, and can be dynamically configured according to the actual number of videos collected.

[0049] See also Figure 2 Another embodiment of the video surveillance warning method in the embodiment of the present invention includes:

[0050] 201. Read a video surveillance address from a preset data table, and the connection state corresponding to the video surveillance address is a waiting state;

[0051] The server reads the video surveillance address from the preset data table, and the connection state corresponding to the video surveillance address is the waiting state. The video surveillance address is pre-stored in the preset data table, wherein the connection priority in the field of the preset data table is used to indicate the order of the video surveillance address connection.

[0052] Specifically, the server generates a query statement according to the connection status, connection priority, preset data table and structured query language SQL syntax rules, wherein the connection status is a pending connection status and the connection priority is in a descending order, wherein the server assigns different digital labels to the connection priority. Furthermore, the server sets a smaller number with a higher priority. For example, the server sets the connection priority to include 0 and 1. According to the order of the numbers from small to large, it can be seen that the connection priority of the 0 mark is higher than the connection priority of the 1 mark. The server preferentially accesses the video surveillance address with the highest connection priority in the preset data table; the server executes the query statement to obtain the video surveillance address.

[0053] It should be noted that the server can dynamically adjust the video surveillance address according to actual business. The dynamic adjustment includes adding operations, modifying operations and deleting operations. When the connection status of the video surveillance address is in the waiting state, the server can add operations, modify operations and delete operations on the video surveillance address.

[0054] 202. Send a connection request according to the video surveillance address and obtain a response result, where the response result is used to indicate whether the video surveillance address is successfully connected;

[0055] The server sends a connection request according to the video surveillance address and obtains a response result, and the response result is used to indicate whether the video surveillance address is successfully connected. Specifically, the server sends a connection request according to the video surveillance address, and updates the connection status corresponding to the video surveillance address to the connecting state according to the structured query language SQL syntax rules; the server receives the returned response result, and the response result is used to indicate whether the video surveillance address is successfully connected; the server parses the response result to obtain a status code; the server updates the connection status corresponding to the video surveillance address to a successful connection state or a failed connection state according to the status code and SQL syntax rules, and records the connection time. For example, when the status code is 200, the server updates the connection status corresponding to the video surveillance address to a successful connection state; when the status code is 500, the server updates the connection status corresponding to the video surveillance address to a failed connection state. The server can use 2 to indicate a successful connection state and 3 to indicate a failed connection state, which is not limited here.

[0056] Optionally, the server periodically queries the video surveillance address in the connection failure state through the first preset task; the server reconnects the video surveillance address in the connection failure state until the connection is successfully made.

[0057] Furthermore, the server obtains the process number corresponding to the connection request, and records the process number and the video surveillance address whose connection request has been accepted into the preset file; the server reads the process number from the preset file regularly through the second preset task, and determines whether the process exists according to the process number; if the process does not exist, the server reconnects according to the video surveillance address corresponding to the process number until the connection is successfully made. For example, the server determines whether the connection process of all videos exists every 30 seconds. When the video connection fails, the corresponding process will be disconnected. If the video process does not exist, the video connection task will be restarted.

[0058] 203. Perform video acquisition according to the multimedia video processing tool ffmpeg and the successfully connected video monitoring address to obtain an original video stream, where the original video stream is a video segment generated by segmentation according to a preset duration;

[0059] The server performs video acquisition based on the multimedia video processing tool ffmpeg and the successfully connected video surveillance address to obtain the original video stream. The original video stream is a video segment generated by segmentation according to the preset duration. Further, the server reads the successfully connected video surveillance address from the preset data table according to the SQL syntax rules and the successful connection status; the server concurrently executes the process of obtaining the original video stream based on the successfully connected video surveillance address. The concurrent process is to execute the preset script in the background in a loop, as shown below:

[0060] nohup / data / shell / getVideo.sh${video[0]}${video[1]}${video[2]}>> / data / shell / logs / getVideo_${video[0]}_${video[1]}.out 2>&1&

[0061] Among them, the preset script uses ffmpeg to perform video acquisition and encoding and decoding according to the successfully connected video surveillance address to obtain the original video stream. For example, the video acquisition format is H.264, and H.264 encoding and decoding is implemented through ffmpeg. Furthermore, the server can pre-set the video storage duration, that is, the preset duration, which includes 1 minute, and 1 minute is the shortest video segment, so as to accurately locate the abnormal video segment.

[0062] 204. Name the original video stream according to a preset video naming rule and store it in a first directory, where the first directory is used to store original monitoring data;

[0063] The server names the original video stream according to the preset video naming rules and stores it in the first directory. The first directory is used to store the original monitoring data. Furthermore, the server stores the original video stream in multiple levels according to the monitoring site name, monitoring channel name, monitoring date and monitoring timestamp. The preset video naming rules include monitoring timestamp and video file format. The first directory is mainly composed of the original video, monitoring site name, monitoring channel name, and monitoring date, for example, original video / monitoring site name / monitoring channel name / monitoring date / video_monitoring timestamp.mp4.

[0064] 205. Using ffmpeg to extract frames from the original video stream at a preset interval to obtain at least one video key frame, and naming the at least one video key frame according to a preset image naming rule and storing it in a first directory;

[0065] The server uses ffmpeg to extract frames from the original video stream at preset intervals to obtain at least one video key frame, and names and stores at least one video key frame in the first directory according to the preset image naming rules, wherein the preset interval is used to indicate that the original video stream is intercepted at equal intervals within the preset duration, for example, at least one video key frame is intercepted from the original video stream every 10 seconds. Among them, extracting frames means intercepting images of the target video stream data of the preset duration, supporting multiple image interception ratios, including original resolution screenshots and high, medium and low resolution interceptions of the target video stream data of the preset duration, which are not specifically limited here. Then, the server names and stores at least one video key frame in the first directory according to the preset image naming rules, and the preset image naming rules include monitoring timestamps and image file formats, for example, original video / monitoring site name / monitoring channel name / monitoring date / video_monitoring timestamp.jpg.

[0066] 206. Identify at least one key frame of the video using a deep learning-based target detection algorithm to obtain a recognition result, where the recognition result includes a coordinate point position of the abnormal information;

[0067] The server recognizes at least one key frame of the video based on the deep learning target detection algorithm to obtain the recognition result, which includes the coordinate point location of the abnormal information. For the original video stream, it is impossible to determine whether an abnormality occurs. By recognizing the key frame through the preset target detection algorithm, the intruder target appearing in the original video stream can be identified.

[0068] Specifically, the server performs multiple convolution and pooling processing on at least one key frame of the video through the feature pyramid network FPN algorithm to obtain the first feature map; the server approximates the feature map corresponding to at least one key frame of the video after scaling through the image feature pyramid rule to obtain the second feature map; the server generates candidate windows by passing the first feature map and the second feature map through the region proposal network RPN respectively; the server determines and summarizes the candidate windows according to the preset abnormal target size distribution, and the preset abnormal target size distribution includes the preset sizes of suspicious persons, suspicious objects, water quality and water body changes, and air pollution; the server processes the summarized candidate windows through the preset classifier to obtain the confidence. Among them, the confidence is also called reliability, or confidence level, confidence coefficient, using the interval estimation algorithm in mathematical statistics. When the estimated value and the overall parameter are within a certain allowable error range, how great is the corresponding probability, and this corresponding probability is called the confidence. For example, in 100 75% confidence intervals of 100 sampling results, the probability value of 75% is called the confidence level; the server compares the confidence level with the preset threshold to obtain the recognition result, which includes the coordinate point position of the abnormal information.

[0069] It can be understood that the target detection algorithms based on deep learning include the feature pyramid network FPN algorithm and the single-shot multi-box detector SSD algorithm. Suspicious persons, suspicious objects, changes in water quality and air pollution are set as training samples, and then detected and identified after being trained by the corresponding algorithms.

[0070] 207. Use the coordinate point positions of the abnormal information to perform image annotation on at least one video key frame to obtain annotated video key frames;

[0071] The server annotates at least one frame of video keyframe according to the coordinate point position of the abnormal information to obtain the annotated video keyframe. When a picture is detected to indicate an abnormality, for example, a suspicious person breaks into the monitoring area, the coordinate point position of the abnormal information is obtained from the recognition result, and at least one frame of video keyframe is annotated, wherein the appearance of a suspicious person in the monitoring area is abnormal, and conversely, the absence of a suspicious person in the monitoring area is normal, and the point coordinates are the pixel values ​​of at least one frame of video keyframe. Further, the server determines that the original video stream corresponding to at least one frame of video keyframe is an abnormal video, and checks the abnormal behavior by playing the abnormal video.

[0072] Specifically, first, the server reads at least one frame of video key frame from the first directory, wherein the at least one frame of video key frame includes height information, width information and channel information of the key frame image. For example, the height information of the key frame image is 1500 pixels, and the width information of the key frame image is 1000 pixels, which is a three-channel image, that is, the key frame image is an RGB image, and each pixel point is represented by 3 values.

[0073] Secondly, the server marks at least one frame of the video key frame according to the preset border color, the preset border line width, and the coordinate point position of the abnormal information to obtain the marked video key frame. For example, the server sets the preset border color to green (0, 255, 0), the server sets the preset border line width to 3, and the coordinate point position of the abnormal information includes (200, 200) and (600, 600), where (200, 200) is the pixel coordinate of the upper left corner of the border to be marked, and (600, 600) is the pixel coordinate of the lower right corner of the border to be marked, and a rectangular frame is obtained. The server marks the key frame image according to the rectangular frame.

[0074] 208. Obtain preset resolution, preset video bit rate, and preset video transcoding format;

[0075] The server obtains preset resolution, preset bit rate and preset transcoding format. The preset resolution is set in advance according to the video format of the target terminal; the preset video bit rate is the number of data bits transmitted per unit time during data transmission, and the unit is kilobits per second kbps; the preset video transcoding format is used to indicate the conversion of the original video stream from one format to another, for example, converting the original video stream in MPEG format to a video in MP4 format.

[0076] 209. Perform ffmpeg transcoding and compression on the original video stream corresponding to the marked video key frame according to the preset resolution, preset video bit rate and preset video transcoding format to obtain a target video stream;

[0077] The server performs ffmpeg transcoding and compression on the original video stream corresponding to the marked video keyframes according to the preset resolution, preset video bit rate and preset video transcoding format to obtain the target video stream. The original video is the bit rate output by the camera. After the server collects the original video stream video_1.mp4 with a video size of about 8M per minute, it is transcoded and compressed by ffmpeg to a video with a video resolution of 640*360 and a video size of 2M. The target video stream obtained after transcoding and compression is the bit rate size for direct playback, which saves traffic and has acceptable clarity.

[0078] 210. Store the marked video key frames and target video stream in a second directory, obtain multiple storage addresses, and send a warning message. The second directory is used to store abnormal monitoring data.

[0079] The server stores the marked video keyframes and target video stream in the second directory and sends a warning message. The second directory is used to store abnormal monitoring data. For example, the second directory is "monitoring site name / monitoring channel name / monitoring date", and the server names the target video stream "video_timestamp.mp4" and the marked video keyframe "video_timestamp.jpg".

[0080] It is understandable that the server sets the warning information according to the preset mobile phone number or email address and the preset abnormal target type, and sends the warning information via SMS or email to promptly notify the target person for processing.

[0081] 211. Send multiple storage addresses to a target terminal, where the target terminal is used to instruct to click and play a target video stream according to the marked video key frames;

[0082] The multiple storage addresses are sent to the target terminal, and the target terminal is used to instruct to click and play the target video stream according to the marked video key frame. For example, the terminal uses playback software to display a thumbnail of the video key frame (including the marked frame), and when the thumbnail is clicked, the target video stream is played.

[0083] It should be noted that all target video streams and marked video key frames are not transmitted directly, but are played and displayed through the media playback component according to the uniform resource locator address.

[0084] 212. Regularly clean up at least one video key frame and the original video stream in the first directory.

[0085] The server regularly cleans up at least one video key frame and original video stream in the first directory, for example, cleans up all original video data monitored one week ago, the original video data includes at least one video key frame and original video stream in the first directory, and backs up the video data in the abnormal storage area in real time and permanently stores it.

[0086] Optionally, statistical analysis is performed regularly on the original video stream and the target video stream of the video monitoring website to obtain report information, and the report information is used to indicate the proportion of abnormal situations obtained according to monitoring.

[0087] In the embodiment of the present invention, the video is collected and intercepted by directly accessing the video monitoring address, and the video with abnormality is marked and warned, which reduces the cost of video collection and monitoring, and can be dynamically configured according to the actual number of videos collected.

[0088] The video monitoring warning method in the embodiment of the present invention is described above. The video monitoring warning device in the embodiment of the present invention is described below. Figure 3 , an embodiment of the video monitoring warning device in the embodiment of the present invention includes:

[0089] A reading unit 301 is used to read a video surveillance address from a preset data table, and the connection state corresponding to the video surveillance address is a waiting state;

[0090] A connection unit 302 is used to send a connection request according to the video surveillance address and obtain a response result, wherein the response result is used to indicate whether the video surveillance address is successfully connected;

[0091] A frame extraction unit 303 is used to collect video and extract frames according to the successfully connected video surveillance address to obtain an original video stream and at least one video key frame;

[0092] The recognition unit 304 is used to recognize at least one key frame of the video based on a deep learning target detection algorithm to obtain a recognition result, where the recognition result includes a coordinate point position of abnormal information;

[0093] The labeling unit 305 is used to label at least one video key frame by using the coordinate point position of the abnormal information to obtain a labeled video key frame;

[0094] The transcoding warning unit 306 is used to transcode the original video stream corresponding to the marked video key frame according to the preset resolution, preset video bit rate and preset video transcoding format and send a warning message, and send the marked video key frame and the corresponding original video stream to the target terminal.

[0095] In the embodiment of the present invention, the video is collected and intercepted by directly accessing the video monitoring address, and the video with abnormality is marked and information is provided, thereby reducing the cost of video collection and monitoring, and can be dynamically configured according to the actual number of videos collected.

[0096] See also Figure 4 Another embodiment of the video surveillance warning device in the embodiment of the present invention includes:

[0097] A reading unit 301 is used to read a video surveillance address from a preset data table, and the connection state corresponding to the video surveillance address is a waiting state;

[0098] A connection unit 302 is used to send a connection request according to the video surveillance address and obtain a response result, wherein the response result is used to indicate whether the video surveillance address is successfully connected;

[0099] A frame extraction unit 303 is used to collect video and extract frames according to the successfully connected video surveillance address to obtain an original video stream and at least one video key frame;

[0100] The recognition unit 304 is used to recognize at least one key frame of the video based on a deep learning target detection algorithm to obtain a recognition result, where the recognition result includes a coordinate point position of abnormal information;

[0101] The labeling unit 305 is used to label at least one video key frame by using the coordinate point position of the abnormal information to obtain a labeled video key frame;

[0102] The transcoding warning unit 306 is used to transcode the original video stream corresponding to the marked video key frame according to the preset resolution, preset video bit rate and preset video transcoding format and send a warning message, and send the marked video key frame and the corresponding original video stream to the target terminal.

[0103] Optionally, the connecting unit 302 may further include:

[0104] The first connection subunit 3021 is used to send a connection request according to the video surveillance address, and update the connection state corresponding to the video surveillance address to a connecting state according to the structured query language SQL grammar rule;

[0105] The receiving subunit 3022 is used to receive a returned response result, where the response result is used to indicate whether the video surveillance address is successfully connected;

[0106] The parsing subunit 3023 is used to parse the response result to obtain a status code;

[0107] The updating subunit 3024 is used to update the connection status corresponding to the video monitoring address to a connection success status or a connection failure status according to the status code and the SQL syntax rule, and record the connection time.

[0108] Optionally, the frame extraction unit 303 may also be specifically configured to:

[0109] Read the successfully connected video surveillance address from the preset data table according to the SQL syntax rules and the connection success status;

[0110] Video is collected using the multimedia video processing tool ffmpeg and the successfully connected video surveillance address to obtain the original video stream, which is the video clip generated by segmentation according to the preset duration;

[0111] The original video stream is named according to a preset video naming rule and stored in a first directory, where the first directory is used to store the original monitoring data;

[0112] ffmpeg is used to extract frames from the original video stream at preset intervals to obtain at least one video key frame, and the at least one video key frame is named according to a preset image naming rule and stored in the first directory.

[0113] Optionally, the identification unit 304 may also be specifically configured to:

[0114] Perform multiple convolution and pooling processes on at least one key frame of the video through a feature pyramid network (FPN) algorithm to obtain a first feature map;

[0115] Approximately calculating a feature map corresponding to at least one video key frame after scaling using an image feature pyramid rule to obtain a second feature map;

[0116] The first feature map and the second feature map are respectively passed through the region proposal network RPN to generate candidate windows;

[0117] Determine and summarize candidate proposal windows according to preset abnormal target size distribution;

[0118] The aggregated candidate windows are processed by a preset classifier to obtain confidence;

[0119] The confidence level is compared with a preset threshold to obtain a recognition result, which includes the coordinate point position of the abnormal information.

[0120] Optionally, the labeling unit 305 may also be specifically used for:

[0121] Read at least one video key frame from the first directory;

[0122] At least one video key frame is marked according to a preset border color, a preset border line width, and a coordinate point position of abnormal information to obtain a marked video key frame.

[0123] Optionally, the transcoding warning unit 306 may also be specifically used for:

[0124] Get preset resolution, preset video bit rate and preset video transcoding format;

[0125] Perform ffmpeg transcoding and compression on the original video stream corresponding to the marked video key frames according to the preset resolution, preset video bit rate and preset video transcoding format to obtain the target video stream;

[0126] The marked video key frames and target video streams are stored in a second directory, a plurality of storage addresses are obtained, and a warning message is sent, the second directory is used to store abnormal monitoring data;

[0127] Sending multiple storage addresses to a target terminal, the target terminal is used to instruct to click and play a target video stream according to the marked video key frames;

[0128] At least one video key frame and the original video stream in the first directory are cleaned up regularly.

[0129] Optionally, the connecting unit 302 may further include:

[0130] The query subunit 3025 is used to query the video surveillance address in the connection failure state regularly through the first preset task;

[0131] The second connection subunit 3026 is used to resend the connection request according to the video surveillance address in the connection failure state until the connection is successfully completed.

[0132] In the embodiment of the present invention, the video is collected and intercepted by directly accessing the video monitoring address, and the video with abnormality is marked and warned, which reduces the cost of video collection and monitoring, and can be dynamically configured according to the actual number of videos collected.

[0133] above Figure 3 and Figure 4The video surveillance warning device in the embodiment of the present invention is described in detail from the perspective of modular functional entities. The video surveillance warning device in the embodiment of the present invention is described in detail from the perspective of hardware processing.

[0134] Figure 5 : is a structural diagram of a video surveillance warning device provided by an embodiment of the present invention. The video surveillance warning device 500 may have relatively large differences due to different configurations or performances, and may include one or more processors (central processing units, CPU) 501 (for example, one or more processors) and a memory 509, and one or more storage media 508 (for example, one or more massive storage devices) storing application programs 507 or data 506. Among them, the memory 509 and the storage medium 508 can be short-term storage or permanent storage. The program stored in the storage medium 508 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations in the video surveillance warning device. Furthermore, the processor 501 can be configured to communicate with the storage medium 508 to execute a series of instruction operations in the storage medium 508 on the video surveillance warning device 500.

[0135] The video surveillance warning device 500 may also include one or more power supplies 502, one or more wired or wireless network interfaces 503, one or more input and output interfaces 504, and / or one or more operating systems 505, such as Windows Server, Mac OS X, Unix, Linux, FreeBSD, etc. It will be appreciated by those skilled in the art that Figure 5 The video surveillance warning device structure shown in the figure does not constitute a limitation on the video surveillance warning device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0136] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0137] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program codes.

[0138] As described above, the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features thereof may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A video surveillance alarm method, characterized in that: include: Reading a video surveillance address from a preset data table, wherein the connection state corresponding to the video surveillance address is a waiting-to-connect state; Send a connection request according to the video surveillance address, and update the connection state corresponding to the video surveillance address to a connecting state according to the structured query language SQL grammar rule; Receive a returned response result, wherein the response result is used to indicate whether the video surveillance address is successfully connected; Parsing the response result to obtain a status code; Update the connection status corresponding to the video surveillance address to a connection success status or a connection failure status according to the status code and the SQL syntax rule, and record the connection time; Read the successfully connected video surveillance address from the preset data table according to the SQL grammar rule and the successful connection status; Video acquisition is performed according to the multimedia video processing tool ffmpeg and the successfully connected video surveillance address to obtain an original video stream, wherein the original video stream is a video segment generated by segmentation according to a preset duration; Naming the original video stream according to a preset video naming rule and storing it in a first directory, where the first directory is used to store original monitoring data; Using the ffmpeg to extract frames from the original video stream at a preset interval to obtain at least one video key frame, and naming the at least one video key frame according to a preset image naming rule and storing it in the first directory; Identify the at least one key frame of the video based on a deep learning target detection algorithm to obtain a recognition result, wherein the recognition result includes a coordinate point position of the abnormal information; Using the coordinate point position of the abnormal information to perform image annotation on the at least one video key frame to obtain annotated video key frame; The original video stream corresponding to the marked video key frame is transcoded according to a preset resolution, a preset video bit rate and a preset video transcoding format and an alarm message is sent, and the marked video key frame and the corresponding original video stream are sent to a target terminal.

2. The video surveillance alarm method according to claim 1, characterized in that: The deep learning-based target detection algorithm recognizes the at least one video key frame to obtain a recognition result, wherein the recognition result includes the coordinate point position of the abnormal information, including: Performing multiple convolution and pooling processes on the at least one video key frame by using a feature pyramid network (FPN) algorithm to obtain a first feature map; Approximately calculating the feature map corresponding to the at least one video key frame after scaling by using an image feature pyramid rule to obtain a second feature map; The first feature map and the second feature map are respectively passed through a region proposal network RPN to generate candidate windows; Determine and summarize candidate windows according to preset abnormal target size distribution; The aggregated candidate windows are processed by a preset classifier to obtain confidence; The confidence level is compared with a preset threshold to obtain a recognition result, which includes the coordinate point position of the abnormal information.

3. The video surveillance alarm method according to claim 1, characterized in that: The step of using the coordinate point position of the abnormal information to perform image annotation on the at least one video key frame to obtain annotated video key frame includes: Reading the at least one video key frame from the first directory; The at least one video key frame is marked according to a preset border color, a preset border line width, and the coordinate point position of the abnormal information to obtain a marked video key frame.

4. The video surveillance alarm method according to claim 1 or 3, characterized in that: The method of transcoding the original video stream corresponding to the marked video key frame according to the preset resolution, the preset video bit rate and the preset video transcoding format and sending the alarm information, and sending the marked video key frame and the corresponding original video stream to the target terminal includes: Get preset resolution, preset video bit rate and preset video transcoding format; Performing ffmpeg transcoding and compression on the original video stream corresponding to the marked video key frame according to the preset resolution, the preset video bit rate and the preset video transcoding format to obtain a target video stream; The marked video key frames and the target video stream are stored in a second directory to obtain a plurality of storage addresses, and an alarm message is sent, wherein the second directory is used to store abnormal monitoring data; Sending the multiple storage addresses to a target terminal, the target terminal being used to instruct to click and play a target video stream according to the marked video key frames; Regularly clean up at least one video key frame in the first directory and the original video stream.

5. The video surveillance alarm method according to claim 1, characterized in that: After updating the connection status corresponding to the video surveillance address to a connection success status or a connection failure status according to the status code and the SQL grammar rule and recording the connection time, the video surveillance alarm method further includes: Regularly query the video surveillance address in the connection failure state through the first preset task; The connection request is resent according to the video surveillance address in the connection failure state until the connection is successfully made.

6. A video monitoring alarm device, characterized in that: The video monitoring alarm device comprises: A reading unit, used for reading a video surveillance address from a preset data table, wherein the connection state corresponding to the video surveillance address is a waiting state; A connection unit, used to send a connection request according to the video surveillance address, and update the connection state corresponding to the video surveillance address to a connecting state according to a structured query language SQL grammar rule; receive a returned response result, the response result is used to indicate whether the video surveillance address is successfully connected; parse the response result to obtain a status code; update the connection state corresponding to the video surveillance address to a connection success state or a connection failure state according to the status code and the SQL grammar rule, and record the connection time; A frame extraction unit is used to read the successfully connected video surveillance address from the preset data table according to the SQL syntax rule and the successful connection status; perform video acquisition according to the multimedia video processing tool ffmpeg and the successfully connected video surveillance address to obtain an original video stream, wherein the original video stream is a video segment generated by segmentation according to a preset duration; name the original video stream according to a preset video naming rule and store it in a first directory, wherein the first directory is used to store original monitoring data; use the ffmpeg to extract frames from the original video stream at a preset interval to obtain at least one video key frame, and name the at least one video key frame according to a preset image naming rule and store it in the first directory; An identification unit, configured to identify the at least one key frame of the video based on a deep learning target detection algorithm to obtain an identification result, wherein the identification result includes a coordinate point position of the abnormal information; A labeling unit, configured to label the at least one video key frame by using the coordinate point positions of the abnormal information to obtain a labeled video key frame; The transcoding alarm unit is used to transcode the original video stream corresponding to the marked video key frame according to the preset resolution, preset video bit rate and preset video transcoding format and send alarm information, and send the marked video key frame and the corresponding original video stream to the target terminal.

7. A video surveillance alarm device, characterized in that: The video surveillance alarm device comprises: a memory and at least one processor, the memory stores instructions, and the memory and the at least one processor are interconnected via a line; The at least one processor calls the instruction in the memory so that the video surveillance alarm device executes the video surveillance alarm method as described in any one of claims 1-5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the video surveillance alarm method as described in any one of claims 1 to 5 is implemented.

Images