Remote real-time monitoring control system and method based on video monitoring camera

Abstract

The invention relates to the technical field of remote monitoring, in particular to a remote real-time monitoring control system and method based on a video monitoring camera, and the method comprises the steps: obtaining the network connection information of a plurality of monitoring nodes in a current monitoring network, and analyzing the network state change characteristics; according to the current network state change feature analysis result, triggering a corresponding video processing logic, and processing video data collected by all video monitoring cameras of each monitoring node; defining a key frame dynamic adjustment rule, and judging whether each key frame is complete or not; if yes, multiple key frames in the key frame sequence are selected to be subjected to enhancement processing, the transmission priority of the key frame sequence is improved, and the key frame sequence is transmitted to a designated display terminal to be displayed; otherwise, recombining the key frame sequence, and transmitting the recombined key frame sequence to a specified display terminal for displaying; and locally caching the video data associated with the key frame sequence, and transmitting the video data to different display terminals.

Description

Technical Field

[0001] The present invention relates to the field of remote monitoring technology, and in particular to a remote real-time monitoring control system and method based on a video monitoring camera. Background Art

[0002] With the continuous development and popularization of IoT technology, surveillance cameras with IoT functions are becoming more and more common, providing people with more efficient and convenient monitoring and management methods. It collects images and data through built-in sensors and cameras, and transmits these data to the Internet through wireless networks. Freed from the limitations of communication line length, once the data is transmitted to the cloud or local server, users can view real-time images and receive alarm notifications at any time through mobile devices or computers. It is widely used in traffic management, smart city construction, smart home security systems, and chemical production monitoring.

[0003] Surveillance cameras usually achieve network communication functions by connecting to nearby base stations. Existing base stations include 4G and 5G networks, and the communication speed, bandwidth, latency and other parameters of these two networks are different; if video surveillance cameras are connected to the 5G network, data can be transmitted quickly and efficiently.

[0004] However, if the 5G network fails or is overloaded, in order to ensure the normal operation of the monitoring network, it is usually necessary to switch from the 5G network to the 4G network. After changing the connected network, how to ensure the normal operation of the monitoring network and the safe and fast transmission of monitoring data is an urgent problem to be solved. Summary of the invention

[0005] The present invention can monitor the network connection status of the monitoring network, control the transmission of video data according to the connected network, distinguish the video data of key areas and key time periods, and transmit the key video data to a designated display terminal for display.

[0006] The technical solution proposed by the present invention is: a remote real-time monitoring control method based on a video surveillance camera, the method comprising:

[0007] Obtain network connection information of multiple monitoring nodes in the current monitoring network and analyze network status change characteristics; the network connection information includes the network type and signal strength of the current network;

[0008] According to the analysis results of the current network status change characteristics, the corresponding video processing logic is triggered to process the video data collected by all video surveillance cameras of each monitoring node;

[0009] Define the dynamic adjustment rules of key frames, dynamically select multiple key frames from the processed video data, and construct a time length of Tl The key frame sequence is obtained, and each key frame is judged to be complete;

[0010] If yes, multiple key frames in the key frame sequence are selected for enhanced processing, the transmission priority of the key frame sequence is increased, and the key frame sequence is transmitted to a designated display terminal for display;

[0011] Otherwise, reorganize the key frame sequence, select multiple key frames in the reorganized key frame sequence for enhanced processing, increase the transmission priority of the reorganized key frame sequence, and transmit the reorganized key frame sequence to a designated display terminal for display; the reorganized key frame sequence includes reorganization on the time axis and / or reorganization on the integrity of the target object;

[0012] The video data associated with the key frame sequence is cached locally and then transmitted to different display terminals.

[0013] Preferably, the obtaining of network connection information of multiple monitoring nodes in the current monitoring network and analyzing network status change characteristics comprises the following steps:

[0014] Acquire network connection information of multiple video surveillance cameras in a monitoring node within a preset time length according to a collection frequency to form a network connection information set;

[0015] Extracting multiple network status features from the network connection information set to form a network status feature set for each video surveillance camera;

[0016] The network status characteristics include: one or more of the number of network type changes, the maximum interval time of network type changes, the minimum interval time of network type changes, the maximum signal strength and the minimum signal strength; the network types include 5G network and 4G network;

[0017] Use the pre-trained network status monitoring model to analyze the network status characteristics of each camera and predict the changes in the network connection status of each camera in the control node.

[0018] Preferably, the triggering of corresponding video processing logic according to the current network status change feature analysis result to process the video data collected by all video surveillance cameras of each monitoring node includes the following steps:

[0019] If the network status monitoring model predicts that the network connection of the video surveillance camera changes, that is, switches from the 5G network to the 4G network, the video segmentation processing logic is triggered. Specifically:

[0020] The video data transmitted by the video surveillance camera is divided into multiple small segments to form a video time sequence. Each small segment can be transmitted independently, and the automatic retransmission request mechanism is used to ensure the reliable transmission of each small segment.

[0021] Get the network transmission bandwidth of the current network, and determine the number of small segments of video data according to the network transmission bandwidth of the current network; specifically: get the bandwidth of the current network, and determine the number of video segments N according to the bandwidth of the current network i_j =γB i , where N i_j represents the number of segments of the surveillance video of the jth video surveillance camera of the i-th monitoring node, γ represents the segmentation ratio, and B i represents the network bandwidth of the i-th monitoring node;

[0022] If the network status monitoring model predicts that the network connection of the video surveillance camera changes, that is, switches from a 4G network to a 5G network, the video high-quality processing logic is triggered. Specifically:

[0023] Adjust the video bit rate to adapt to the current network environment and improve the transmission efficiency of video data;

[0024] Through the distributed cache system, cache management is provided for large-scale video data to reduce the transmission delay of video data.

[0025] Preferably, before defining the key frame dynamic adjustment rule, the following steps are also included:

[0026] According to the regional attributes of the video surveillance camera performing the monitoring task, the monitoring nodes in the monitoring network are divided into key area nodes and non-key area nodes, and the data transmitted by each video surveillance camera in the monitoring node is divided into key area video data and non-key area video data;

[0027] According to the time attribute of the video surveillance camera performing the monitoring task, the video data transmitted by the monitoring nodes in the monitoring network are divided into video data in the key time period and video data in the non-key time period;

[0028] According to the burst attribute of the video surveillance camera performing the monitoring task, the video data transmitted by the monitoring nodes in the monitoring network are divided into emergency event video data and non-emergency event video data;

[0029] The video data of key area nodes, the video data of key time periods and the video data of emergencies constitute the key frame data pool.

[0030] Preferably, the key frame dynamic adjustment rule is defined, and multiple key frames are dynamically selected from the processed video data to form a time length T l The key frame sequence includes the following steps:

[0031] Determine the regional attribute of the acquired surveillance video data, if it is key area video data, extract key frame data from the acquired surveillance video data, otherwise proceed to the next step;

[0032] Determine the time attribute of the acquired surveillance video data. If it is video data of a key time period, extract key frame data from the acquired surveillance video data, otherwise proceed to the next step;

[0033] The acquired monitoring video data is judged as burst data. If it is burst event video data, key frame data is extracted from the acquired monitoring video data. Otherwise, the monitoring video data is acquired again.

[0034] Preferably, the step of determining whether each key frame is complete comprises the following steps:

[0035] Obtain the video time sequence and extract the video features of each small segment; the video features include the start time and end time of each small segment, the marker features in the small segment, the color histogram, the SIFT feature, and the HOG feature;

[0036] Determine whether the key frame is divided into two small segments by analyzing video features, including:

[0037] Calculating the similarity between two adjacent small fragments, including calculating the Euclidean distance or cosine similarity between the two adjacent small fragments;

[0038] A preset similarity threshold is obtained. If the similarity is greater than the similarity threshold, it is determined that the key frame is not segmented; otherwise, it is determined that the key frame is segmented.

[0039] Preferably, the step of selecting a plurality of key frames in the key frame sequence for enhanced processing, increasing the transmission priority of the key frame sequence, and transmitting the key frame sequence to a designated display terminal for display includes:

[0040] Improve the clarity of each key frame, including sharpening and denoising operations;

[0041] At the network transmission layer, the highest priority is set for key frames to ensure the transmission efficiency of key frames, which is achieved through the Quality of Service (QoS) mechanism.

[0042] Creating a buffer in the server for storing key frames;

[0043] Establish a connection between the specified display terminal and the server through the socket method, including:

[0044] Create TCP / IP sockets on the server and the designated display terminal respectively, and connect the server and the designated display terminal;

[0045] Specify the display terminal to bind a server port and listen for connections;

[0046] The server sends key frame data through the bound server port;

[0047] Specify the display terminal to receive the key frame and display it;

[0048] The reorganizing the key frame sequence and selecting a plurality of key frames in the reorganized key frame sequence for enhancement processing include:

[0049] Reorganize the key frame sequence on the timeline, specifically:

[0050] Get the time interval of the key time period [t i0 , t i1 ], where t i0 ,t i1 Respectively represent the start time and end time of the key time period;

[0051] Traverse all the small fragments and obtain the time interval of each small fragment [T i0 , T i1 ], extract the start time T of each small segment i0 and end time T i1 , i=1, 2, ..., n, where n is the number of small segments; determine whether the start time and end time of the key time period fall within the time interval of the corresponding small segment, and if so, obtain the small segment i and small segment i+m where the start time and end time of the key time period are located;

[0052] The small fragment i and the small fragment i+1 are spliced ​​together to form a new fragment 1, and the small fragment i+m and the small fragment i+m-1 are spliced ​​together to form a new fragment 2.

[0053] Use the start time and end time of the key time period as the segmentation points, and segment the new segment 1 and the new segment 2 to separate the segments that do not contain t i0 and t i1 The small fragment i′ and the small fragment i+m′;

[0054] All time intervals in the key time period [t i0 , t i1 ] constitute the reorganized key frame sequence;

[0055] Reconstruction frame sequence on target integrity, specifically:

[0056] Get all the small clips in the key frame sequence;

[0057] Extracting a plurality of continuous recognition frames for identifying the features of the target object from two continuous small segments respectively;

[0058] Extract feature points of the target object from two consecutive frames;

[0059] Using a feature matching algorithm, the geometric transformation matrix between two recognition frames is estimated through the matching feature points in two consecutive recognition frames;

[0060] Align the two recognition frames according to the geometric transformation matrix;

[0061] The two aligned recognition frames are fused to generate a new recognition frame, wherein the new recognition frame includes complete target object features; then, the new recognition frame is updated to the key frame sequence;

[0062] A sharpening or denoising operation is performed on each small segment in the reorganized focus frame sequence.

[0063] Preferably, the step of caching the video data associated with the key frame sequence locally and then transmitting it to different display terminals comprises the following steps:

[0064] identifying key frame data and video data associated with a sequence of key frames;

[0065] The video data associated with the key frame sequence includes non-key frame data connected to the key frame data on the time axis and video data containing images around the key monitoring area;

[0066] The video data associated with the key frame sequence is sent to different display terminals for display.

[0067] The present invention also provides a remote real-time monitoring control system based on a video monitoring camera, and the system is used to execute the remote real-time monitoring control method based on a video monitoring camera.

[0068] The present invention also provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and the computer program is executed by a processor to implement the remote real-time monitoring and control method based on a video surveillance camera.

[0069] Beneficial effects of the present invention:

[0070] 1. The present invention adopts different data processing strategies to control the video data processing mode and transmission mode when the communication network connected to the video surveillance camera changes, thereby solving the problem that when the 5G network switches to the 4G network, the timeliness of video data transmission is easily affected due to changes in network transmission speed and delay.

[0071] 2. The present invention, when dividing the video data into multiple small segments for transmission, detects whether the key frames in each small segment are complete, thereby ensuring that the monitoring data of the key monitoring area, key monitoring time period and emergency time period can be transmitted to the designated display terminal in a timely and complete manner, making it convenient for relevant personnel to obtain video data. BRIEF DESCRIPTION OF THE DRAWINGS

[0072] Figure 1 The present invention is a flow chart of a remote real-time monitoring control method based on a video surveillance camera. DETAILED DESCRIPTION

[0073] The following description is used to disclose the present invention so that those skilled in the art can implement the present invention. The preferred embodiments described below are only examples, and those skilled in the art can think of other obvious variations. The basic principles of the present invention defined in the following description can be applied to other embodiments, variations, improvements, equivalents, and other technical solutions that do not deviate from the spirit and scope of the present invention.

[0074] It is to be understood that the term "one" should be understood as "at least one" or "one or more", that is, in one embodiment, the number of an element may be one, while in another embodiment, the number of the element may be multiple, and the term "one" should not be understood as a limitation on the quantity.

[0075] refer to Figure 1 The technical solution provided by the present invention is: a remote real-time monitoring control method based on a video surveillance camera, comprising the following steps:

[0076] Step 1: Obtain network connection information of multiple monitoring nodes in the current monitoring network and analyze network status change characteristics; the network connection information includes the network type and signal strength of the current network; specifically includes:

[0077] Acquire network connection information of multiple video surveillance cameras in a monitoring node within a preset time length according to a collection frequency to form a network connection information set;

[0078] Extracting multiple network status features from the network connection information set to form a network status feature set for each video surveillance camera;

[0079] The network status characteristics include: one or more of the number of network type changes, the maximum interval time of network type changes, the minimum interval time of network type changes, the maximum signal strength and the minimum signal strength; the network types include 5G network and 4G network;

[0080] Use the pre-trained network status monitoring model to analyze the network status characteristics of each camera and predict the changes in the network connection status of each camera in the control node.

[0081] The pre-trained network status monitoring model can be constructed based on decision tree, random forest, support vector machine and other algorithms. In this embodiment, the network status monitoring model is constructed by the random forest algorithm, and the probability of network switching is output. If the output network switching probability is greater than the preset switching probability threshold, it is determined that the network is about to switch.

[0082] Step 2: According to the analysis result of the current network status change characteristics, trigger the corresponding video processing logic to process the video data collected by all video surveillance cameras of each monitoring node; including the following steps:

[0083] If the network status monitoring model predicts that the network connection of the video surveillance camera changes, that is, switches from the 5G network to the 4G network, the video segmentation processing logic is triggered. Specifically:

[0084] The video data transmitted by the video surveillance camera is divided into multiple small segments to form a video time sequence. Each small segment can be transmitted independently, and the automatic retransmission request mechanism is used to ensure the reliable transmission of each small segment.

[0085] Get the network transmission bandwidth of the current network, and determine the number of small segments of video data according to the network transmission bandwidth of the current network; specifically: get the bandwidth of the current network, and determine the number of video segments N according to the bandwidth of the current network i_j =γB i , where N i_j represents the number of segments of the surveillance video of the jth video surveillance camera of the i-th monitoring node, γ represents the segmentation ratio, and B i Represents the network bandwidth of the i-th monitoring node.

[0086] If the network status monitoring model predicts that the network connection of the video surveillance camera changes, that is, switches from a 4G network to a 5G network, the video high-quality processing logic is triggered. Specifically:

[0087] Adjust the video bit rate to adapt to the current network environment and improve the transmission efficiency of video data;

[0088] Through the distributed cache system, cache management is provided for large-scale video data to reduce the transmission delay of video data.

[0089] In a monitoring network, multiple monitoring nodes are usually divided, and each node monitors different areas and working hours. Each monitoring node is usually equipped with at least one video surveillance camera. In order to achieve the purpose of remote monitoring, the selected video surveillance camera usually has an Internet of Things communication function, which can ensure long-distance transmission of video data and reception of control instructions. Therefore, the attributes of each monitoring point node are different (differences in working area, working hours, etc.). Due to these attributes, the video data of the corresponding monitoring node also has different attributes.

[0090] Step 3: According to the regional attributes of the video surveillance camera performing the monitoring task, the monitoring nodes in the monitoring network are divided into key area nodes and non-key area nodes, and the data transmitted by each video surveillance camera in the monitoring node is divided into key area video data and non-key area video data;

[0091] According to the time attribute of the video surveillance camera performing the monitoring task, the video data transmitted by the monitoring nodes in the monitoring network are divided into video data in the key time period and video data in the non-key time period;

[0092] According to the burst attribute of the video surveillance camera performing the monitoring task, the video data transmitted by the monitoring nodes in the monitoring network are divided into emergency event video data and non-emergency event video data;

[0093] The video data of key area nodes, the video data of key time periods and the video data of emergencies constitute a key frame data pool. The terminal frame data pool can be set in the cloud or in a local server.

[0094] Define the dynamic adjustment rules of key frames, dynamically select multiple key frames from the processed video data, and construct a time length of T l The key frame sequence specifically includes: determining the regional attribute of the acquired monitoring video data, if it is the key regional video data, extracting the key frame data from the acquired monitoring video data, otherwise proceeding to the next step;

[0095] Determine the time attribute of the acquired surveillance video data. If it is video data of a key time period, extract key frame data from the acquired surveillance video data, otherwise proceed to the next step;

[0096] The acquired monitoring video data is judged as burst data. If it is burst event video data, key frame data is extracted from the acquired monitoring video data. Otherwise, the monitoring video data is acquired again.

[0097] Then it is necessary to determine whether the key frames included in the video data in each small segment are complete. The specific steps are as follows: obtain the video time series and extract the video features of each small segment; the video features include the start time and end time of each small segment, the marker features in the small segment, the color histogram, the SIFT feature, and the HOG feature;

[0098] Determine whether the key frame is divided into two small segments by analyzing video features, including:

[0099] Calculate the similarity between two adjacent small segments, including calculating the Euclidean distance or cosine similarity between the two adjacent small segments; obtain a preset similarity threshold, if the similarity is greater than the similarity threshold, determine that the key frame is not segmented; otherwise, determine that the key frame is segmented.

[0100] Step 4: If the key frames included in the video sequence of each small segment are complete, multiple key frames in the key frame sequence are selected for enhanced processing, the transmission priority of the key frame sequence is increased, and the key frame sequence is transmitted to the designated display terminal for display. The specific process is as follows:

[0101] Improve the clarity of each key frame, including sharpening and denoising operations;

[0102] At the network transmission layer, the highest priority is set for key frames to ensure the transmission efficiency of key frames, which is achieved through the Quality of Service (QoS) mechanism; a buffer area is created in the server to store key frames;

[0103] Establish a connection between the specified display terminal and the server through the socket method, including:

[0104] Create TCP / IP sockets on the server and the designated display terminal respectively, and connect the server and the designated display terminal;

[0105] The designated display terminal is bound to a server port and listens for connections; the server sends key frame data through the bound server port; the key frame is received through the designated display terminal and the received key frame is displayed.

[0106] If the key frames included in the video clips in each small segment are not complete, it is necessary to reorganize the key frame sequence, select multiple key frames in the reorganized key frame sequence for enhanced processing, increase the transmission priority of the reorganized key frame sequence, and transmit the reorganized key frame sequence to the designated display terminal for display.

[0107] The reorganization here includes the reorganization on the timeline and the reorganization on the integrity of the target object;

[0108] The reorganization on the time axis includes: obtaining the time interval of the key time period [t i0 , t i1 ], where t i0 ,t i1 Respectively represent the start time and end time of the key time period;

[0109] Traverse all the small fragments and obtain the time interval of each small fragment [T i0 , T i1 ], extract the start time T of each small segment i0 and end time T i1 , i=1, 2, ..., n, where n is the number of small segments; determine whether the start time and end time of the key time period fall within the time interval of the corresponding small segment, and if so, obtain the small segment i and small segment i+m where the start time and end time of the key time period are located;

[0110] The small fragment i and the small fragment i+1 are spliced ​​together to form a new fragment 1, and the small fragment i+m and the small fragment i+m-1 are spliced ​​together to form a new fragment 2.

[0111] Use the start time and end time of the key time period as the segmentation points, and segment the new segment 1 and the new segment 2 to separate the segments that do not contain t i0 and t i1 The small fragment i′ and the small fragment i+m′;

[0112] All time intervals in the key time period [t i0 , t i1 ] constitute the reorganized key frame sequence;

[0113] A sharpening or denoising operation is performed on each small segment in the reorganized focus frame sequence.

[0114] Among them, the reconstruction of the integrity of the target includes:

[0115] Get all the small clips in the key frame sequence;

[0116] Extracting a plurality of continuous recognition frames for identifying the features of the target object from two continuous small segments respectively;

[0117] Extract feature points of the target object from two consecutive frames; in this embodiment, an ORB feature detector is used to extract key points and descriptors in the image;

[0118] Using the feature matching algorithm, the geometric transformation matrix between the two recognition frames is estimated by the matching feature points in two consecutive recognition frames; in this embodiment, Brute Force Matcher (BFMatcher) is used for feature point matching. BFMatcher finds the best match by calculating the distance between descriptors. Using the matched feature points, the affine transformation matrix between the two frames is estimated by the cv2.estimateAffinePartial2D function. This matrix describes how to transform the second frame image into the coordinate system of the first frame image.

[0119] The two recognition frames are aligned according to the geometric transformation matrix. In this embodiment, the cv2.warpAffine function is used to align the second frame image to the coordinate system of the first frame image according to the estimated transformation matrix.

[0120] The two aligned recognition frames are fused to generate a new recognition frame, which includes complete target features; in this embodiment, the two aligned frames are weighted averaged using the cv2.addWeighted function to generate the final fused image. The above functions are included in Python and OpenCV libraries.

[0121] For example, in a public security monitoring system, if the transmitted video data is the video data of a key monitored area, the monitoring point has two video surveillance cameras, the monitoring area of ​​the first video surveillance camera is the key area, and the monitoring area of ​​the other is the area outside the key area. The video data of the first camera is transmitted to the designated display terminal for display. If a suspicious person (target object) appears in the key monitored area and moves from the key monitored area to the outside of the key area, some video frames will appear with only half of the target object. If these video frames include some key information, it is necessary to call the video data of another camera, splice the target object together, and then display it on the designated display terminal.

[0122] Step 5: After caching the video data associated with the key frame sequence locally, transmit it to different display terminals. The designated display terminal can send a video call instruction to the server as needed to obtain the video data associated with the key frame sequence from the server. The corresponding video data can be transmitted through the bound server port to ensure transmission security. Specifically, the following steps are included:

[0123] identifying key frame data and video data associated with a sequence of key frames;

[0124] The video data associated with the key frame sequence includes non-key frame data connected to the key frame data on the time axis and video data containing images around the key monitoring area;

[0125] The video data associated with the key frame sequence is sent to different display terminals for display.

[0126] The present invention also provides a remote real-time monitoring control system based on a video monitoring camera, and the system is used to execute the remote real-time monitoring control method based on a video monitoring camera.

[0127] The present invention also provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and the computer program is executed by a processor to implement the remote real-time monitoring and control method based on a video surveillance camera.

[0128] The embodiments disclosed in the present invention, the process described above with reference to the flowchart can be implemented as a computer software program. The embodiments disclosed in the present invention include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program contains a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part, and / or installed from a removable medium. When the computer program is executed by the central processing unit (CPU), the above-mentioned functions defined in the method of the present application are executed. It should be noted that the above-mentioned computer-readable medium of the present application can be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium can be, for example, but not limited to, a system, device or device of electricity, magnetism, light, electromagnetic, infrared or semiconductor, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to, an electrical connection with one or more wire segments, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In the present application, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries a computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code embodied on the computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, electrical wire, optical cable, RF, etc., or any suitable combination of the foregoing.

[0129] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present invention. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of a code, and the module, a program segment or a part of a code contains one or more executable instructions for implementing the specified logical functions. It should also be noted that in some alternative implementations, the functions marked in the boxes can also occur in a different order than the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0130] Those skilled in the art should understand that the embodiments of the present invention described above and shown in the accompanying drawings are only examples and do not limit the present invention. The objectives of the present invention have been fully and effectively achieved. The functions and structural principles of the present invention have been demonstrated and explained in the embodiments. Without departing from the principles, the implementation methods of the present invention may be subject to any changes or modifications.

Claims

1. A remote real-time monitoring control method based on a video surveillance camera, characterized in that: The method comprises: Obtain network connection information of multiple monitoring nodes in the current monitoring network and analyze network status change characteristics; the network connection information includes the network type and signal strength of the current network; According to the analysis results of the current network status change characteristics, the corresponding video processing logic is triggered to process the video data collected by all video surveillance cameras of each monitoring node; Define the dynamic adjustment rules of key frames, dynamically select multiple key frames from the processed video data, and construct a time length of T l The key frame sequence of Determine whether each key frame is complete; if so, select multiple key frames in the key frame sequence for enhanced processing, increase the transmission priority of the key frame sequence, and transmit the key frame sequence to a designated display terminal for display; otherwise, reorganize the key frame sequence, select multiple key frames in the reorganized key frame sequence for enhanced processing, increase the transmission priority of the reorganized key frame sequence, and transmit the reorganized key frame sequence to a designated display terminal for display; the reorganized key frame sequence includes reorganization on the time axis and / or reorganization on the integrity of the target object; The video data associated with the key frame sequence is cached locally and then transmitted to different display terminals.

2. A remote real-time monitoring control method based on a video surveillance camera according to claim 1, characterized in that: The method of obtaining network connection information of multiple monitoring nodes in the current monitoring network and analyzing network status change characteristics includes the following steps: Acquire network connection information of multiple video surveillance cameras in a monitoring node within a preset time length according to a collection frequency to form a network connection information set; Extracting multiple network status features from the network connection information set to form a network status feature set for each video surveillance camera; The network status characteristics include: one or more of the number of network type changes, the maximum interval time of network type changes, the minimum interval time of network type changes, the maximum signal strength and the minimum signal strength; the network types include 5G network and 4G network; Use the pre-trained network status monitoring model to analyze the network status characteristics of each camera and predict the changes in the network connection status of each camera in the control node.

3. A remote real-time monitoring control method based on a video surveillance camera according to claim 2, characterized in that: The method triggers the corresponding video processing logic according to the analysis result of the current network status change characteristics, and processes the video data collected by all video surveillance cameras of each monitoring node, including the following steps: If the network status monitoring model predicts that the network connection of the video surveillance camera changes, that is, switches from a 5G network to a 4G network, the video segmentation processing logic is triggered. Specifically: The video data transmitted by the video surveillance camera is divided into multiple small segments to form a video time sequence. Each small segment can be transmitted independently, and the automatic retransmission request mechanism is used to ensure the reliable transmission of each small segment. Get the network transmission bandwidth of the current network, and determine the number of small segments of video data according to the network transmission bandwidth of the current network; specifically: get the bandwidth of the current network, and determine the number of video segments N according to the bandwidth of the current network i_j =γB i , where N i_j represents the number of segments of the surveillance video of the jth video surveillance camera of the i-th monitoring node, γ represents the segmentation ratio, and B i represents the network bandwidth of the i-th monitoring node; If the network status monitoring model predicts that the network connection of the video surveillance camera changes, that is, switches from a 4G network to a 5G network, the video high-quality processing logic is triggered. Specifically: Adjust the video bit rate to adapt to the current network environment and improve the transmission efficiency of video data; Through the distributed cache system, cache management is provided for large-scale video data to reduce the transmission delay of video data.

4. A remote real-time monitoring control method based on a video surveillance camera according to claim 3, characterized in that: Before defining the key frame dynamic adjustment rules, the following steps are also included: According to the regional attributes of the video surveillance camera performing the monitoring task, the monitoring nodes in the monitoring network are divided into key area nodes and non-key area nodes, and the data transmitted by each video surveillance camera in the monitoring node is divided into key area video data and non-key area video data; According to the time attribute of the video surveillance camera performing the monitoring task, the video data transmitted by the monitoring nodes in the monitoring network are divided into video data in the key time period and video data in the non-key time period; According to the burst attribute of the video surveillance camera performing the monitoring task, the video data transmitted by the monitoring nodes in the monitoring network are divided into emergency event video data and non-emergency event video data; The video data of key area nodes, the video data of key time periods and the video data of emergencies constitute the key frame data pool.

5. A remote real-time monitoring control method based on a video surveillance camera according to claim 4, characterized in that: The key frame dynamic adjustment rule is defined, and multiple key frames are dynamically selected from the processed video data to form a time length T l The key frame sequence includes the following steps: Determine the regional attribute of the acquired surveillance video data, if it is key area video data, extract key frame data from the acquired surveillance video data, otherwise proceed to the next step; Determine the time attribute of the acquired surveillance video data. If it is video data of a key time period, extract key frame data from the acquired surveillance video data, otherwise proceed to the next step; The acquired monitoring video data is judged as burst data. If it is burst event video data, key frame data is extracted from the acquired monitoring video data. Otherwise, the monitoring video data is acquired again.

6. A remote real-time monitoring control method based on a video surveillance camera according to claim 5, characterized in that: The step of judging whether each key frame is complete comprises the following steps: Obtain the video time sequence and extract the video features of each small segment; the video features include the start time and end time of each small segment, the marker features in the small segment, the color histogram, the SIFT feature, and the HOG feature; Determine whether the key frame is divided into two small segments by analyzing video features, including: Calculating the similarity between two adjacent small fragments, including calculating the Euclidean distance or cosine similarity between the two adjacent small fragments; A preset similarity threshold is obtained. If the similarity is greater than the similarity threshold, it is determined that the key frame is not segmented; otherwise, it is determined that the key frame is segmented.

7. A remote real-time monitoring control method based on a video surveillance camera according to claim 6, characterized in that: The step of selecting a plurality of key frames in the key frame sequence for enhanced processing, increasing the transmission priority of the key frame sequence, and transmitting the key frame sequence to a designated display terminal for display includes: Improve the clarity of each key frame, including sharpening and denoising operations; At the network transmission layer, the highest priority is set for key frames to ensure the transmission efficiency of key frames, which is achieved through the Quality of Service (QoS) mechanism. Creating a buffer in the server for storing key frames; Establish a connection between the specified display terminal and the server through the socket method, including: Create TCP / IP sockets on the server and the designated display terminal respectively, and connect the server and the designated display terminal; Specify the display terminal to bind a server port and listen for connections; The server sends key frame data through the bound server port; Specify the display terminal to receive the key frame and display it; The reorganizing the key frame sequence and selecting a plurality of key frames in the reorganized key frame sequence for enhancement processing include: Reorganize the key frame sequence on the timeline, specifically: Get the time interval of the key time period [t i0 , t i1 ], where t i0 ,t i1 Respectively represent the start time and end time of the key time period; Traverse all the small fragments and obtain the time interval of each small fragment [T i0 , T i1 ], extract the start time T of each small segment i0 and end time T i1 , i=1, 2, ..., n, where n is the number of small segments; determine whether the start time and end time of the key time period fall within the time interval of the corresponding small segment, and if so, obtain the small segment i and small segment i+m where the start time and end time of the key time period are located; The small fragment i and the small fragment i+1 are spliced ​​together to form a new fragment 1, and the small fragment i+m and the small fragment i+m-1 are spliced ​​together to form a new fragment 2. Use the start time and end time of the key time period as the segmentation points, and segment the new segment 1 and the new segment 2 to separate the segments that do not contain t i0 and t i1 The small fragment i′ and the small fragment i+m′; All time intervals in the key time period [t i0 , t i1 ] constitute the reorganized key frame sequence; Reconstruction frame sequence on target integrity, specifically: Get all the small clips in the key frame sequence; Extracting a plurality of continuous recognition frames for identifying the features of the target object from two continuous small segments respectively; Extract feature points of the target object from two consecutive frames; Using a feature matching algorithm, the geometric transformation matrix between two recognition frames is estimated through the matching feature points in two consecutive recognition frames; Align the two recognition frames according to the geometric transformation matrix; The two aligned recognition frames are fused to generate a new recognition frame, wherein the new recognition frame includes complete target object features; then, the new recognition frame is updated to the key frame sequence; A sharpening or denoising operation is performed on each small segment in the reorganized focus frame sequence.

8. A remote real-time monitoring control method based on a video surveillance camera according to claim 7, characterized in that: The method of transmitting the video data associated with the key frame sequence to different display terminals after caching it locally includes the following steps: identifying key frame data and video data associated with a sequence of key frames; The video data associated with the key frame sequence includes non-key frame data connected to the key frame data on the time axis and video data containing images around the key monitoring area; The video data associated with the key frame sequence is sent to different display terminals for display.

9. A remote real-time monitoring and control system based on a video surveillance camera, characterized in that: The system is used to execute a remote real-time monitoring control method based on a video surveillance camera as described in any one of claims 1 to 8 above.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and the computer program is executed by a processor to implement a remote real-time monitoring and control method based on a video surveillance camera as described in any one of claims 1 to 8.

Images