A remote real-time monitoring control system and method based on video surveillance camera

Abstract

The present invention relates to the field of remote monitoring technology, and in particular to a remote real-time monitoring control system and method based on a video surveillance camera. The method comprises: obtaining network connection information of multiple monitoring nodes in a current monitoring network, and analyzing network state change characteristics; triggering corresponding video processing logic according to the analysis results of the current network state change characteristics, and processing video data collected by all video surveillance cameras of each monitoring node; defining a key frame dynamic adjustment rule, and judging whether each key frame is complete; if so, selecting multiple key frames in a key frame sequence for enhanced processing, improving the transmission priority of the key frame sequence, and transmitting the key frame sequence to a designated display terminal for display; otherwise, reorganizing the key frame sequence, and transmitting the reorganized key frame sequence to a designated display terminal for display; and caching the video data associated with the key frame sequence locally, and then transmitting it to different display terminals.

Description

Technical Field

[0001] The present invention relates to the field of remote monitoring technology, and in particular to a remote real-time monitoring control system and method based on a video monitoring camera. Background Art

[0002] With the continuous development and popularization of IoT technology, IoT-enabled surveillance cameras are becoming increasingly common, providing more efficient and convenient monitoring and management methods. They use built-in sensors and cameras to collect images and data, and transmit this data to the internet via wireless networks. Free from the limitations of communication lines, once the data is transmitted to the cloud or a local server, users can view real-time images and receive alert notifications anytime via mobile devices or computers. These cameras are widely used in traffic management, smart city development, smart home security systems, and chemical production monitoring.

[0003] Surveillance cameras typically connect to nearby base stations for network communication. Existing base stations offer both 4G and 5G networks, each with different communication speeds, bandwidth, and latency. Connecting to a 5G network ensures fast and efficient data transmission.

[0004] However, if the 5G network fails or is overloaded, it is usually necessary to switch from 5G to 4G to ensure normal monitoring network operation. After changing the connected network, how to ensure the normal operation of the monitoring network and the safe and fast transmission of monitoring data is an urgent problem that needs to be solved. Summary of the Invention

[0005] The present invention can monitor the network connection status of the monitoring network, control the transmission of video data according to the connected network, distinguish video data of key areas and key time periods, and transmit these key video data to a designated display terminal for display.

[0006] The technical solution proposed by the present invention is: a remote real-time monitoring and control method based on a video surveillance camera, the method comprising:

[0007] Obtain network connection information of multiple monitoring nodes in the current monitoring network and analyze network status change characteristics; the network connection information includes the network type and signal strength of the current network;

[0008] According to the analysis results of the current network status change characteristics, the corresponding video processing logic is triggered to process the video data collected by all video surveillance cameras of each monitoring node;

[0009] Define the dynamic adjustment rules of key frames, dynamically select multiple key frames from the processed video data, and construct a time length of Tl The key frame sequence is obtained, and each key frame is judged to be complete;

[0010] If yes, multiple key frames in the key frame sequence are selected for enhanced processing, the transmission priority of the key frame sequence is increased, and the key frame sequence is transmitted to a designated display terminal for display;

[0011] Otherwise, reorganizing the key frame sequence, selecting multiple key frames in the reorganized key frame sequence for enhanced processing, increasing the transmission priority of the reorganized key frame sequence, and transmitting the reorganized key frame sequence to a designated display terminal for display; the reorganizing the key frame sequence includes reorganizing on the time axis and / or reorganizing on the integrity of the target object;

[0012] The video data associated with the key frame sequence is cached locally and then transmitted to different display terminals.

[0013] Preferably, obtaining network connection information of multiple monitoring nodes in the current monitoring network and analyzing network status change characteristics include the following steps:

[0014] Acquire network connection information of multiple video surveillance cameras in a monitoring node within a preset time length according to a collection frequency to form a network connection information set;

[0015] Extracting multiple network status features from the network connection information set to form a network status feature set for each video surveillance camera;

[0016] The network status characteristics include: one or more of the number of network type changes, the maximum interval time for network type changes, the minimum interval time for network type changes, the maximum signal strength, and the minimum signal strength; the network types include 5G networks and 4G networks;

[0017] Use the pre-trained network status monitoring model to analyze the network status characteristics of each camera and predict the changes in the network connection status of each camera in the control node.

[0018] Preferably, the method of triggering corresponding video processing logic based on the analysis result of the current network state change characteristics to process the video data collected by all video surveillance cameras of each monitoring node includes the following steps:

[0019] If the network status monitoring model predicts that the network connection of the video surveillance camera changes, that is, switches from a 5G network to a 4G network, the video segmentation processing logic is triggered. Specifically:

[0020] The video data transmitted by the video surveillance camera is divided into multiple small segments to form a video time sequence. Each small segment can be transmitted independently, and the automatic retransmission request mechanism is used to ensure the reliable transmission of each small segment.

[0021] Get the network transmission bandwidth of the current network, and determine the number of small segments of video data based on the network transmission bandwidth of the current network; Specifically: Get the bandwidth of the current network, and determine the number of video segments N based on the bandwidth of the current network i_j =γB i , where N i_j represents the number of segments of the surveillance video of the jth video surveillance camera of the i-th monitoring node, γ represents the segmentation ratio, and B i represents the network bandwidth of the i-th monitoring node;

[0022] If the network status monitoring model predicts that the network connection of the video surveillance camera has changed, that is, switching from a 4G network to a 5G network, the video high-quality processing logic is triggered. Specifically:

[0023] Adjust the video bit rate to adapt to the current network environment and improve the transmission efficiency of video data;

[0024] Through the distributed cache system, cache management is provided for large-scale video data, reducing the transmission delay of video data.

[0025] Preferably, before defining the key frame dynamic adjustment rule, the following steps are also included:

[0026] Based on the regional attributes of the video surveillance cameras performing monitoring tasks, the monitoring nodes in the monitoring network are divided into key area nodes and non-key area nodes, and the data transmitted by each video surveillance camera in the monitoring node is divided into key area video data and non-key area video data;

[0027] According to the time attribute of the video surveillance camera performing the monitoring task, the video data transmitted by the monitoring nodes in the monitoring network are divided into video data in the key time period and video data in the non-key time period;

[0028] According to the burst properties of the monitoring tasks performed by the video surveillance cameras, the video data transmitted by the monitoring nodes in the monitoring network are divided into emergency video data and non-emergency video data;

[0029] The video data of key area nodes, key time period video data and emergency video data constitute the key frame data pool.

[0030] Preferably, the key frame dynamic adjustment rule is defined, and multiple key frames are dynamically selected from the processed video data to form a time length T l The key frame sequence includes the following steps:

[0031] Determine the regional attributes of the acquired surveillance video data. If it is key area video data, extract key frame data from the acquired surveillance video data; otherwise, proceed to the next step.

[0032] Determine the time attribute of the acquired surveillance video data. If it is video data of a key time period, extract key frame data from the acquired surveillance video data; otherwise, proceed to the next step.

[0033] The acquired monitoring video data is judged as burst data. If it is emergency event video data, key frame data is extracted from the acquired monitoring video data. Otherwise, the monitoring video data is acquired again.

[0034] Preferably, the determining whether each key frame is complete comprises the following steps:

[0035] Obtain the video time series and extract the video features of each small segment; the video features include the start time and end time of each small segment, the marker features within the small segment, the color histogram, SIFT features, and HOG features;

[0036] Determine whether the key frame is split into two small segments by analyzing video features, including:

[0037] Calculating the similarity between two adjacent small segments, including calculating the Euclidean distance or cosine similarity between the two adjacent small segments;

[0038] A preset similarity threshold is obtained. If the similarity is greater than the similarity threshold, it is determined that the key frame is not segmented; otherwise, it is determined that the key frame is segmented.

[0039] Preferably, the selecting of multiple key frames in the key frame sequence for enhanced processing, increasing the transmission priority of the key frame sequence, and transmitting the key frame sequence to a designated display terminal for display includes:

[0040] Improve the clarity of each key frame, including sharpening and denoising operations;

[0041] At the network transmission layer, the highest priority is set for key frames to ensure the transmission efficiency of key frames, which is achieved through the Quality of Service (QoS) mechanism.

[0042] Creating a buffer in the server for storing key frames;

[0043] Establish a connection between the specified display terminal and the server through the socket method, including:

[0044] Create TCP / IP sockets on the server and the designated display terminal respectively to connect the server and the designated display terminal;

[0045] Specify the display terminal to bind to a server port and listen for connections;

[0046] The server sends key frame data through the bound server port;

[0047] Specify the display terminal to receive and display the key frames;

[0048] The reorganizing the key frame sequence and selecting a plurality of key frames in the reorganized key frame sequence for enhancement processing include:

[0049] Reorganize the key frame sequence on the timeline, specifically:

[0050] Get the time interval of the key time period [t i0 , t i1 ], where t i0 , t i1 Respectively represent the start time and end time of the key time period;

[0051] Traverse all the small fragments and get the time interval of each small fragment [T i0 , T i1 ], extract the start time T of each small segment i0 and end time T i1 , i = 1, 2, ..., n, where n is the number of small segments; determine whether the start time and end time of the key time segment fall within the time interval of the corresponding small segment, and if so, obtain the small segment i and small segment i+m where the start time and end time of the key time segment are located;

[0052] Splice small fragment i and small fragment i+1 to form new fragment 1, and splice small fragment i+m and small fragment i+m-1 to form new fragment 2.

[0053] Use the start time and end time of the key time period as the segmentation points, and segment the new segment 1 and the new segment 2 to separate the segments that do not contain t i0 and t i1 Small fragment i′ and small fragment i+m′;

[0054] All time intervals in the key time period [t i0 , t i1 ] constitute the reorganized key frame sequence;

[0055] Reconstruct the frame sequence based on the integrity of the target, specifically:

[0056] Get all the small clips in the key frame sequence;

[0057] Extracting multiple consecutive recognition frames for identifying target features from two consecutive small segments respectively;

[0058] Extract the feature points of the target object from two consecutive frames;

[0059] Using feature matching algorithm, the geometric transformation matrix between two recognition frames is estimated through the matching feature points in two consecutive recognition frames;

[0060] Align the two recognition frames according to the geometric transformation matrix;

[0061] The two aligned recognition frames are fused to generate a new recognition frame, wherein the new recognition frame includes complete target object features; then, the new recognition frame is updated to the key frame sequence;

[0062] A sharpening or denoising operation is performed on each small segment in the reorganized key frame sequence.

[0063] Preferably, the step of caching the video data associated with the key frame sequence locally and then transmitting the video data to different display terminals comprises the following steps:

[0064] identifying key frame data and video data associated with a sequence of key frames;

[0065] The video data associated with the key frame sequence includes non-key frame data connected to the key frame data on the time axis and video data containing images around the key monitoring area;

[0066] The video data associated with the key frame sequence is sent to different display terminals for display.

[0067] The present invention also provides a remote real-time monitoring control system based on a video surveillance camera, and the system is used to execute the remote real-time monitoring control method based on a video surveillance camera.

[0068] The present invention also provides a computer-readable storage medium storing a computer program, wherein the computer program is executed by a processor to implement the remote real-time monitoring and control method based on a video surveillance camera.

[0069] Beneficial effects of the present invention:

[0070] 1. The present invention adopts different data processing strategies to control the video data processing and transmission modes when the communication network to which the video surveillance camera is connected changes. This solves the problem that when the 5G network switches to the 4G network, the timeliness of video data transmission is easily affected by changes in network transmission speed and delay.

[0071] 2. The present invention, when dividing the video data into multiple small segments for transmission, detects whether the key frames in each small segment are complete, thereby ensuring that the monitoring data of the key monitoring areas, key monitoring time periods and emergency time periods can be transmitted to the designated display terminal in a timely and complete manner, making it convenient for relevant personnel to obtain video data. BRIEF DESCRIPTION OF THE DRAWINGS

[0072] Figure 1 The present invention is a flow chart of a remote real-time monitoring control method based on a video surveillance camera. DETAILED DESCRIPTION

[0073] The following description is intended to disclose the present invention so that those skilled in the art can implement the present invention. The preferred embodiments described below are for illustrative purposes only, and those skilled in the art will readily appreciate other obvious variations. The basic principles of the present invention defined in the following description may be applied to other embodiments, variations, improvements, equivalents, and other technical solutions that do not depart from the spirit and scope of the present invention.

[0074] It is to be understood that the term "one" should be understood as "at least one" or "one or more", that is, in one embodiment, the number of an element may be one, while in another embodiment, the number of the elements may be multiple, and the term "one" should not be understood as a limitation on the quantity.

[0075] refer to Figure 1 The present invention provides a technical solution: a remote real-time monitoring and control method based on a video surveillance camera, comprising the following steps:

[0076] Step 1: Obtain network connection information of multiple monitoring nodes in the current monitoring network and analyze network status change characteristics; the network connection information includes the network type and signal strength of the current network; specifically includes:

[0077] Acquire network connection information of multiple video surveillance cameras in a monitoring node within a preset time length according to a collection frequency to form a network connection information set;

[0078] Extracting multiple network status features from the network connection information set to form a network status feature set for each video surveillance camera;

[0079] The network status characteristics include: one or more of the number of network type changes, the maximum interval time for network type changes, the minimum interval time for network type changes, the maximum signal strength, and the minimum signal strength; the network types include 5G networks and 4G networks;

[0080] Use the pre-trained network status monitoring model to analyze the network status characteristics of each camera and predict the changes in the network connection status of each camera in the control node.

[0081] The pre-trained network status monitoring model can be constructed based on algorithms such as decision trees, random forests, and support vector machines. In this embodiment, the network status monitoring model is constructed using a random forest algorithm to output the probability of network switching. If the output network switching probability is greater than a preset switching probability threshold, it is determined that the network is about to switch.

[0082] Step 2: Based on the analysis results of the current network status change characteristics, trigger the corresponding video processing logic to process the video data collected by all video surveillance cameras of each monitoring node; including the following steps:

[0083] If the network status monitoring model predicts that the network connection of the video surveillance camera changes, that is, switches from a 5G network to a 4G network, the video segmentation processing logic is triggered. Specifically:

[0084] The video data transmitted by the video surveillance camera is divided into multiple small segments to form a video time sequence. Each small segment can be transmitted independently, and the automatic retransmission request mechanism is used to ensure the reliable transmission of each small segment.

[0085] Get the network transmission bandwidth of the current network, and determine the number of small segments of video data based on the network transmission bandwidth of the current network; Specifically: Get the bandwidth of the current network, and determine the number of video segments N based on the bandwidth of the current network i_j =γB i , where N i_j represents the number of segments of the surveillance video of the jth video surveillance camera of the i-th monitoring node, γ represents the segmentation ratio, and B i Indicates the network bandwidth of the i-th monitoring node.

[0086] If the network status monitoring model predicts that the network connection of the video surveillance camera has changed, that is, switching from a 4G network to a 5G network, the video high-quality processing logic is triggered. Specifically:

[0087] Adjust the video bit rate to adapt to the current network environment and improve the transmission efficiency of video data;

[0088] Through the distributed cache system, cache management is provided for large-scale video data, reducing the transmission delay of video data.

[0089] In a surveillance network, multiple monitoring nodes are typically deployed, each monitoring area and operating hours varying. Each monitoring node typically houses at least one video surveillance camera. To enable remote monitoring, these cameras typically include IoT communication capabilities, enabling long-distance transmission of video data and reception of control commands. Consequently, each monitoring node possesses distinct attributes (e.g., operating area, operating hours), which in turn impart distinct properties to the video data from the corresponding monitoring node.

[0090] Step 3: Based on the regional attributes of the video surveillance cameras performing the monitoring tasks, the monitoring nodes in the monitoring network are divided into key area nodes and non-key area nodes, and the data transmitted by each video surveillance camera in the monitoring node is divided into key area video data and non-key area video data;

[0091] According to the time attribute of the video surveillance camera performing the monitoring task, the video data transmitted by the monitoring nodes in the monitoring network are divided into video data in the key time period and video data in the non-key time period;

[0092] According to the burst properties of the monitoring tasks performed by the video surveillance cameras, the video data transmitted by the monitoring nodes in the monitoring network are divided into emergency video data and non-emergency video data;

[0093] The video data of key area nodes, key time period video data and emergency video data constitute the key frame data pool. The terminal frame data pool can be set in the cloud or local server.

[0094] Define the dynamic adjustment rules of key frames, dynamically select multiple key frames from the processed video data, and construct a time length of T l The key frame sequence specifically includes: judging the regional attributes of the acquired surveillance video data, if it is key area video data, extracting key frame data from the acquired surveillance video data, otherwise proceeding to the next step;

[0095] Determine the time attribute of the acquired surveillance video data. If it is video data of a key time period, extract key frame data from the acquired surveillance video data; otherwise, proceed to the next step.

[0096] The acquired monitoring video data is judged as burst data. If it is emergency event video data, key frame data is extracted from the acquired monitoring video data. Otherwise, the monitoring video data is acquired again.

[0097] Then, it is necessary to determine whether the key frames included in the video data of each small segment are complete. The specific steps are as follows: obtain the video time series and extract the video features of each small segment; the video features include the start and end time of each small segment, the marker features in the small segment, the color histogram, the SIFT feature, and the HOG feature;

[0098] Determine whether the key frame is split into two small segments by analyzing video features, including:

[0099] Calculate the similarity between two adjacent small segments, including calculating the Euclidean distance or cosine similarity between the two adjacent small segments; obtain a preset similarity threshold, if the similarity is greater than the similarity threshold, determine that the key frame is not segmented; otherwise, determine that the key frame is segmented.

[0100] Step 4: If the key frames included in the video sequence of each small segment are complete, multiple key frames in the key frame sequence are selected for enhancement processing, the transmission priority of the key frame sequence is increased, and the key frame sequence is transmitted to the designated display terminal for display. The specific process is as follows:

[0101] Improve the clarity of each key frame, including sharpening and denoising operations;

[0102] At the network transport layer, the highest priority is set for key frames to ensure their transmission efficiency, specifically through the Quality of Service (QoS) mechanism; a buffer area is created within the server to store key frames;

[0103] Establish a connection between the specified display terminal and the server through the socket method, including:

[0104] Create TCP / IP sockets on the server and the designated display terminal respectively to connect the server and the designated display terminal;

[0105] The designated display terminal is bound to a server port and listens for connections; the server sends key frame data through the bound server port; the designated display terminal receives the key frame and displays the received key frame.

[0106] If the key frames included in the video clips in each small segment are incomplete, it is necessary to reorganize the key frame sequence, select multiple key frames in the reorganized key frame sequence for enhanced processing, increase the transmission priority of the reorganized key frame sequence, and transmit the reorganized key frame sequence to the designated display terminal for display.

[0107] The reorganization here includes the reorganization on the timeline and the reorganization on the integrity of the target object;

[0108] Among them, the reorganization on the time axis includes: obtaining the time interval of the key time period [t i0 , t i1 ], where t i0 , t i1 Respectively represent the start time and end time of the key time period;

[0109] Traverse all the small fragments and get the time interval of each small fragment [T i0 , T i1 ], extract the start time T of each small segment i0 and end time T i1 , i = 1, 2, ..., n, where n is the number of small segments; determine whether the start time and end time of the key time segment fall within the time interval of the corresponding small segment, and if so, obtain the small segment i and small segment i+m where the start time and end time of the key time segment are located;

[0110] Splice small fragment i and small fragment i+1 to form new fragment 1, and splice small fragment i+m and small fragment i+m-1 to form new fragment 2.

[0111] Use the start time and end time of the key time period as the segmentation points, and segment the new segment 1 and the new segment 2 to separate the segments that do not contain t i0 and t i1 Small fragment i′ and small fragment i+m′;

[0112] All time intervals in the key time period [t i0 , t i1 ] constitute the reorganized key frame sequence;

[0113] A sharpening or denoising operation is performed on each small segment in the reorganized key frame sequence.

[0114] Among them, the reconstruction of the target integrity includes:

[0115] Get all the small clips in the key frame sequence;

[0116] Extracting multiple consecutive recognition frames for identifying target features from two consecutive small segments respectively;

[0117] Extract feature points of the target object from two consecutive frames; in this embodiment, an ORB feature detector is used to extract key points and descriptors in the image;

[0118] Using a feature matching algorithm, the geometric transformation matrix between two consecutive recognition frames is estimated using matching feature points in the two frames. In this embodiment, Brute Force Matcher (BFMatcher) is used for feature point matching. BFMatcher finds the best match by calculating the distance between descriptors. Using the matched feature points, the affine transformation matrix between the two frames is estimated using the cv2.estimateAffinePartial2D function. This matrix describes how to transform the second frame image into the coordinate system of the first frame image.

[0119] The two recognition frames are aligned according to the geometric transformation matrix. In this embodiment, the cv2.warpAffine function is used to align the second frame image to the coordinate system of the first frame image according to the estimated transformation matrix.

[0120] The two aligned recognition frames are fused to generate a new recognition frame that includes the complete target object features. In this embodiment, the two aligned frames are weighted averaged using the cv2.addWeighted function to generate the final fused image. The above function is included in the Python and OpenCV libraries.

[0121] For example, in a public security monitoring system, if the transmitted video data is the video data of a key monitored area, the monitoring point has two video surveillance cameras, the monitoring area of ​​the first video surveillance camera is the key area, and the monitoring area of ​​the other is the area outside the key area, the video data of the first camera is transmitted to the designated display terminal for display, if a suspicious person (target object) appears in the key monitored area and moves from the key monitored area to the outside of the key area, some video frames will have only half of the target object. If these video frames include some key information, it is necessary to call the video data of another camera, splice the target object together, and then display it on the designated display terminal.

[0122] Step 5: After caching the video data associated with the key frame sequence locally, transmit it to different display terminals. The designated display terminal can send a video call instruction to the server as needed to obtain the video data associated with the key frame sequence from the server. The corresponding video data can be transmitted through the bound server port to ensure transmission security. The specific steps include:

[0123] identifying key frame data and video data associated with a sequence of key frames;

[0124] The video data associated with the key frame sequence includes non-key frame data connected to the key frame data on the time axis and video data containing images around the key monitoring area;

[0125] The video data associated with the key frame sequence is sent to different display terminals for display.

[0126] The present invention also provides a remote real-time monitoring control system based on a video surveillance camera, and the system is used to execute the remote real-time monitoring control method based on a video surveillance camera.

[0127] The present invention also provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and the computer program is executed by a processor to implement the remote real-time monitoring and control method based on a video surveillance camera.

[0128] The embodiments disclosed in the present invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. The embodiments disclosed in the present invention include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part, and / or installed from a removable medium. When the computer program is executed by the central processing unit (CPU), the above-mentioned functions defined in the method of the present application are executed. It should be noted that the computer-readable medium mentioned above in the present application can be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to, an electrical connection having one or more wire segments, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In this application, a computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this application, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries computer-readable program code. Such a propagated data signal may take a variety of forms, including, but not limited to, an electromagnetic signal, an optical signal, or any suitable combination thereof. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transfer a program for use by or in conjunction with an instruction execution system, apparatus, or device. Program code embodied on a computer-readable medium may be transmitted using any appropriate medium, including but not limited to wireless, electrical wire, optical fiber cable, RF, etc., or any suitable combination thereof.

[0129] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of the systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or portion of code that contains one or more executable instructions for implementing the specified logical functions. It should also be noted that in some alternative implementations, the functions marked in the boxes can also occur in an order different from that marked in the accompanying drawings. For example, two boxes shown in succession can actually be executed substantially in parallel, or they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, as well as combinations of boxes in the block diagram and / or flowchart, can be implemented using a dedicated hardware-based system that performs the specified functions or operations, or can be implemented using a combination of dedicated hardware and computer instructions.

[0130] Those skilled in the art should understand that the embodiments of the present invention described above and shown in the accompanying drawings are only examples and do not limit the present invention. The objectives of the present invention have been fully and effectively achieved. The functional and structural principles of the present invention have been demonstrated and explained in the embodiments. Without departing from the principles, the implementation methods of the present invention may be subject to any changes or modifications.

Claims

1. A remote real-time monitoring and control method based on a video surveillance camera, characterized in that: The method comprises: Obtain network connection information of multiple monitoring nodes in the current monitoring network and analyze network status change characteristics; the network connection information includes the network type and signal strength of the current network; Based on the analysis results of the current network status change characteristics, the corresponding video processing logic is triggered to process the video data collected by all video surveillance cameras at each monitoring node. This includes: if the network status monitoring model predicts that the network connection of the video surveillance camera has changed, that is, switching from a 5G network to a 4G network, the video segmentation processing logic is triggered. Specifically: The video data transmitted by the video surveillance camera is divided into multiple small segments to form a video time sequence. Each small segment can be transmitted independently, and the automatic retransmission request mechanism is used to ensure the reliable transmission of each small segment. Get the network transmission bandwidth of the current network, and determine the number of small segments of video data based on the network transmission bandwidth of the current network; Specifically: Get the bandwidth of the current network, and determine the number of video segments N based on the bandwidth of the current network i_j =γB i , where N i_j represents the number of segments of the surveillance video of the jth video surveillance camera of the i-th monitoring node, γ represents the segmentation ratio, and B i represents the network bandwidth of the i-th monitoring node; Define the dynamic adjustment rules of key frames, dynamically select multiple key frames from the processed video data, and construct a time length of T l The key frame sequence of Determine whether each key frame is complete; if so, select multiple key frames within the key frame sequence for enhanced processing, increase the transmission priority of the key frame sequence, and transmit the key frame sequence to a designated display terminal for display; otherwise, reorganize the key frame sequence, select multiple key frames within the reorganized key frame sequence for enhanced processing, increase the transmission priority of the reorganized key frame sequence, and transmit the reorganized key frame sequence to a designated display terminal for display; the reorganization of the key frame sequence includes reorganization on the time axis and / or reorganization on the integrity of the target object; The video data associated with the key frame sequence is cached locally and then transmitted to different display terminals.

2. A remote real-time monitoring control method based on a video surveillance camera according to claim 1, characterized in that: The method of obtaining network connection information of multiple monitoring nodes in the current monitoring network and analyzing network status change characteristics includes the following steps: Acquire network connection information of multiple video surveillance cameras in a monitoring node within a preset time length according to a collection frequency to form a network connection information set; Extracting multiple network status features from the network connection information set to form a network status feature set for each video surveillance camera; The network status characteristics include: one or more of the number of network type changes, the maximum interval time for network type changes, the minimum interval time for network type changes, the maximum signal strength, and the minimum signal strength; the network types include 5G networks and 4G networks; Use the pre-trained network status monitoring model to analyze the network status characteristics of each camera and predict the changes in the network connection status of each camera in the control node.

3. A remote real-time monitoring control method based on a video surveillance camera according to claim 2, characterized in that: The method further includes the following steps: triggering corresponding video processing logic according to the analysis result of the current network status change characteristics to process the video data collected by all video surveillance cameras of each monitoring node: If the network status monitoring model predicts that the network connection of the video surveillance camera has changed, that is, switching from a 4G network to a 5G network, the video high-quality processing logic is triggered. Specifically: Adjust the video bit rate to adapt to the current network environment and improve the transmission efficiency of video data; Through the distributed cache system, cache management is provided for large-scale video data, reducing the transmission delay of video data.

4. A remote real-time monitoring and control method based on a video surveillance camera according to claim 3, characterized in that: Before defining the key frame dynamic adjustment rules, the following steps are also included: Based on the regional attributes of the video surveillance cameras performing monitoring tasks, the monitoring nodes in the monitoring network are divided into key area nodes and non-key area nodes, and the data transmitted by each video surveillance camera in the monitoring node is divided into key area video data and non-key area video data; According to the time attribute of the video surveillance camera performing the monitoring task, the video data transmitted by the monitoring nodes in the monitoring network are divided into video data in the key time period and video data in the non-key time period; According to the burst properties of the monitoring tasks performed by the video surveillance cameras, the video data transmitted by the monitoring nodes in the monitoring network are divided into emergency video data and non-emergency video data; The video data of key area nodes, key time period video data and emergency video data constitute the key frame data pool.

5. A remote real-time monitoring and control method based on a video surveillance camera according to claim 4, characterized in that: The key frame dynamic adjustment rule is defined, and multiple key frames are dynamically selected from the processed video data to form a time length T l The key frame sequence includes the following steps: Determine the regional attributes of the acquired surveillance video data. If it is key area video data, extract key frame data from the acquired surveillance video data; otherwise, proceed to the next step. Determine the time attribute of the acquired surveillance video data. If it is video data of a key time period, extract key frame data from the acquired surveillance video data; otherwise, proceed to the next step. The acquired monitoring video data is judged as burst data. If it is emergency event video data, key frame data is extracted from the acquired monitoring video data. Otherwise, the monitoring video data is acquired again.

6. A remote real-time monitoring and control method based on a video surveillance camera according to claim 5, characterized in that: Determining whether each key frame is complete includes the following steps: Obtain the video time series and extract the video features of each small segment; the video features include the start time and end time of each small segment, the marker features within the small segment, the color histogram, SIFT features, and HOG features; Determine whether the key frame is split into two small segments by analyzing video features, including: Calculating the similarity between two adjacent small segments, including calculating the Euclidean distance or cosine similarity between the two adjacent small segments; A preset similarity threshold is obtained. If the similarity is greater than the similarity threshold, it is determined that the key frame is not segmented; otherwise, it is determined that the key frame is segmented.

7. A remote real-time monitoring and control method based on a video surveillance camera according to claim 6, characterized in that: The step of selecting a plurality of key frames in the key frame sequence for enhanced processing, increasing the transmission priority of the key frame sequence, and transmitting the key frame sequence to a designated display terminal for display includes: Improve the clarity of each key frame, including sharpening and denoising operations; At the network transmission layer, the highest priority is set for key frames to ensure the transmission efficiency of key frames, which is achieved through the Quality of Service (QoS) mechanism. Creating a buffer in the server for storing key frames; Establish a connection between the specified display terminal and the server through the socket method, including: Create TCP / IP sockets on the server and the designated display terminal respectively to connect the server and the designated display terminal; Specify the display terminal to bind to a server port and listen for connections; The server sends key frame data through the bound server port; Specify the display terminal to receive and display the key frames; The reorganizing the key frame sequence and selecting a plurality of key frames in the reorganized key frame sequence for enhancement processing include: Reorganize the key frame sequence on the timeline, specifically: Get the time interval of the key time period [t i0 , t i1 ], where t i0 , t i1 Respectively represent the start time and end time of the key time period; Traverse all the small fragments and get the time interval of each small fragment [T i0 , T i1 ], extract the start time T of each small segment i0 and end time T i1 , i = 1, 2, ..., n, where n is the number of small segments; determine whether the start time and end time of the key time segment fall within the time interval of the corresponding small segment, and if so, obtain the small segment i and small segment i+m where the start time and end time of the key time segment are located; Splice small fragment i and small fragment i+1 to form new fragment 1, and splice small fragment i+m and small fragment i+m-1 to form new fragment 2. Use the start time and end time of the key time period as the segmentation points, and segment the new segment 1 and the new segment 2 to separate the segments that do not contain t i0 and t i1 Small fragment i′ and small fragment i+m′; All time intervals in the key time period [t i0 , t i1 ] constitute the reorganized key frame sequence; Reconstruct the frame sequence based on the integrity of the target, specifically: Get all the small clips in the key frame sequence; Extracting multiple consecutive recognition frames for identifying target features from two consecutive small segments respectively; Extract the feature points of the target object from two consecutive frames; Using feature matching algorithm, the geometric transformation matrix between two recognition frames is estimated through the matching feature points in two consecutive recognition frames; Align the two recognition frames according to the geometric transformation matrix; The two aligned recognition frames are fused to generate a new recognition frame, wherein the new recognition frame includes complete target object features; then, the new recognition frame is updated to the key frame sequence; A sharpening or denoising operation is performed on each small segment in the reorganized key frame sequence.

8. The remote real-time monitoring control method based on a video surveillance camera according to claim 7, characterized in that: The method of caching the video data associated with the key frame sequence locally and then transmitting the video data to different display terminals includes the following steps: identifying key frame data and video data associated with a sequence of key frames; The video data associated with the key frame sequence includes non-key frame data connected to the key frame data on the time axis and video data containing images around the key monitoring area; The video data associated with the key frame sequence is sent to different display terminals for display.

9. A remote real-time monitoring and control system based on a video surveillance camera, characterized in that: The system is used to execute a remote real-time monitoring and control method based on a video surveillance camera as described in any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and the computer program is executed by a processor to implement the remote real-time monitoring and control method based on a video surveillance camera as described in any one of claims 1 to 8.

Images