Remote real-time monitoring control system and method based on video monitoring camera

Abstract

This invention relates to the field of video surveillance technology, specifically disclosing a remote real-time monitoring and control system and method based on video surveillance cameras. The system uses an edge intelligent processing center to perform semantic analysis on the video stream to identify regions of interest, and utilizes a non-uniform coding transmission unit to execute differentiated coding strategies for different regions, achieving on-demand allocation of bandwidth resources. The system integrates an instruction prediction execution mechanism and a virtual window synthesis device, using a historical habit model for feedforward calculation and combining pixel compensation logic to generate predicted image frames, providing operators with immediate visual feedback. In conjunction with a network status perception unit and an adaptive dual-channel transmission unit, the transmission weights of video and control instructions are dynamically adjusted, and embedded control slices are constructed. This invention solves the problems of high-definition transmission and low-latency response, eliminating the risk of blind operation while improving interaction accuracy and system robustness.

Description

Technical Field

[0001] This invention belongs to the field of video surveillance technology, specifically relating to a remote real-time monitoring and control system and method based on video surveillance cameras. Background Technology

[0002] With the rapid evolution of IoT and smart security technologies, remote real-time monitoring systems are increasingly being used in traffic management, industrial inspection, and smart cities. As a core means of achieving remote sensing and real-time interaction, monitoring systems, through deep collaboration between front-end camera devices and cloud control platforms, provide crucial decision-making support for various emergency dispatching and security assurance. As users' demands for monitoring image quality continue to rise, high-definition and high-frame-rate real-time video interaction has become an important indicator for measuring monitoring effectiveness.

[0003] Remote PTZ control and lens adjustment technology based on video streaming is a core direction for improving the interactive experience of monitoring. To ensure monitoring flexibility, existing technologies typically use standard encoding protocols to compress the acquired video data and transmit it back to the control terminal via a network link. Simultaneously, mechanical control commands issued by the operator are fed back to the camera via a signaling channel. This technical solution aims to achieve a closed-loop interaction between image information and physical actions.

[0004] Existing technologies suffer from severe resource allocation conflicts when prioritizing the transmission of high-definition video streams and control signaling. During full-scale transmission of high-resolution images, the massive bitstream load often causes signaling transmission congestion and queuing. Traditional encoding methods perform equal-intensity calculations across the entire frame, resulting in a large amount of irrelevant and redundant information occupying critical signaling channels. This leads to a perceived lag in remote control feedback, making it easy for operators to operate blindly in complex environments due to latency. Existing control logic relies too heavily on linear feedback, lacking the ability to dynamically adapt to network bandwidth fluctuations and predict and compensate for control actions, making it difficult to achieve real-time control feedback in unbalanced bandwidth environments. Summary of the Invention

[0005] The purpose of this invention is to provide a remote real-time monitoring and control system based on video surveillance cameras, which can solve the problems in the background art of the contradiction between high-resolution real-time transmission and low-latency control response, the risk of blind operation caused by control lag, the occupation of bandwidth resources by invalid redundant data, and the decrease in the accuracy of monitoring interaction under network fluctuations.

[0006] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0007] The remote real-time monitoring and control system based on video surveillance cameras includes a video acquisition device, an edge intelligent processing center, a non-uniform encoding transmission unit, a remote control terminal, an instruction prediction and execution mechanism, a PTZ drive component, a network status sensing unit, a virtual window synthesis device, an adaptive dual-channel transmission unit, and a system management server.

[0008] The video acquisition device is used to capture continuous light signals in the monitored area through an optical sensor array and convert them into raw digital video streams. The video acquisition device supports real-time adjustment of focus parameters and exposure parameters.

[0009] The edge intelligent processing center is connected to the video acquisition device and is used to run visual attention analysis logic to perform real-time semantic parsing of the original digital video stream. By extracting spatial and temporal motion features from the image sequence, it identifies the region of interest and non-interest background region in the monitoring screen.

[0010] The non-uniform encoding transmission unit is connected to the edge intelligent processing center and is used to dynamically adjust the quantization parameter allocation strategy of video encoding according to the coordinate information and importance level of the region of interest. It adopts a preset high-resolution encoding configuration for the region of interest and a preset low bitrate contour maintenance encoding configuration for the non-interest background region, forming a non-uniformly distributed compressed bitstream.

[0011] The remote control terminal is used to provide an interactive interface for the operator, display the received monitoring screen in real time, and collect the physical control commands input by the operator. The physical control commands include adjustment parameters for the pan-tilt unit's rotation angle, pitch state, and lens magnification.

[0012] The instruction prediction and execution mechanism is connected to the remote control terminal and the gimbal drive component, respectively, and is used to store historical operation habit models. When it receives a physical control instruction issued by the remote control terminal, it performs feedforward calculation based on the historical operation habit model to generate local real-time execution instructions and expected motion vectors.

[0013] The gimbal drive component is connected to the instruction prediction and execution mechanism, and is used to receive the local real-time execution instruction, drive the camera mechanical structure to perform real-time actions, and feed back the real-time physical position information of the gimbal to the instruction prediction and execution mechanism.

[0014] The network status sensing unit is used to monitor the bandwidth utilization, packet loss rate and round-trip delay of the data transmission link in real time, and feed back the monitoring results to the non-uniform coding transmission unit and the adaptive dual-channel transmission unit.

[0015] The virtual window synthesis device is connected to the instruction prediction execution mechanism. It is used to use the background frame information of the previous time sequence and the expected motion vector to quickly synthesize a predicted image frame locally through geometric transformation and pixel compensation logic, and to transmit the predicted image frame to the remote control terminal for pre-rendering.

[0016] The adaptive dual-channel transmission unit is used to construct an embedded control slice structure, embed high-priority control status synchronization signals into the idle field of the video transmission message, and dynamically switch the transmission weight of video data and control commands according to the monitoring results of the network status perception unit.

[0017] The system management server is used to coordinate communication synchronization between various components, manage user permissions, and store semantically annotated monitoring data.

[0018] Preferably, when identifying the region of interest, the edge intelligent processing center adopts a multi-scale feature fusion strategy, which determines the boundary range of the target object by calculating the gradient change rate at the pixel level and the motion displacement vector at the macroblock level, and identifies human bodies, vehicles or preset targets in motion as the highest level region of interest.

[0019] Furthermore, when performing quantization parameter allocation, the non-uniform coding transmission unit reduces the sampling rate of the chroma components in the non-interested background region and increases its quantization step size. Under the premise of ensuring that the background outline is recognizable, it significantly reduces the bitstream proportion of the background data and prioritizes the allocation of the saved bandwidth resources to the texture detail encoding of the region of interest.

[0020] Furthermore, the remote control terminal includes a high-precision rendering engine that supports a dual-buffering mechanism. This engine can simultaneously receive the predicted image frames generated by the virtual window synthesis device and the real video frames transmitted back by the adaptive dual-channel transmission unit, and uses a smooth weighting algorithm to achieve seamless switching between the two.

[0021] Furthermore, the instruction prediction and execution mechanism integrates user behavior learning logic. This user behavior learning logic records the operator's operation trajectory, click frequency, and control habits in different monitoring scenarios, and uses statistical distribution characteristics to construct personalized operation feature vectors to improve the accuracy of predicting physical control instructions.

[0022] Furthermore, when synthesizing the predicted image frame, the virtual window synthesis device uses an affine transformation matrix to calculate the coordinate mapping relationship of the image in three-dimensional space. In response to the field of view changes caused by the rotation of the gimbal, it fills the visual gap area caused by the motion by tiling and stretching the historical background pixels.

[0023] Furthermore, when the network status sensing unit detects that the network packet loss rate exceeds a preset threshold, it immediately sends an emergency mode trigger signal to the adaptive dual-channel transmission unit.

[0024] Furthermore, upon receiving the emergency mode trigger signal, the adaptive dual-channel transmission unit executes data stream truncation control, temporarily stopping the transmission of non-critical video frame data, and retaining only the differential information of critical video frames and the highest priority PTZ control commands, ensuring the absolute smooth operation of the control link in a narrowband environment.

[0025] Furthermore, the system management server also includes a load balancer, which dynamically allocates computing resources of the edge intelligent processing center based on the geographical location and network quality of each terminal when multiple remote control terminals access the system concurrently.

[0026] Furthermore, the video acquisition device is also equipped with an ambient light sensor, which detects ambient illuminance information and feeds it back to the edge intelligent processing center in real time as a reference for adjusting the contrast threshold in the visual attention analysis logic.

[0027] Furthermore, the gimbal drive component employs a high-resolution encoder for closed-loop control, with a position feedback frequency higher than the preset video frame rate, to ensure that the execution accuracy of the mechanical action remains highly consistent with the expected motion vector generated by the command prediction execution mechanism.

[0028] Furthermore, the non-uniform encoding transmission unit also supports dynamic adjustment of the frame rate based on the importance of the region of interest. For static background areas, a very low frame rate is used for transmission, while for regions of interest with high-speed moving targets, a preset maximum frame rate is used for data acquisition and encoding.

[0029] Furthermore, when generating the expected motion vector, the command prediction execution mechanism combines the current angular velocity and angular acceleration sensor data of the camera to compensate for and correct the motion overshoot caused by mechanical inertia.

[0030] Furthermore, the adaptive dual-channel transmission unit defines a dedicated signaling priority field in the transmission protocol stack, which is configured to enjoy the highest forwarding priority in router and switch devices.

[0031] Furthermore, the interactive interface of the remote control terminal can automatically overlay augmented reality guidance markers on the screen based on the received coordinates of the region of interest, assisting the operator in quickly locking onto potential threat targets.

[0032] Furthermore, when processing zoom instructions, the virtual window synthesis device performs interpolation calculations on the pixels in the central region to present a digitally magnified visual effect to the operator before the actual optical zoom feedback arrives.

[0033] Furthermore, the edge intelligent processing center also supports abnormal behavior warning. When the target movement trajectory in the region of interest matches a preset abnormal pattern, the system will automatically increase the encoding priority of the region of interest and trigger the alarm device of the remote control terminal.

[0034] Furthermore, the system management server records the timestamp of each physical control command being sent, the timestamp of execution completion, and the corresponding network status parameters, which are used for offline analysis and optimization of the system control latency performance in the later stages.

[0035] Furthermore, after the network bandwidth recovers to a preset normal range, the non-uniform coding transmission unit will automatically reduce the quantization step size and gradually restore the image clarity of non-interested background areas.

[0036] Furthermore, the gimbal drive component has overload protection logic. When mechanical resistance is detected to exceed a preset safety threshold, it will automatically enter the protection state and report a fault code to the remote control terminal.

[0037] Furthermore, the virtual window synthesis device can also combine geographic information system data to embed surrounding terrain, landforms and building outlines into the predicted image frame, enhancing the operator's remote spatial perception capabilities.

[0038] The present invention also provides a remote real-time monitoring and control method based on a video surveillance camera, which uses the aforementioned remote real-time monitoring and control system based on a video surveillance camera to achieve remote real-time monitoring.

[0039] Compared with the prior art, the present invention has the following beneficial effects:

[0040] 1. This invention breaks through the inherent limitations of traditional video surveillance's full-frame uniform encoding by introducing a visual attention mechanism and non-uniform coding technology. By dynamically allocating computing power and bandwidth resources to the region of interest, the image clarity of key monitoring targets is improved without increasing physical bandwidth costs. This semantic-first resource allocation strategy filters out redundant background information, freeing up valuable channel space for the transmission of remote control commands and resolving the bandwidth competition problem between high-definition transmission and real-time control.

[0041] 2. This invention, by deploying a command prediction execution mechanism at the camera end, particularly through the application of a virtual window synthesis device, enables the operator to obtain visual feedback by predicting image frames even when the actual video frames arrive late due to network latency. This visual compensation mechanism based on feedforward logic subjectively creates a zero-latency control experience for the operator, eliminates the risk of blind operation in remote monitoring, and improves the accuracy of tracking high-speed moving targets.

[0042] 3. This invention possesses extremely strong network adaptability. The deep coupling between the adaptive dual-channel transmission unit and the network status awareness unit enables the system to dynamically adjust data flow strategies based on real-time link quality. In harsh network environments or situations with severe signal fluctuations, the system ensures absolute priority of control signaling through embedded control slicing and emergency modes, avoiding control failures or system disconnections caused by network congestion, and guaranteeing the robustness and reliability of the monitoring system.

[0043] 4. This invention, through semantic analysis at the edge intelligent processing center, not only reduces network transmission pressure but also alleviates the decoding and storage burden on the remote monitoring center server. This edge computing-based architecture aligns with the development trend of IoT intelligence, providing efficient system support for large-scale, high-density intelligent monitoring networks.

[0044] 5. This invention, through architectural innovation and the integration of interdisciplinary technologies, demonstrates technological advancements and broad application prospects despite the limitations of physical network conditions. Attached Figure Description

[0045] Figure 1 This is a schematic diagram of the overall technical solution architecture proposed in this invention;

[0046] Figure 2 This is a schematic diagram of the core principle framework of this invention;

[0047] Figure 3 This is a schematic diagram of the core principle framework of instruction prediction execution and virtual window synthesis based on feedforward calculation in this invention;

[0048] Figure 4 This is a flowchart illustrating the logical process framework for real-time semantic parsing and region of interest extraction of raw digital video streams in this invention.

[0049] Figure 5 This is a schematic diagram of the multi-level interaction relationship and data flow between the remote control terminal, the instruction prediction and execution mechanism, and the gimbal drive component in this invention;

[0050] Figure 6 This is a flowchart illustrating the logic of adaptive dual-channel transmission weight dynamic adjustment based on network state awareness in this invention. Detailed Implementation

[0051] Example 1: Please refer to the appendix Figure 1 To be continued Figure 6A remote real-time monitoring and control system based on video surveillance cameras includes a video acquisition device, an edge intelligent processing center, a non-uniform encoding transmission unit, a remote control terminal, an instruction prediction and execution mechanism, a PTZ drive component, a network status sensing unit, a virtual window synthesis device, an adaptive dual-channel transmission unit, and a system management server.

[0052] The video acquisition device is configured to capture optical signals of the monitored scene in real time using a high-sensitivity optical sensor array and convert them into a high-bit-depth raw digital video stream. The video acquisition device integrates a lens control module that supports real-time fine-tuning of focus and exposure parameters. Specifically, the video acquisition device includes a multi-layer optical lens group and a stepper motor coupled to it. The stepper motor is controlled by a front-end controller and can automatically adjust the focal length according to changes in scene depth. The optical sensor array adopts a back-illuminated complementary metal-oxide-semiconductor structure, which can maintain the signal-to-noise ratio of the image in a wide dynamic range environment.

[0053] The edge intelligent processing center, physically connected to the video acquisition device via a high-speed serial interface, is configured to run complex visual attention analysis logic. This edge intelligent processing center is equipped with an edge-side AI computing chip, which possesses multi-core parallel processing capabilities for real-time semantic parsing of the raw digital video stream. The edge intelligent processing center extracts spatial and temporal motion features from the image sequence by running a lightweight convolutional neural network. The spatial features are calculated based on the image's gradient distribution, color contrast, and texture complexity.

[0054] When extracting spatial features, the edge intelligent processing center specifically uses the Sobel operator to calculate the gradient magnitude distribution of the image and quantifies the texture complexity through the pixel variance within a local window. For the input raw digital video stream... Frame luminance channel image Its pixel Spatial gradient magnitude at Calculated using the following formula:

[0055]

[0056] in, This represents the spatial gradient magnitude feature of the pixel. This represents the grayscale value of the pixel. This represents the two-dimensional spatial coordinates of the image. Further, when extracting temporal motion features, the edge intelligent processing center employs a block matching algorithm, obtaining the motion displacement vector by calculating the sum of absolute errors (SAD) between the current frame macroblock and the reference frame macroblock. For the top-left corner coordinates of the current frame... Size is The macroblock, whose offset vector is in the reference frame is The absolute error of the candidate macroblocks and Calculated using the following formula:

[0057]

[0058] in, Indicates the sum of absolute errors. and These represent the width and height of the macroblock in pixels, respectively. Represents the candidate motion displacement vector. and These represent the image pixel matrices of the current frame and the previous frame, respectively. The search is performed by traversing all pixels within a preset search range. , will make smallest The temporal motion characteristics of the macroblock are output to the non-uniform coding transmission unit.

[0059] The temporal motion features are obtained through optical flow field analysis or block matching algorithms between consecutive frames. Based on these features, the edge intelligent processing center can automatically identify regions of interest and non-interested background regions in the monitoring screen, and output the coordinate range, target category, and motion trajectory of the regions of interest in real time.

[0060] The non-uniform coding transmission unit, connected to the edge intelligent processing center via an internal bus, is configured to dynamically adjust the quantization parameter allocation strategy for video encoding based on the coordinate information and importance level of the region of interest. The non-uniform coding transmission unit includes a multi-rate control encoder, which breaks the limitation of uniform distribution of macroblock quantization parameters in traditional video encoding. For image blocks marked as regions of interest by the edge intelligent processing center, the non-uniform coding transmission unit employs a preset high-resolution coding configuration, i.e., allocating small quantization steps to preserve fine texture details and edge information.

[0061] The non-uniform coding transmission unit constructs a quantization parameter mapping model based on spatial location weights when executing the quantization parameter allocation strategy. Specifically, for any coded macroblock in a video frame... The final quantization parameters assigned to it Its regional level weight It exhibits an inverse linear relationship. This quantization parameter Calculated dynamically using the following formula:

[0062]

[0063] in, Indicates the first The actual quantization parameters used by each macroblock This represents the basic quantization parameter set for non-interested background areas (with a value range of 35 to 51). The step size for adjusting the quantization parameters of the region of interest (default is a fixed constant of 20). This represents the spatial weighting coefficient of the macroblock. When the macroblock... When completely within the highest-level region of interest, ,at this time Reaching the minimum value (i.e., a small quantization step size, preserving high texture detail); when macroblock When completely within a non-interesting background area,

[0064] ,at this time The low-rate profile-preserving coding configuration is performed. The non-uniform coding transmission unit utilizes the calculated... Subsequent video frames are subjected to non-uniform compression to form a compressed bitstream, which is then sent to the adaptive dual-channel transmission unit.

[0065] For non-interested background regions, a preset low bit rate profile is used to maintain the encoding configuration, that is, to increase its quantization step size, or even to retain only its low-frequency DC component in extremely low bandwidth environments, forming a compressed bitstream with highly non-uniform information density in the spatial domain.

[0066] The remote control terminal, deployed in a monitoring center or on a mobile control device, is configured to provide operators with a high-performance interactive interface. This terminal receives non-uniformly compressed video streams via a network and decodes and renders them using a high-performance graphics processor, displaying the monitoring footage in real time. Simultaneously, the remote control terminal integrates various physical command acquisition sensors, including a 3D joystick, high-precision knobs, and a multi-touch panel, for acquiring physical control commands input by the operator. These physical control commands precisely define the rotation angle, pitch, and roll parameters of the remote camera's pan / tilt unit, as well as the zoom, focus, and aperture adjustment parameters of the lens.

[0067] The instruction prediction and execution mechanism, connected to both the remote control terminal and the PTZ drive component via low-latency signaling channels, is configured to store and run historical operation habit models. Internally, this mechanism includes non-volatile storage to record the operator's search patterns, tracking habits, and reaction characteristics developed over long-term work. When the instruction prediction and execution mechanism receives a physical control command from the remote control terminal, it does not immediately wait for video feedback. Instead, it performs feedforward calculations based on the historical operation habit model to generate a local, immediate execution command and the corresponding expected motion vector. This design allows the system to anticipate the operator's intentions and initiate a local action sequence before physical network round-trip latency occurs.

[0068] The personalized operation feature vector constructed internally by the instruction prediction and execution mechanism It includes operation sequence features in the time dimension. In one specific embodiment, a first-order linear autoregressive model combined with historical operation trajectories is used for feedforward estimation to generate the expected motion vector. The expected motion vector Calculated using the following formula:

[0069]

[0070] in, This represents the predicted motion vector for the next moment (including horizontal yaw rate and vertical pitch rate). Indicates the length of the historical time window (default is 5 frame instruction cycles). This represents the number of times the least squares method was learned offline. Autoregressive weight coefficients, Indicates the number of data collected from the remote control terminal. The actual physical control command vector at any given moment. This represents the random error compensation term. Further, after generating the expected motion vector, the command prediction actuator combines the current real-time angular acceleration data measured by the multi-axis gyroscope. The mechanical inertia overshoot is compensated and corrected, and the final local real-time execution command is generated and sent to the gimbal drive component. This instruction is calculated using the following damping compensation formula:

[0071]

[0072] in, This indicates that the command is executed locally on the spot. This represents the preset reverse damping diagonal matrix (its diagonal elements are obtained based on the gimbal's mechanical inertia calibration). This represents the real-time angular acceleration vector obtained from the accelerometer. This is achieved by applying a reverse damping term. To offset the mechanical momentum in advance, It is directly sent to the gimbal drive component to eliminate mechanical response lag.

[0073] The gimbal drive component, physically connected to the command prediction and execution mechanism, is configured to receive the local real-time execution command and drive the camera's mechanical structure to perform real-time, high-precision physical movements. The gimbal drive component includes a high-torque brushless DC motor, a precision reducer, and a high-resolution absolute encoder. The absolute encoder can detect the physical position information of the gimbal in real time at a frequency far exceeding the video frame rate and feed this physical position information back to the command prediction and execution mechanism, forming a closed-loop motion control logic. In this way, the camera can respond to commands in a very short time, reducing mechanical response lag.

[0074] The network state awareness unit is configured to monitor the end-to-end communication quality of the data transmission link in real time. Its monitoring parameters include real-time available bandwidth utilization, packet loss rate, round-trip time, and jitter. The network state awareness unit assesses the current physical link's carrying capacity in real time by embedding probe packets at the transport layer and statistically analyzing feedback acknowledgments from the receiving end. The monitoring results are fed back in real time to the non-uniform coding transmission unit and the adaptive dual-channel transmission unit, serving as a basis for adjusting data coding strategies and command transmission priorities.

[0075] The virtual window synthesis device, connected to the instruction prediction execution mechanism via a high-speed data interface, is configured to synthesize a predicted image frame using the background frame information from the previous time sequence and the expected motion vector. The virtual window synthesis device employs geometric transformation logic to perform coordinate translation, rotation, and radial transformation on the background pixels in the historical frame according to the expected motion vector. This virtual window synthesis device utilizes pixel compensation logic to perform texture synthesis or fill in missing areas caused by motion using reference neighboring pixels. The synthesized predicted image frame does not depend on the remotely transmitted real video stream but is based on virtual feedback generated locally. This predicted frame is preferentially pushed to the remote control terminal for pre-rendering.

[0076] The virtual window synthesis device predicts the expected motion vector (including the horizontal rotation angle) output by the actuator according to the instructions. and vertical pitch angle Construct a two-dimensional affine transformation matrix for background pixel coordinate mapping. The affine transformation matrix Defined by the following formula:

[0077]

[0078] in, express The affine transformation matrix, and These represent the horizontal rotation angle and pitch angle obtained from the expected motion vector decomposition, respectively. This represents the translational shift of the image center due to rotation. When synthesizing the predicted image frame, it is used for any source pixel coordinates in the previous time-series background frame.

[0079] The virtual window synthesis device calculates the target coordinates in the prediction frame using matrix multiplication, as shown in the following formula:

[0080]

[0081] in, This represents the homogeneous coordinate vector of the target pixel in the predicted image frame. This represents the homogeneous coordinate vector of the source pixel in the previous time-series background frame. When the calculated... When the coordinates are non-integer coordinates or fall within visual gaps caused by motion, the virtual window synthesis device uses a bilinear interpolation algorithm to extrapolate and fill edge pixels. pixel value at Calculated using the following formula:

[0082]

[0083] in, Indicates the predicted image frame in floating-point coordinates The pixel values ​​after interpolation and synthesis. , , , Representing coordinates The known pixel values ​​of the four nearest neighbor integer coordinates, and These represent the decimal parts of the floating-point coordinates. The predicted image frames generated by the above logic are preferentially transmitted to the high-precision rendering engine of the remote control terminal.

[0084] The adaptive dual-channel transmission unit is configured to construct a unique embedded control slice structure. Within the transport layer protocol stack, this unit directly embeds high-priority control state synchronization signals into the macroblock free space or inter-frame space of the video transmission message. Through this deep coupling, control signaling is no longer subject to application layer queue blocking. Based on feedback from the network state awareness unit, the adaptive dual-channel transmission unit dynamically switches the transmission weights of the video data stream and the control command stream. When bandwidth is limited, this unit can ensure that control commands have absolute bandwidth priority by compressing the video channel bandwidth.

[0085] The adaptive dual-channel transmission unit, when constructing the embedded control slice structure, achieves transmission priority switching by building a dynamic weight allocation model based on a network state penalty function. In one specific embodiment, the transmission bandwidth of the control command channel is weighted. The following formula is used to calculate the parameters based on the real-time parameters fed back by the network state awareness unit:

[0086]

[0087] in, This indicates the proportion of total available bandwidth occupied by the control command flow. This represents the packet loss rate monitored in real time by the network status awareness unit. This indicates the preset security threshold (e.g., 5%).

[0088] This indicates the real-time available bandwidth utilization. Indicates the total bandwidth capacity of the link. This represents the base value for packet loss penalty (default is 0.5). This represents the exponential amplification factor for packet loss rate (default value is 10). This represents the bandwidth usage penalty factor (default is 0.3). When packet loss rate is detected... Exceeding the security threshold At that time, the exponent term The sharp increase forces the weight It quickly approaches the maximum value of 1. At this point, the adaptive dual-channel transmission unit executes data stream truncation control, stops transmitting non-critical video frames, and retains only the highest priority PTZ control commands and their status signaling to ensure the absolute smooth operation of the control link in a narrowband environment.

[0089] The system management server, connected to all the aforementioned components via a local area network (LAN) or wide area network (WAN), is configured to coordinate the communication synchronization of the entire system. This server manages all user access permissions and device configuration information, and is responsible for the structured storage of monitoring data semantically annotated at the edge. The system management server also integrates fault diagnosis logic, enabling real-time monitoring of the operational status of each hardware module.

[0090] The edge intelligent processing center employs an advanced multi-scale feature fusion strategy when identifying the region of interest. This strategy accurately captures edges and subtle textures in the image by calculating the gradient change rate at the pixel level.

[0091] By calculating macroblock-level motion displacement vectors, objects in the image that are displaced relative to the background are identified. The edge intelligent processing center is pre-configured with a target classification operator that can automatically identify moving targets such as people, vehicles, or operators as the highest-level regions of interest. This determination logic ensures that the system can concentrate limited computing resources on the most valuable image areas.

[0092] Furthermore, the non-uniform coding transmission unit performs an extreme bitrate reduction operation when allocating quantization parameters. For regions determined to be non-interesting background areas, the unit increases the quantization step size by reducing the sampling rate of their chroma components. This operation reduces the proportion of background data stream while ensuring that the background outline remains largely discernible. The saved bandwidth resources are fully reallocated to the texture detail encoding of the region of interest, thereby improving the sharpness of key targets while maintaining the same total bandwidth.

[0093] The remote control terminal includes a high-precision rendering engine optimized for low-latency display. This high-precision rendering engine supports a sophisticated dual-buffering mechanism, enabling parallel processing of two independent data streams: one from predicted image frames generated by the virtual window compositing device, and the other from real video frames transmitted and decoded by the adaptive dual-channel transmission unit. The rendering engine employs a smooth weighting algorithm, based on the arrival timestamps and spatial overlap of the two streams.

[0094] When the operator issues a rapid rotation command, the terminal first displays a predicted image to provide immediate visual feedback. Then, after the actual video frames arrive, it gradually transitions to the real image through pixel-level blending technology.

[0095] The instruction prediction and execution mechanism integrates deep learning-based user behavior learning logic. This logic not only records the operator's physical commands but also analyzes the operator's operational trajectory, click frequency, and control habits in different monitoring scenarios through statistical distribution characteristics. For example, the system can identify the sinusoidal scanning path that the operator typically uses when searching for a target. By constructing personalized operational feature vectors, the mechanism can improve the accuracy of predicting physical control commands, ensuring that the generated expected motion vector closely matches the operator's psychological expectations.

[0096] At the specific image processing level, the virtual window synthesis device calculates the coordinate mapping relationship of the image in three-dimensional space using an affine transformation matrix when synthesizing predicted image frames. To address the complex field-of-view changes caused by gimbal rotation, the system tiles and stretches historical background pixels and uses edge pixel extrapolation technology to fill visual gaps caused by motion. For trapezoidal distortion caused by gimbal pitch, the device applies perspective projection transformation for correction, ensuring the predicted image is geometrically accurate and avoiding visual errors for the operator.

[0097] The network status awareness unit has millisecond-level response capability. When the network packet loss rate exceeds a preset security threshold, the network status awareness unit immediately sends an emergency mode trigger signal to the adaptive dual-channel transmission unit. This mechanism constitutes the first line of robust defense for the system, preventing drastic network fluctuations from causing control logic paralysis.

[0098] Upon receiving the emergency mode trigger signal, the adaptive dual-channel transmission unit immediately executes strict data stream truncation control. The system temporarily stops transmitting non-critical video frame data that consumes bandwidth, i.e., temporarily discarding bidirectional prediction frames and prediction frames. At this time, the transmission channel only retains the differential information of critical frames, and allocates all remaining bandwidth to the highest priority PTZ control commands and their status feedback signaling. This strategy ensures that even in a narrowband environment where the network is almost disconnected, the operator can still maintain absolute control over the camera position.

[0099] The system management server also includes an intelligent load balancer. When multiple remote control terminals concurrently access the same edge processing center, the intelligent load balancer dynamically allocates computing resources to the edge intelligent processing center based on the geographical location, network latency, and processing performance of each terminal. For example, for terminals with poor network quality, the load balancer will request the edge center to provide a lower-resolution ROI slice to ensure the continuity of interaction.

[0100] The video acquisition device is also equipped with a high-precision ambient light sensor. The ambient light sensor detects environmental illumination information, such as the current lux value, and feeds it back to the edge intelligent processing center in real time. The edge center then adjusts the contrast threshold and noise reduction intensity in the visual attention analysis logic accordingly. In low-light night mode, the system automatically reduces the analysis weight of background areas, enhancing the sensitivity to identify targets with self-illuminating or infrared reflective features.

[0101] The gimbal drive assembly employs a high-resolution photoelectric encoder for fully closed-loop control. The position feedback frequency of this photoelectric encoder is set to be more than twice the preset video frame rate. This high-frequency feedback mechanism ensures that the mechanical action execution accuracy of the camera maintains a very high consistency with the expected motion vector generated by the command prediction execution mechanism, eliminating overshoot or oscillation caused by mechanical inertia.

[0102] Furthermore, the non-uniform coding transmission unit also supports a time-based optimization technique, namely, dynamic frame rate adjustment based on the importance of the region of interest. For background regions that remain static for extended periods, the system reduces their transmission frame rate or even lowers it.

[0103] For regions of interest containing high-speed moving targets, a preset maximum frame rate is used for data acquisition, encoding, and transmission. This spatiotemporal combined non-uniform strategy further squeezes the limits of physical bandwidth.

[0104] When generating the expected motion vector, the command prediction execution mechanism not only refers to the operation command but also incorporates data from the multi-axis gyroscope and accelerometer integrated within the camera. By acquiring real-time angular velocity and angular acceleration, the system can compensate for motion overshoot caused by mechanical inertia in real time. When the gimbal rotates at high speed and suddenly stops, the system automatically applies a reverse damping prediction vector to counteract mechanical momentum and ensure the stability of the prediction window.

[0105] The adaptive dual-channel transmission unit defines an eight-bit signaling priority field within its custom protocol stack. This signaling priority field is configured to be recognizable by commercial routers and switches along the network path. By configuring differentiated service code points, control signaling can enter the highest-priority forwarding queue when passing through network nodes, reducing instruction queuing delays caused by network congestion.

[0106] The remote control terminal's interface features augmented reality assistance. Based on the coordinates of the region of interest transmitted from the edge intelligent processing center, the interface automatically overlays dynamic augmented reality guidance markers, such as red tracking boxes or yellow warning icons, onto the video feed. This helps operators quickly locate potential threat targets in complex, multi-target backgrounds.

[0107] The virtual window synthesis device exhibits strong predictive capabilities when processing zoom commands. When it detects an operator rapidly pushing the zoom lever, the device performs rapid digital interpolation calculations on the pixels in the central region of the image, such as using a bilinear interpolation algorithm. Before the actual optical zoom mechanism of the lens has fully engaged, it presents the operator with a digitally magnified visual preview. As the actual optical magnification increases, the system smoothly replaces the digitally magnified image with a high-resolution optical image.

[0108] The edge intelligent processing center also features advanced abnormal behavior early warning capabilities. When the trajectory of a target within a region of interest matches preset abnormal patterns such as crossing boundaries, loitering, or running rapidly, the system automatically increases the coding importance level of that region of interest and simultaneously triggers the audible and visual alarm device on the remote control terminal. This intelligent predictive mechanism transforms the monitoring system from a passive recorder into a proactive security early warning provider.

[0109] The system management server meticulously records the timestamp of each physical control command sent, the timestamp of command completion at the front end, and real-time network status parameters at the time the command was issued. This data is stored in a structured database for subsequent offline big data analysis. By learning from millions of operation data points, the system management server can automatically optimize the parameter weights in the command prediction model.

[0110] After the network environment improves, the non-uniform coding transmission unit has an automatic recovery mechanism. When the network bandwidth returns to the preset normal range and the packet loss rate continues to decrease, the system will automatically reduce the quantization step size of the encoder, gradually restore the image clarity of non-interested background areas, and return to full-frame monitoring status.

[0111] The gimbal drive component integrates hardware-level overload protection logic. When the mechanical resistance on the gimbal motor rotor exceeds the preset safe torque threshold, such as due to foreign object obstruction, the driver will automatically cut off the power supply and enter a protection state, while simultaneously reporting a fault code to the remote control terminal to prevent motor burnout or permanent damage to the mechanical structure.

[0112] The virtual window synthesis device possesses geographic information system (GIS) fusion capabilities. This device can combine the camera's current latitude and longitude coordinates, orientation angle, and a geographic information database to embed virtual information such as surrounding terrain, landforms, building outlines, and underground pipe networks into the predicted image frame. This enhances the operator's remote spatial perception capability during blind operation, enabling them to clearly perceive the physical location relationship of the monitored target relative to its surrounding geographical environment.

[0113] Example 2: As an alternative implementation architecture of the system of the present invention, this example describes a distributed remote real-time monitoring and control system based on the collaborative operation of a cloud computing cluster and an edge gateway. The functions of the edge intelligent processing center, originally integrated at the camera end, are split into two layers: near-end preliminary filtering and cloud-based deep analysis.

[0114] A distributed remote real-time monitoring and control system includes an edge acquisition gateway, a cloud-based large-scale computing cluster, a multi-path adaptive transmission network, a lightweight operation terminal, a feedforward control compensation engine, and a distributed system controller.

[0115] The edge acquisition gateway is deployed at the camera side and is responsible for preliminary noise reduction and feature pre-extraction of multiple video signals. Compared to the previous embodiments, the edge acquisition gateway is more streamlined in hardware and focuses on data stream convergence and preliminary localization. It is configured to calculate the local entropy distribution of the image, initially identify moving targets in the scene, and assign them preliminary semantic labels.

[0116] The cloud-based large-scale computing cluster is connected to the edge acquisition gateway via a backbone network. This cluster consists of multiple servers equipped with high-performance graphics processing units (GPUs) and tensor processors. The cluster runs deep residual networks and long short-term memory networks to perform in-depth visual attention analysis on the primary features uploaded by the edge gateway. The cloud cluster can simultaneously process semantic parsing tasks from thousands of video streams and generate refined region-of-interest coordinates.

[0117] The multipath adaptive transmission network is configured to establish multiple parallel virtual transmission links between the edge gateway, cloud cluster, and operating terminal. This multipath adaptive transmission network employs a multipath transmission control protocol and can simultaneously utilize 5G mobile communication technology, fiber optic access, and low-Earth orbit satellite links. The network status awareness unit is deployed at each transmission node to measure the physical performance indicators of each path in real time.

[0118] The feedforward control compensation engine employs a distributed prediction algorithm. When an operator initiates a control command, the command is first parsed by the local feedforward engine to generate a local virtual feedback screen; the command is then quickly pushed to the cloud and edge gateway, initiating a multi-level linked physical response. This feedforward control compensation engine can dynamically adjust the prediction compensation time window based on the latency characteristics of different links.

[0119] In this embodiment, the non-uniform coding transmission unit employs a cross-layer optimization strategy. The quantization weight matrix generated by the cloud computing cluster is sent to the edge acquisition gateway via the reverse control link. After receiving the weight matrix, the gateway applies different compression ratios to each macroblock of the video stream. For regions of interest with high security threats, the cloud instructs the gateway to enable lossless encoding mode or use an extremely low compression ratio.

[0120] The lightweight operating terminal is configured as a browser-based thin client, primarily responsible for command acquisition and virtual screen rendering. Because some of the core synthesis logic is moved to the cloud or edge gateway, the lightweight terminal's requirements for local computing resources are further reduced, enabling the system to run stably on ordinary mobile devices.

[0121] The distributed system controller is responsible for signaling synchronization and state maintenance throughout the distributed architecture. When an edge gateway fails, the controller can quickly migrate tasks to a nearby gateway or have them taken over directly by the cloud, ensuring high system availability.

[0122] In terms of instruction prediction logic, this embodiment introduces group behavior learning. The system controller aggregates a large amount of operational data from different operators on the same type of target, forming a standardized control habit library. Even for novice operators, the system can provide the most likely prediction window and motion compensation based on group experience.

[0123] The virtual window compositing device employs a multi-level rendering mode within a distributed architecture. The local terminal handles simple geometric translation rendering, providing initial feedback with extremely low latency;

[0124] The edge gateway or cloud is responsible for high-quality texture composition and occlusion culling, and then sends back higher-quality predicted frames at a later time. This tiered rendering strategy achieves a better balance between latency and image quality.

[0125] Example 3: This example describes a highly reliable remote real-time monitoring and control system for deployment in extremely harsh environments. In its hardware implementation, this highly reliable remote real-time monitoring and control system emphasizes redundant design and physical protection; in its control logic, it emphasizes anti-interference capabilities and link self-healing ability.

[0126] A highly reliable remote real-time monitoring and control system includes a ruggedized visual acquisition unit, a dual-redundant edge processing chassis, a self-healing communication protocol module, a special gimbal drive system, and a redundant command center.

[0127] The ruggedized visual acquisition unit is equipped with a self-cleaning protective cover and an internal active heat dissipation and defogging system. Its optical sensor features a radiation-resistant design, enabling it to output stable digital video signals even in environments with strong electromagnetic interference.

[0128] The dual-redundant edge processing chassis contains two sets of fully symmetrical intelligent processing boards. The two sets operate in a primary / backup mode, synchronizing visual analysis status and region-of-interest tracking parameters in real time via heartbeat messages. In the event of a hardware failure on the mainboard, the backup board can seamlessly take over, ensuring uninterrupted monitoring.

[0129] The self-healing communication protocol module is configured to run an adaptive error correction algorithm based on erasure coding technology. For unstable wireless links, this self-healing communication protocol module not only embeds control signals at the transport layer but also increases the proportion of redundant parity bits, enabling it to fully recover critical control signaling and video data of the region of interest through forward error correction.

[0130] The specialized gimbal drive system employs a dual rotary transformer structure, providing higher physical shock resistance and environmental tolerance than ordinary photoelectric encoders. Its motor drive circuit features comprehensive current and voltage monitoring logic, enabling real-time compensation for power fluctuations.

[0131] The redundant command centers are deployed in two different geographical locations. When one center is interrupted due to a disaster, the system management server can automatically identify the change in network topology and redirect video streams and control links to the other command center. Simultaneously, the system's load balancer will automatically adjust the data distribution strategy, prioritizing connectivity for monitoring points around the disaster-stricken area.

[0132] With this high-reliability configuration, the virtual window synthesis device is granted higher privileges. When the communication link is completely interrupted, the virtual window synthesis device can provide the operator with a physically simulated virtual monitoring environment based on the background information acquired at the last moment and the predicted trajectory of the target. The operator can continue to perform search and deployment operations in the virtual environment. After the link is restored, the system will automatically compare the virtual trajectory with the real trajectory and make synchronous corrections.

[0133] The non-uniform coding unit in this embodiment also supports hotspot encryption. Before data transmission, the system performs high-strength asymmetric encryption only on the region of interest identified by the edge processing center, while weakly encrypting or not encrypting the background region. This ensures the privacy and security of key monitoring targets while reducing the overall encryption and decryption computational overhead of the system.

[0134] Example 4: This example describes a remote real-time monitoring and control system based on multi-sensor fusion and augmented reality guidance. In addition to utilizing visual signals, this remote real-time monitoring and control system integrates lidar and acoustic array detectors.

[0135] A multi-dimensional perception real-time monitoring and control system includes a multimodal sensor array, a fusion perception processing unit, a three-dimensional virtual window synthesis device, and an augmented reality interactive terminal.

[0136] The multimodal sensor array consists of a high-resolution camera, a single-line or multi-line LiDAR, and a microphone array. The LiDAR is configured to acquire three-dimensional point cloud data of the monitored area, while the microphone array is used to locate the direction of the sound source.

[0137] The fusion sensing processing unit is configured to perform cross-modal feature alignment. This unit correlates the 3D target detected by the LiDAR with 2D pixel blocks in the visual image to obtain the target's precise spatial coordinates and physical dimensions. This multi-dimensional semantic information is fed back to the non-uniform coding transmission unit, enabling the system to allocate coding weights based on the target's actual physical distance.

[0138] The described 3D virtual window synthesis device utilizes a point cloud model acquired by LiDAR as a background reference. When the operator rotates the pan-tilt unit, the synthesis of the predicted image no longer relies solely on the geometric transformation of two-dimensional pixels, but rather on a reprojection based on the 3D scene. This 3D model-based predicted frame exhibits higher perspective accuracy and more realistic occlusion effects.

[0139] The augmented reality interactive terminal is configured to overlay the sound source location, radar ranging information, and visually identified target attributes onto the monitoring screen in real time. When the system detects an abnormal sound source outside the field of view, the augmented reality interface will automatically display a guide arrow pointing in the direction of the sound source, prompting the operator to rotate the pan-tilt unit to investigate.

[0140] In this multimodal environment, the predictive model of the command prediction execution mechanism incorporates spatial dynamic constraints. The system can identify the target being tracked by the operator as a physical entity with acceleration capabilities. Combining precise displacement data provided by the lidar, the predictive model can calculate the target's most likely physical position at the next moment and automatically adjust the gimbal's motion trend.

[0141] The adaptive dual-channel transmission unit dynamically prunes data based on its real-time value when processing multimodal data. For example, when the visual signal quality deteriorates, the system automatically increases the transmission ratio of LiDAR point cloud data, using the point cloud contours to maintain the operator's basic spatial perception.

[0142] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above descriptions are merely specific embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention. Without departing from the concept of the present invention, those skilled in the art can make various free combinations among the various features of the present invention, and all of these fall within the protection scope of the present invention.

[0143] The various modules of this invention's system can be implemented using hardware circuits, software functional units, or a combination thereof, and their physical locations can be merged or further split as needed during actual deployment. System performance parameters, such as encoding quantization step size, perception threshold, and prediction time window, can be dynamically configured and optimized based on actual hardware computing power and network environment; this flexibility in configuration is one of the key features of this invention. This system, through the deep integration of visual attention mechanisms, feedforward prediction logic, and adaptive transmission mechanisms, solves the long-standing problems of interaction lag and resource contention in the field of remote monitoring, and has high engineering application value.

Claims

1. A remote real-time monitoring and control system based on video surveillance cameras, characterized in that, include: Video acquisition devices are used to capture continuous light signals in the monitored area and convert them into raw digital video streams; The edge intelligent processing center, connected to the video acquisition device, is used to run visual attention analysis logic to identify regions of interest and non-interest background regions in the monitoring screen. A non-uniform coding transmission unit, connected to the edge intelligent processing center, is used to perform non-uniformly distributed compression coding on the region of interest and the non-interest background region according to the information of the region of interest to form a compressed bitstream; Remote control terminal, used to display screens and collect physical control commands; The instruction prediction and execution mechanism is connected to the remote control terminal and the gimbal drive component, respectively, and is used to perform feedforward calculations based on the historical operation habit model to generate local real-time execution instructions and expected motion vectors. The gimbal drive component is used to receive the local real-time execution command and drive the mechanical structure of the camera to move. The network status awareness unit is used to monitor the quality parameters of the data transmission link in real time. A virtual window synthesis device, connected to the instruction prediction execution mechanism, is used to synthesize a predicted image frame using the previous time sequence information and the expected motion vector; An adaptive dual-channel transmission unit is used to construct an embedded control slice structure and dynamically switch data transmission weights; The system management server is used to coordinate component communication synchronization and manage data.

2. The remote real-time monitoring and control system based on a video surveillance camera according to claim 1, characterized in that, When identifying the region of interest, the edge intelligent processing center is configured to adopt a multi-scale feature fusion strategy, and extract spatial features and temporal motion features from the image sequence by running a lightweight convolutional neural network through an integrated edge-side artificial intelligence computing chip. The spatial domain features are calculated based on the gradient distribution, color contrast, and texture complexity of the image, while the temporal motion features are obtained through optical flow field analysis or block matching algorithms between consecutive frames. The edge intelligent processing center is further configured with a target classification operator, which is used to calculate the gradient change rate at the pixel level and the motion displacement vector at the macroblock level to determine the boundary range of the target object, and to identify human bodies, vehicles or preset targets in motion as the highest level region of interest.

3. The remote real-time monitoring and control system based on a video surveillance camera according to claim 1, characterized in that, The non-uniform encoding transmission unit includes a multi-rate control encoder, which is used to execute a quantization parameter allocation strategy based on the coordinate information and importance level output by the edge intelligent processing center. When processing the region of interest, the multirate control encoder assigns a preset small quantization step size to preserve texture details and edge information; When processing the non-interested background region, the multi-rate control encoder reduces the sampling rate of the chroma component and increases its quantization step size until only the low-frequency DC component is retained, thereby reducing the proportion of the background data bitstream. The non-uniform encoding transmission unit also supports dynamic frame rate adjustment based on the time dimension. It uses a low frame rate for static background areas, while using a preset maximum frame rate for data acquisition and encoding for regions of interest with high-speed moving targets.

4. The remote real-time monitoring and control system based on a video surveillance camera according to claim 1, characterized in that, The remote control terminal includes a high-precision rendering engine that supports a dual-buffering mechanism, used to simultaneously receive the predicted image frames generated by the virtual window synthesis device and the real video frames transmitted back by the adaptive dual-channel transmission unit. The rendering engine runs a smooth weighted algorithm, which uses pixel-level blending technology to achieve a seamless and smooth switch between the predicted image and the real image based on the arrival timestamps and spatial overlap of the two. The interactive interface of the remote control terminal also has augmented reality assistance function, which can automatically overlay dynamic augmented reality guidance marks, including tracking boxes or warning icons, on the video screen according to the coordinates returned by the edge intelligent processing center, to help the operator lock potential threat targets. The remote control terminal also integrates a three-dimensional joystick, a high-precision knob, and a multi-touch panel, used to collect physical control commands including gimbal rotation angle, pitch status, roll parameters, and lens zoom parameters.

5. The remote real-time monitoring and control system based on a video surveillance camera according to claim 1, characterized in that, The instruction prediction and execution mechanism integrates a deep learning-based user behavior learning logic. This user behavior learning logic records the operator's operation trajectory, click frequency, and control habits in different monitoring scenarios through a non-volatile storage unit, and uses statistical distribution features to construct a personalized operation feature vector. When a physical control command is received, the command prediction actuator uses the operation feature vector to perform feedforward calculations to predict the operator's intention and generate a matching expected motion vector. When generating the expected motion vector, the instruction prediction execution mechanism further combines the angular velocity and angular acceleration data provided by the multi-axis gyroscope and accelerometer integrated inside the camera to compensate for and correct the motion overshoot caused by mechanical inertia. By applying a reverse damping prediction vector, the mechanical momentum is offset to ensure the stability of the prediction window during the process of the gimbal rotating at high speed and stopping.

6. The remote real-time monitoring and control system based on a video surveillance camera according to claim 1, characterized in that, The gimbal drive assembly includes a high-torque brushless DC motor, a precision reducer, and a high-resolution absolute photoelectric encoder. The photoelectric encoder is configured to detect the physical position information of the gimbal in real time at a frequency twice that of a preset video frame rate, and feed the physical position information back to the command prediction and execution mechanism to form a fully closed-loop motion control logic. The gimbal drive component also integrates hardware-level overload protection logic to monitor the mechanical resistance of the gimbal motor rotor in real time. When the mechanical resistance exceeds the preset safe torque threshold, the driver automatically cuts off the power supply and enters the protection state, and reports the fault code to the remote control terminal. The execution accuracy of the gimbal drive component is highly consistent with the expected motion vector generated by the instruction prediction execution mechanism to eliminate mechanical response lag.

7. The remote real-time monitoring and control system based on a video surveillance camera according to claim 1, characterized in that, The virtual window synthesis device is equipped with geometric transformation logic and pixel compensation logic, which are used to calculate the coordinate mapping relationship of the image in three-dimensional space by using the background frame information of the previous time sequence and combining the affine transformation matrix. To address the changes in the field of view caused by gimbal rotation, the virtual window synthesis device performs tiling and stretching of historical background pixels and applies edge pixel extrapolation technology to fill the visual gaps caused by motion. To correct the trapezoidal distortion caused by gimbal pitch, the virtual window synthesis device uses perspective projection transformation. When processing zoom instructions, the virtual window synthesis device performs bilinear interpolation on the pixels in the central region to present a digitally magnified visual preview effect before the lens optical zoom structure is in place. The virtual window synthesis device has geographic information system fusion capabilities, and can combine the latitude and longitude coordinates and orientation angle of the camera to embed virtual information of the surrounding terrain, landforms and building outlines into the predicted image frame.

8. The remote real-time monitoring and control system based on a video surveillance camera according to claim 1, characterized in that, The network status awareness unit is used to monitor communication quality parameters including real-time available bandwidth utilization, packet loss rate, round-trip time, and jitter value. When the data packet loss rate is detected to exceed the 5% security threshold, an emergency mode trigger signal is immediately sent to the adaptive dual-channel transmission unit. After receiving the emergency mode trigger signal, the adaptive dual-channel transmission unit executes data stream truncation control, stops transmitting bidirectional prediction frames and prediction frames in non-critical video frames, retains only the differential information of critical frames, and allocates all remaining bandwidth to the highest priority PTZ control commands. The adaptive dual-channel transmission unit defines an eight-bit signaling priority field in the transmission protocol stack. This signaling priority field is configured to be recognizable by switching devices on the network path, and differentiated service code points are used to ensure that control signaling enters the highest priority forwarding queue.

9. The remote real-time monitoring and control system based on a video surveillance camera according to claim 1, characterized in that, The system management server contains an intelligent load balancer, which dynamically allocates computing resources of the edge intelligent processing center based on the geographical location, network latency, and processing performance of each terminal when multiple remote control terminals access the system concurrently. The system management server is also responsible for the structured storage of the semantically labeled monitoring data, and for recording in detail the timestamp of each physical control command sent, the timestamp of execution completed, and the real-time network status parameters when the command was issued, for later offline big data analysis and model optimization. The system management server also integrates fault diagnosis logic to monitor the operating status of each hardware module in real time. The system management server supports signaling synchronization in a distributed architecture. When a local edge processing node fails, it automatically migrates tasks to a nearby node or a cloud computing cluster to ensure high system availability.

10. A remote real-time monitoring and control method based on a video surveillance camera, characterized in that, Remote real-time monitoring is achieved using the remote real-time monitoring and control system based on video surveillance cameras as described in any one of claims 1-9.

Images