Camera remote real-time control system and method based on video monitoring

Abstract

The invention discloses a camera remote real-time control system and a control method based on video monitoring, and relates to the technical field of camera control, in an image communication link, video and holder control cooperation is realized, a control symbol is embedded into a video code stream, and synchronous correction is carried out based on control and video double-link time delay estimation. The cloud platform motion and control intention is mapped into a coding structure and ROI bit distribution through visual control isomorphism, so that low-delay and high-definition transmission is still carried out in a key target area under network fluctuation, target loss and misoperation are reduced, and the monitoring disposal and evidence obtaining continuity of a chemical industrial park and an oil and gas storage tank area is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of camera control, and particularly to a remote real-time control system and method for cameras based on video surveillance. Background Art

[0002] A large number of pan-tilt cameras are usually deployed in high-risk places such as chemical industrial parks and oil and gas storage tank areas to monitor events such as perimeter intrusion, leakage emergency, personnel intrusion, and vehicle anomalies. The on-site cameras mostly transmit videos back through a wireless private network or a public network VPN, and the network bandwidth, latency, and packet loss will fluctuate over time. Existing remote monitoring systems generally adopt a method where the video encoding transmission link and the pan-tilt control signaling link are independent: on the video side, there is often a buffer for smooth playback. Although the control instruction side can be preferentially transmitted, the delays of the two links are not synchronized, resulting in the inconsistency between the picture seen by the operator and the actual posture of the camera, manifested as what is seen is not what is controlled, rotational lag, and easy loss of the target. At the same time, traditional coding strategies are mostly fixed bitrate or ROI coding, and cannot jointly incorporate the operator's control intention, the pan-tilt movement state, and the network state into the coding structure decision-making; in the scenario of multi-camera linkage, there is a lack of a control information bearing and end-to-end synchronization mechanism that coordinates with the image communication link, resulting in target loss, black screen, or perspective misalignment during switching, thus making it difficult to meet the requirements of industrial parks for real-time disposal and continuous evidence collection.

[0003] Currently, Chinese Patent No. CN202310408624.6 discloses a remote control system and control method for a camera. The remote control system for the camera includes a control end, a controlled end, and a cloud. The cloud is respectively connected to the control end and the controlled end; a remote control method for a camera first obtains head movement posture information by a Bluetooth attitude sensor and forms packet data. Secondly, the first Linux system host transmits the packet data to the second Linux system host through the cloud, and then transmits it to the STM32 single-chip microcomputer. Finally, the STM32 single-chip microcomputer unpacks the received packet data, and then performs attitude data calculation to obtain the movement angle value; the method of performing average value filtering on the movement angle value is used to obtain the secondary processed movement angle value, and thus outputs a pulse width modulation signal PWM to control the rotation angles of the X-axis servo and Y-axis servo of the two-degree-of-freedom servo pan-tilt.

[0004] The above technology is difficult to achieve the coordination of monitoring video transmission and remote camera control within the same image communication link under network fluctuations, so that the picture seen by the operation end is consistent with the actual pan-tilt posture in real time and maintains continuous tracking and seamless control of the target. Summary of the Invention

[0005] The technical problem solved by this invention is that existing technologies are unable to achieve coordinated monitoring video transmission and remote camera control within the same image communication link under network fluctuation conditions, so that the image seen by the operator is consistent with the actual pan-tilt attitude in real time and maintains continuous target tracking and seamless control.

[0006] To solve the above-mentioned technical problems, the present invention provides the following technical solution: A remote real-time control system based on video surveillance cameras includes a video control data acquisition module, a video control co-construction modeling module, a collaborative coding and transmission module, and a control output optimization module. The visual control data acquisition module is used to establish a real-time visual control dataset and acquire historical visual control datasets. The visual control isomorphic modeling module is used to train a visual control isomorphic coding model based on a historical visual control dataset and output a parameter set of the visual control isomorphic coding model. The co-coding transmission module is used to generate a coding structure parameter set based on the visual control isomorphic coding model parameter set and the real-time visual control dataset, and to co-code the video frame data according to the coding structure parameter set to obtain a video bitstream carrying control symbols and send it. The control output optimization module is used to parse control symbols from the video stream, output camera control commands, and estimate link delay based on the control command timestamp and video frame timestamp, and make synchronous adjustments.

[0007] Preferably, the visual control data acquisition module includes a video acquisition unit, an attitude acquisition unit, a control acquisition unit, and a network status acquisition unit; The video acquisition unit is used to acquire video frame data of the monitored scene; The attitude acquisition unit is used to acquire camera gimbal attitude data, which includes pitch angle data, azimuth angle data, and zoom value data. The control acquisition unit is used to acquire control command data issued by the operating terminal, including gimbal rotation commands and zoom commands. The network status acquisition unit is used to collect link network status data, which includes latency estimates, bandwidth estimates, and packet loss rate estimates. The visual control data acquisition module is used to time-align video frame data, gimbal attitude data, control command data, and link network status data to form a real-time visual control dataset and to acquire historical visual control datasets.

[0008] Preferably, the visual control isomorphic modeling module includes a structural feature extraction unit and an isomorphic mapping establishment unit; The structural feature extraction unit is used to extract coded structural features from video frame data based on historical visual control datasets. The coded structural features include reference frame dependencies, motion vector fields, and distribution of candidate blocks for regions of interest. It also extracts control structural features from historical gimbal attitude data and historical control command data. The control structural features include gimbal angular velocity sequences, zoom change sequences, and control command trigger time distribution. The isomorphic mapping establishment unit is used to train the visual-control isomorphic coding model based on the coding structure features and control structure features through a supervised learning regression algorithm, and to map the coding structure feature space to the control structure feature space, outputting a parameter set of the visual-control isomorphic coding model.

[0009] Preferably, the logic of the visual control isomorphic coding model is as follows: Based on the parameter set of the visual control isomorphic coding model and the preset rotation direction and angular velocity in the real-time gimbal control variables, the reference frame dependency is adjusted to obtain a set of reference frames consistent with the gimbal motion direction. Based on the parameter set of the visual control isomorphic coding model and the real-time zoom change, the distribution of candidate blocks of interest is scaled to determine the set of blocks of interest. Based on the visual control isomorphic coding model parameter set and the real-time PTZ angular velocity sequence and zoom change sequence, the search range and accuracy of the motion vector field are adjusted to generate a coding structure parameter set for the current coding period. The coding structure parameter set is used to constrain GOP partitioning, reference frame selection and region of interest block partitioning during video coding.

[0010] Preferably, the cooperative coding transmission module further includes an interest region determination unit and a cooperative coding execution unit; The region of interest determination unit is used to determine the region of interest blocks in the current video frame based on the real-time video control dataset and the coding structure parameter set, and to predict the region of interest blocks at the next moment, generating a set of region of interest blocks and a corresponding set of region of interest block identifiers. The collaborative coding execution unit is used to configure first coding parameters for regions of interest blocks and second coding parameters for non-regions of interest blocks according to the coding structure parameter set, wherein the bit rate of the first coding parameter is higher than the bit rate of the second coding parameter, to encode video frame data and generate a video bitstream carrying control symbols; The collaborative coding execution unit is used to encode the PTZ control command data and the set of region of interest block identifiers into control symbols and embed them into the image communication protocol header of the video stream.

[0011] Preferably, the collaborative coding and transmission module further includes a multi-camera relay decision unit; The multi-camera relay decision unit is used to predict the set of candidate cameras that the target may enter, based on the motion vector field of the target in the video frame data, the current camera gimbal attitude data, and the gimbal attitude data of the neighboring cameras in the real-time visual control dataset. The multi-camera relay decision unit is used to generate relay control command data for candidate cameras and write control symbols, so that the candidate cameras can pre-rotate and pre-zoom according to the relay control command data, and mark the relay camera identifier in the video stream.

[0012] Preferably, the control output optimization module includes a control parsing unit, a time delay estimation unit, and a synchronization correction unit; The control parsing unit is used to parse control symbols from the received video stream, recover the PTZ control command data, and output the camera control command. The delay estimation unit is used to calculate the control link delay and video link delay based on the timestamp of the camera control command transmission, the timestamp of the gimbal attitude change, and the timestamp of the video frame decoding. The synchronization correction unit is used to adjust the decoding buffer queue and display output time according to the control link delay and video link delay, so that the video frame display time and the actual gimbal attitude of the camera are kept within a preset deviation threshold.

[0013] Preferably, the control output optimization module further includes a model update unit; The model update unit is used to collect camera control command execution result data and target tracking and holding time data, and uses the success rate of camera control command execution and the target holding ratio in the region of interest block as performance indicators, and combines them with link network status data to form performance evaluation results. The model update unit is used to update the parameter set of the visual control isomorphic coding model based on the performance evaluation results, and to adjust the generation strategy of the coding structure parameter set and the prediction method of the interest region block set.

[0014] Preferably, the cooperative coding transmission module embeds control symbols using an image communication protocol syntax extension method, and the control symbols include frame-level control symbols and block-level control symbols; The frame-level control symbol is used to indicate the PTZ control command data and candidate relay camera identifiers corresponding to the current video frame. The block-level control symbols are used to indicate the location of the region of interest block and the corresponding coding structure parameters; The decoding end parses frame-level control symbols and block-level control symbols while performing image decoding on the video bitstream.

[0015] A method for remote real-time control of cameras based on video surveillance, characterized by the following steps: Step S1: Establish a real-time visual control dataset and collect historical visual control datasets; Step S2: Train the visual control isomorphic coding model based on the historical visual control dataset, and output the parameter set of the visual control isomorphic coding model; Step S3: Generate a coding structure parameter set based on the visual control isomorphic coding model parameter set and the real-time visual control dataset, and co-code the video frame data according to the coding structure parameter set to obtain a video bitstream carrying control symbols and send it. Step S4: Parse control symbols from video stream to output camera control commands, and estimate link delay based on control command timestamps and video frame timestamps and make synchronous adjustments.

[0016] The beneficial effects of this invention are as follows: This invention achieves video and PTZ control coordination within the image communication link, embeds control symbols into the video bitstream, and performs synchronous correction based on the delay estimation of the control and video dual links to ensure that what is seen is what is controlled. By mapping the PTZ motion and control intention to the coding structure and ROI bit allocation through visual-control isomorphism, the key target area can still be transmitted with low latency and high definition under network fluctuations, reducing target loss and misoperation, and improving the continuity of monitoring, handling and evidence collection in chemical industrial parks and oil and gas storage tank areas. Attached Figure Description

[0017] Figure 1 A schematic diagram of the basic process of a remote real-time control system for video surveillance cameras provided in one embodiment of the present invention; Figure 2 A flowchart illustrating the steps of a remote real-time control method for a camera based on video surveillance, provided in one embodiment of the present invention. Detailed Implementation

[0018] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0019] Example 1, refer to Figure 1 It provides a remote real-time control system for video surveillance cameras, including a video control data acquisition module, a video control co-construction modeling module, a collaborative encoding and transmission module, and a control output optimization module.

[0020] The visual control data acquisition module is used to establish real-time visual control datasets and acquire historical visual control datasets.

[0021] The visual control isomorphic modeling module is used to train a visual control isomorphic coding model based on a historical visual control dataset and output the parameter set of the visual control isomorphic coding model.

[0022] The co-coding transmission module is used to generate a coding structure parameter set based on the visual control isomorphic coding model parameter set and the real-time visual control dataset, and to co-code the video frame data according to the coding structure parameter set to obtain a video bitstream carrying control symbols and send it.

[0023] The control output optimization module is used to parse control symbols from the video bitstream to output camera control commands, and to estimate link delay based on the control command timestamp and video frame timestamp and make synchronous adjustments.

[0024] This invention achieves video and PTZ control coordination within the image communication link, embeds control symbols into the video bitstream, and performs synchronous correction based on the delay estimation of the control and video dual links to ensure that what is seen is what is controlled. By mapping PTZ motion and control intentions to the coding structure and ROI bit allocation through visual-control isomorphism, the invention enables low-latency and high-definition transmission of key target areas even under network fluctuations, reduces target loss and misoperation, and improves the continuity of monitoring, handling, and evidence collection in chemical industrial parks and oil and gas storage tank areas.

[0025] The visual control data acquisition module includes a video acquisition unit, an attitude acquisition unit, a control acquisition unit, and a network status acquisition unit.

[0026] The video acquisition unit is used to acquire video frame data of the monitored scene. Carrying collection timestamp .

[0027] The attitude acquisition unit is used to acquire camera gimbal attitude data, which includes pitch angle data. Azimuth data and variable value data Carrying attitude sampling timestamp .

[0028] The control acquisition unit is used to acquire control command data issued by the operator. The control command data includes gimbal rotation commands and zoom commands, which are uniformly represented as increments. Sending timestamps with instructions .

[0029] The network status acquisition unit is used to collect link network status data, which includes latency estimates. Bandwidth estimates and packet loss rate estimate .

[0030] The video control data acquisition module is used to acquire video frame data, gimbal attitude data, control command data, and link network status data in frames per second. Based on this, within a small time window (±20ms), the most recent attitude sample and command sample are selected and bound to the frame to form a real-time visual control dataset, while historical data is archived to form a historical visual control dataset.

[0031] The visual control isomorphic modeling module includes a structural feature extraction unit and an isomorphic mapping establishment unit.

[0032] The structural feature extraction unit is used for historical visual control datasets. Encoding structure features are extracted from video frame data, including reference frame dependencies, motion vector fields, and distribution of candidate blocks for regions of interest. Control structure features are extracted from historical gimbal attitude data and historical control command data, including gimbal angular velocity sequences, zoom change sequences, and control command trigger time distribution.

[0033] The isomorphic mapping establishment unit is used to train the visual-control isomorphic coding model based on the coding structure features and control structure features through a supervised learning regression algorithm, and maps the coding structure feature space to the control structure feature space, outputting the parameter set of the visual-control isomorphic coding model.

[0034] The logic of the vision control isomorphic coding model is as follows: For every moment in history Construct a training sample: Input features At least include: Control characteristics: ; Network characteristics: , , ; Content features (which can be statistically analyzed within the ROI): such as texture complexity and motion intensity level (which can be the mean of block matching residuals or optical flow intensity). Output Labels The target configuration for the encoding structure parameters shall include at least: GOP length ; Quantitative intensity configuration of ROI and background; Motion estimation search range level ; ROI bit percentage ; Linear regression was selected: ; Ridge Regression Loss Training: ; The parameter set of the visual-control isomorphic coding model after training is complete: ; Based on the parameter set of the visual control isomorphic coding model and the preset rotation direction and angular velocity in the real-time gimbal control variables, the reference frame dependency is adjusted to obtain a set of reference frames consistent with the gimbal's motion direction.

[0035] Based on the parameter set of the visual control isomorphic coding model and the real-time zoom change, the distribution of candidate blocks of interest is scaled to determine the set of interest region blocks.

[0036] Based on the parameter set of the visual control isomorphic coding model and the real-time PTZ angular velocity sequence and zoom change sequence, the search range and accuracy of the motion vector field are adjusted to generate a coding structure parameter set for the current coding period. The coding structure parameter set is used to constrain GOP division, reference frame selection and region of interest block division during video coding.

[0037] For each real-time frame time : Obtained from real-time visual control dataset ; Calculated by the visual control isomorphic coding model ; Will Parsing into an executable configuration, forming a set of encoded structure parameters: ; in Output the strategy for selecting the reference frame set.

[0038] Convert the link bandwidth estimate into the current target total bitrate budget (linear scaling factor) (Fixed in system configuration) ; And budget according to ROI allocation and background bitrate. and .

[0039] Based on the bitrate budget and pixel size, a preset lookup table or empirical mapping is used to obtain... and And limited to Within the range.

[0040] The cooperative coding transmission module also includes a region of interest determination unit and a cooperative coding execution unit.

[0041] The region of interest determination unit analyzes each video frame data. The target bounding box set is obtained by using background modeling, foreground segmentation, and connected component analysis or a lightweight detection network (such as YOLO-tiny). If multiple target bounding boxes exist simultaneously, the main target bounding box is selected based on either the largest area or the largest IOU with the target tracked in the previous frame, denoted as . , Here are the horizontal coordinates of the center point of the target bounding box in the image. Here are the coordinates of the center point of the target bounding box on the vertical axis of the image. The width of the target bounding box, in pixels. The height of the target bounding box is in pixels. Kalman tracing or optical flow tracing is performed on the selected target bounding box, and the smoothed target bounding box is output. and the predicted target box at the next moment To allow the region of interest (ROI) to adaptively increase with the gimbal control intensity and prevent the target from moving out due to control lag, the gimbal angular velocity is calculated. , ; And set the margin for the expansion of the region of interest: , ; The base value for expanding the region of interest is set to 16-128 pixels or equivalent to 1-4 CTU margins.

[0042] for The relationship coefficient represents the degree of influence of pitch angle change on the expansion of region of interest. When the gimbal moves up and down, the expansion of region of interest increases with the increase of angular velocity.

[0043] for The relationship coefficient indicates the degree of influence of azimuth angle change on the expansion of region of interest. When the gimbal rotates left and right, the expansion of region of interest also increases with the rotation angular velocity.

[0044] and The scaling factor is (pixels) / (° / s), ranging from 2 to 20.

[0045] by Expand the center to obtain the region of interest bounding box ,by Expanding to obtain the predicted interest region box .

[0046] With a fixed code block size (e.g., CTU=64×64), Overlaying onto the block coordinate system yields: Interest Area Block Set All and Intersecting block index pairs ; Interest Area Block Identifier Set :right Each block generates a unique identifier.

[0047] Similarly, a set of predicted interest region blocks is obtained. and the corresponding set of identifiers .

[0048] The collaborative coding execution unit is used to configure first coding parameters for regions of interest (ROIs) and second coding parameters for regions of non-ROIs (NROIs) according to the coding structure parameter set. The bitrate of the first coding parameter is higher than that of the second coding parameter. The unit encodes the video frame data to generate a video stream carrying control symbols.

[0049] The collaborative coding execution unit is used to encode the PTZ control command data and the set of region of interest block identifiers into control symbols and embed them into the image communication protocol header of the video stream.

[0050] ; Encoder Press Encoding video frame data: GOP press Segmentation; Motion estimation search range by Configuration; Use blocks in the set of interest region blocks The remaining blocks use ; Output video stream.

[0051] Encapsulating control symbols within the same video stream: Frame-level control symbols contain the following fields: ; Block-level control symbols contain fields: ; Quantification rules: Angles or angle increments are quantized using 0.01° fixed-point quantization; Use 0.001 or the integer value of the gear to adjust the multiplication factor; Timestamps are in milliseconds. The number of block identifier sets should be written first. Write again A number of integer IDs. Package location: Using H.265 / H.266 SEI user_data_unregistered, frame-level and block-level control symbols are written as SEI payloads into the corresponding frames.

[0052] The collaborative coding and transmission module also includes a multi-camera relay decision unit.

[0053] The multi-camera relay decision unit is used to predict the set of candidate cameras that the target may enter, based on the motion vector field of the target in the video frame data, the current camera gimbal attitude data, and the gimbal attitude data of the neighboring cameras in the real-time visual control dataset.

[0054] The multi-camera relay decision unit is used to generate relay control command data for candidate cameras and write control symbols, so that the candidate cameras can perform pre-rotation and pre-magnification according to the relay control command data, and mark the relay camera identifier in the video stream.

[0055] The control output optimization module includes a control parsing unit, a time delay estimation unit, and a synchronization correction unit.

[0056] The control parsing unit is used to parse control symbols from the received video stream, recover the PTZ control command data, and output the camera control commands.

[0057] The decoding end receives the video stream and decodes the video frames. At the same time, it parses the SEI to obtain control symbols, recovers the PTZ control command data, and sends it to the camera for execution in real time.

[0058] The delay estimation unit is used to calculate the control link delay and video link delay based on the timestamp of camera control command transmission, the timestamp of gimbal attitude change, and the timestamp of video frame decoding.

[0059] The synchronization correction unit is used to adjust the decoding buffer queue and display output time according to the control link delay and video link delay, so that the video frame display time and the actual gimbal attitude of the camera are kept within a preset deviation threshold.

[0060] Calculate control link delay: ; Video link latency: ; Define the difference in latency between the control link and the video link: ; Use differential delay to drive the display buffer depth Adaptive adjustment ( To adjust the coefficient, (with upper and lower boundaries) ; when If the deviation exceeds the preset threshold, continue adjusting the buffer; when Synchronization is considered to be achieved if the deviation threshold is not exceeded.

[0061] The control output optimization module also includes a model update unit.

[0062] The model update unit is used to collect data on the execution results of camera control commands and the target tracking and holding time data. It uses the success rate of camera control command execution and the proportion of the target held within the region of interest as performance indicators, and combines them with link network status data to form a performance evaluation result.

[0063] The model update unit is used to update the parameter set of the visual control isomorphic coding model based on the performance evaluation results, and to adjust the generation strategy of the coding structure parameter set and the prediction method of the interest region block set.

[0064] System periodic statistical performance indicators: Target Preservation Ratio: The percentage of frames in which the center of the target falls within the region of interest box; Control success rate: The percentage of attitude changes that achieve the expected increment after a command is issued; Combined with the magnitude of changes in network status.

[0065] Incremental updates are triggered when the target maintenance ratio or control success rate falls below a threshold: Add the most recent samples to the training cache and update them using the same loss function.

[0066] The cooperative coding transmission module uses the image communication protocol syntax extension method to embed control symbols, which include frame-level control symbols and block-level control symbols.

[0067] Frame-level control symbols are used to indicate the PTZ control command data and candidate relay camera identifiers corresponding to the current video frame.

[0068] Block-level control symbols are used to indicate the location of the region of interest block and the corresponding coding structure parameters.

[0069] The decoding end parses frame-level control symbols and block-level control symbols while performing image decoding on the video bitstream.

[0070] Example 2, refer to Figure 2 This provides a method for remote real-time control of cameras based on video surveillance, including the following steps: Step S1: Establish a real-time visual control dataset and collect historical visual control datasets.

[0071] Step S2: Train the visual control isomorphic coding model based on the historical visual control dataset and output the parameter set of the visual control isomorphic coding model.

[0072] Step S3: Generate a coding structure parameter set based on the visual control isomorphic coding model parameter set and the real-time visual control dataset, and co-code the video frame data according to the coding structure parameter set to obtain a video bitstream carrying control symbols and send it.

[0073] Step S4: Parse control symbols from video stream to output camera control commands, and estimate link delay based on control command timestamps and video frame timestamps and make synchronous adjustments.

[0074] This invention transmits control symbols along with the video stream in the image communication link, and performs dual-link delay estimation and synchronous correction based on the control command timestamp, attitude feedback timestamp, and video frame timestamp. Even under jitter in public or private wireless networks, it can still control the attitude of the displayed screen and the actual gimbal attitude within a preset deviation threshold, significantly reducing the lag and probability of misoperation in remote gimbal operation, and improving the controllability of emergency response in chemical industrial parks.

[0075] By mapping the PTZ control variables to the coding structure parameters through the visual control isomorphism mechanism, the encoder prioritizes the encoding and transmission of the areas of interest that may be of interest in the current and next moments. Even when bandwidth is limited or packet loss is high, it can still ensure the clarity and low latency of the area of ​​interest where the target is located, and reduce the probability of the target running out of the field of view.

[0076] Compared to the traditional method of parallel video and independent control signaling, this invention embeds frame-level and block-level control symbols into the video stream through protocol extension, enabling the receiving end to parse control information and execute control outputs while decoding the video. This reduces the risk of mismatch caused by multi-channel synchronization and middleware coupling, and is especially suitable for complex link deployments across networks and gateways in a campus.

[0077] This invention incorporates link network status data into the coding structure parameter generation logic, and adaptively configures ROI / background bits and coding strength according to bandwidth budget and packet loss risk. This avoids queuing delays and stuttering caused by pushing streams at a fixed bit rate even when the network deteriorates, and improves the monitoring availability and continuous evidence collection capabilities in low-bandwidth and weak-coverage areas.

[0078] In oil and gas storage tank areas or chemical industrial parks, when remotely tracking suspicious personnel and vehicles or suspected leak points, the system can complete target locking, zooming, following, and stable centering faster under the same network conditions, reducing the time lost due to switching perspectives, and improving the efficiency of incident handling and the continuity and usability of video evidence.

[0079] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media containing computer-usable program code. The storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read-Only Memory (EPROM), Programmable Read-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0080] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the protection scope of the present invention.

Claims

1. A remote real-time control system for cameras based on video surveillance, characterized in that, It includes a visual control data acquisition module, a visual control co-construction modeling module, a collaborative coding and transmission module, and a control output optimization module; The visual control data acquisition module is used to establish a real-time visual control dataset and acquire historical visual control datasets. The visual control isomorphic modeling module is used to train a visual control isomorphic coding model based on a historical visual control dataset and output a parameter set of the visual control isomorphic coding model. The co-coding transmission module is used to generate a coding structure parameter set based on the visual control isomorphic coding model parameter set and the real-time visual control dataset, and to co-code the video frame data according to the coding structure parameter set to obtain a video bitstream carrying control symbols and send it. The control output optimization module is used to parse control symbols from the video stream, output camera control commands, and estimate link delay based on the control command timestamp and video frame timestamp, and make synchronous adjustments.

2. The remote real-time control system for cameras based on video surveillance as described in claim 1, characterized in that, The visual control data acquisition module includes a video acquisition unit, an attitude acquisition unit, a control acquisition unit, and a network status acquisition unit; The video acquisition unit is used to acquire video frame data of the monitored scene; The attitude acquisition unit is used to acquire camera gimbal attitude data, which includes pitch angle data, azimuth angle data, and zoom value data. The control acquisition unit is used to acquire control command data issued by the operating terminal, including gimbal rotation commands and zoom commands. The network status acquisition unit is used to collect link network status data, which includes latency estimates, bandwidth estimates, and packet loss rate estimates. The visual control data acquisition module is used to time-align video frame data, gimbal attitude data, control command data, and link network status data to form a real-time visual control dataset and to acquire historical visual control datasets.

3. The remote real-time control system for cameras based on video surveillance as described in claim 2, characterized in that, The visual control isomorphic modeling module includes a structural feature extraction unit and an isomorphic mapping establishment unit; The structural feature extraction unit is used to extract coded structural features from video frame data based on historical visual control datasets. The coded structural features include reference frame dependencies, motion vector fields, and distribution of candidate blocks for regions of interest. It also extracts control structural features from historical gimbal attitude data and historical control command data. The control structural features include gimbal angular velocity sequences, zoom change sequences, and control command trigger time distribution. The isomorphic mapping establishment unit is used to train the visual-control isomorphic coding model based on the coding structure features and control structure features through a supervised learning regression algorithm, and to map the coding structure feature space to the control structure feature space, outputting a parameter set of the visual-control isomorphic coding model.

4. The remote real-time control system for cameras based on video surveillance as described in claim 3, characterized in that, The logic of the vision control isomorphic coding model is as follows: Based on the parameter set of the visual control isomorphic coding model and the preset rotation direction and angular velocity in the real-time gimbal control variables, the reference frame dependency is adjusted to obtain a set of reference frames consistent with the gimbal motion direction. Based on the parameter set of the visual control isomorphic coding model and the real-time zoom change, the distribution of candidate blocks of interest is scaled to determine the set of blocks of interest. Based on the visual control isomorphic coding model parameter set and the real-time PTZ angular velocity sequence and zoom change sequence, the search range and accuracy of the motion vector field are adjusted to generate a coding structure parameter set for the current coding period. The coding structure parameter set is used to constrain GOP partitioning, reference frame selection and region of interest block partitioning during video coding.

5. The remote real-time control system for cameras based on video surveillance as described in claim 4, characterized in that, The cooperative coding transmission module further includes an interest region determination unit and a cooperative coding execution unit; The region of interest determination unit is used to determine the region of interest blocks in the current video frame based on the real-time video control dataset and the coding structure parameter set, and to predict the region of interest blocks at the next moment, generating a set of region of interest blocks and a corresponding set of region of interest block identifiers. The collaborative coding execution unit is used to configure first coding parameters for regions of interest blocks and second coding parameters for non-regions of interest blocks according to the coding structure parameter set, wherein the bit rate of the first coding parameter is higher than the bit rate of the second coding parameter, to encode video frame data and generate a video bitstream carrying control symbols; The collaborative coding execution unit is used to encode the PTZ control command data and the set of region of interest block identifiers into control symbols and embed them into the image communication protocol header of the video stream.

6. The remote real-time control system for cameras based on video surveillance as described in claim 5, characterized in that, The collaborative coding transmission module also includes a multi-camera relay decision-making unit; The multi-camera relay decision unit is used to predict the set of candidate cameras that the target may enter, based on the motion vector field of the target in the video frame data, the current camera gimbal attitude data, and the gimbal attitude data of the neighboring cameras in the real-time visual control dataset. The multi-camera relay decision unit is used to generate relay control command data for candidate cameras and write control symbols, so that the candidate cameras can pre-rotate and pre-zoom according to the relay control command data, and mark the relay camera identifier in the video stream.

7. The remote real-time control system for cameras based on video surveillance as described in claim 6, characterized in that, The control output optimization module includes a control parsing unit, a time delay estimation unit, and a synchronization correction unit. The control parsing unit is used to parse control symbols from the received video stream, recover the PTZ control command data, and output the camera control command. The delay estimation unit is used to calculate the control link delay and video link delay based on the timestamp of the camera control command transmission, the timestamp of the gimbal attitude change, and the timestamp of the video frame decoding. The synchronization correction unit is used to adjust the decoding buffer queue and display output time according to the control link delay and video link delay, so that the video frame display time and the actual gimbal attitude of the camera are kept within a preset deviation threshold.

8. The remote real-time control system for cameras based on video surveillance as described in claim 7, characterized in that, The control output optimization module also includes a model update unit; The model update unit is used to collect camera control command execution result data and target tracking and holding time data, and uses the success rate of camera control command execution and the target holding ratio in the region of interest block as performance indicators, and combines them with link network status data to form performance evaluation results. The model update unit is used to update the parameter set of the visual control isomorphic coding model based on the performance evaluation results, and to adjust the generation strategy of the coding structure parameter set and the prediction method of the interest region block set.

9. The remote real-time control system for cameras based on video surveillance as described in claim 8, characterized in that, The cooperative coding transmission module embeds control symbols using an image communication protocol syntax extension method. The control symbols include frame-level control symbols and block-level control symbols. The frame-level control symbol is used to indicate the PTZ control command data and candidate relay camera identifiers corresponding to the current video frame. The block-level control symbols are used to indicate the location of the region of interest block and the corresponding coding structure parameters; The decoding end parses frame-level control symbols and block-level control symbols while performing image decoding on the video bitstream.

10. A remote real-time control method for cameras based on video surveillance, applied in a remote real-time control system for cameras based on video surveillance as described in any one of claims 1-9, characterized in that, Includes the following steps: Step S1: Establish historical and real-time video surveillance datasets; Step S2: Train the visual control isomorphic coding model based on the historical visual control dataset, and output the parameter set of the visual control isomorphic coding model; Step S3: Generate a coding structure parameter set based on the visual control isomorphic coding model parameter set and the real-time visual control dataset, and co-code the video frame data according to the coding structure parameter set to obtain a video bitstream carrying control symbols and send it. Step S4: Parse control symbols from video stream to output camera control commands, and estimate link delay based on control command timestamps and video frame timestamps and make synchronous adjustments.

Images