Method, device and medium for monitoring and compressing video based on lightweight deep model

Abstract

The application discloses a kind of based on light weight deep model's monitoring video compression method, equipment and medium, it is related to video compression technical field, including to input monitoring video sequence, through interframe difference and time sequence difference degree calculation, front and rear frame comparison, determine original key frame set;In set, with local dynamic target detection and enhancement model, generate enhanced key frame data.Based on enhanced key frame data, do difference coding to redundant frame set, according to the difference degree result of enhanced key frame allocation difference coding, obtain different coding frame set.The set is input light weight deep compression model, after parameter quantization, pruning and distillation operation, output compressed video data.Finally, compressed video data and original key frame set mapping comparison, correct target area distortion in compression, generate and store compression correction video.The method greatly reduces model calculation complexity and storage overhead, while maintaining high-precision compression performance, meets the real-time operation demand of edge device.

Description

Technical Field

[0001] This invention relates to the field of video compression technology, and in particular to a method, device and medium for compressing surveillance videos based on a lightweight depth model. Background Technology

[0002] With the ever-increasing demand for urban public safety, surveillance video has become an important component of security systems and smart cities. The continuous acquisition and storage of massive amounts of video data brings enormous computational and transmission pressures, especially in scenarios where low-computing-power devices are widely deployed. Achieving efficient video compression and intelligent analysis has become a pressing technical challenge. In recent years, deep learning models have been introduced into the field of video compression and optimization, combining inter-frame differencing, object detection, and intelligent coding strategies to improve the performance of video storage and transmission. However, traditional deep learning models are computationally intensive, making them difficult to deploy on edge devices or low-power terminals. Furthermore, existing video compression methods still have shortcomings in keyframe selection, target region fidelity, and adaptability to dynamic scenes.

[0003] CN108062349B discloses a video surveillance method and system based on structured video data and deep learning. By performing target detection, tracking, and structured processing on video, and combining this with deep analysis via a cloud server, the method improves the management and application capabilities of surveillance data. However, this method primarily focuses on the structure and behavior analysis of video, relying on cloud computing resources for post-processing. It lacks consideration for compression and storage optimization in low-computing-power environments at the front end, limiting its application value in scenarios where edge devices have limited computing power.

[0004] CN119155447B discloses a method for optimizing the transmission of security surveillance video and a video surveillance terminal. This method calculates dynamic change features through differential calculation of adjacent frames, combines a deep learning model to distinguish between rapidly changing dynamics and normal scenes, and dynamically adjusts the video bitrate to ensure image detail. While this method can adaptively adjust compression efficiency during transmission, its optimization focus remains primarily on bitrate control. It lacks a lightweight design for the deep learning model and suffers from insufficient enhancement of keyframes and fidelity processing of local targets, making it difficult to achieve efficient compression and distortion control in computationally limited terminal environments.

[0005] In summary, existing video surveillance compression and optimization methods generally suffer from problems such as reliance on high-computing-power environments, lack of keyframe enhancement and lightweight processing mechanisms, and insufficient fidelity in dynamic scenes. Summary of the Invention

[0006] This invention provides a method, device, and medium for video compression based on a lightweight depth model. By using keyframe dynamic enhancement, differential coding, and depth model quantization pruning and distillation, it achieves efficient video compression and target area fidelity on low-computing-power devices, solving the problem that existing technologies are unable to balance compression efficiency and image quality in resource-constrained environments.

[0007] According to one aspect of the present invention, a method for compressing surveillance video based on a lightweight depth model is provided, comprising:

[0008] The input surveillance video sequence is subjected to inter-frame difference and temporal difference calculation. The original key frame set is determined by comparing the previous and next frames. Enhanced key frame data is generated within the original key frame set through local region dynamic target detection and enhancement model.

[0009] Based on the enhanced keyframe data, the redundant frame set is differentially divided, and differential codes are assigned to the redundant frames according to the difference degree calculation results with the enhanced keyframes, thus obtaining a differentially coded frame set.

[0010] The differentially encoded frame set is input into the lightweight deep compression model, and quantization, pruning and distillation operations are performed on the model parameters to output compressed video data.

[0011] The compressed video data is mapped and compared with the original keyframe set to correct the distortion of the target area during the compression process, generate a compressed and corrected video, and store the compressed and corrected video.

[0012] According to another aspect of the present invention, a surveillance video compression device based on a lightweight depth model is provided, comprising:

[0013] The enhanced keyframe data generation module is used to perform inter-frame difference and temporal difference calculation on the input surveillance video sequence, determine the original keyframe set by comparing the previous and next frames, and generate enhanced keyframe data within the original keyframe set through local region dynamic target detection and enhancement model.

[0014] The differential coding frame set determination module is used to perform differential analysis on the redundant frame set based on the enhanced key frame data, and assign differential codes to the redundant frames according to the difference degree calculation results with the enhanced key frame to obtain the differential coding frame set.

[0015] The compressed video data output module is used to input the set of differentially coded frames into the lightweight deep compression model, perform quantization, pruning and distillation operations on the model parameters, and output compressed video data.

[0016] The compressed and corrected video generation module is used to map and compare the compressed video data with the original keyframe set, correct the distortion of the target area during the compression process, generate the compressed and corrected video, and store the compressed and corrected video.

[0017] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0018] At least one processor;

[0019] and memory that is communicatively connected to at least one processor;

[0020] The memory stores a computer program that can be executed by at least one processor, which enables the at least one processor to execute the video compression method based on a lightweight depth model according to any embodiment of the present invention.

[0021] According to another aspect of the present invention, a computer-readable storage medium is provided, which stores computer instructions for causing a processor to execute and implement the surveillance video compression method based on a lightweight depth model according to any embodiment of the present invention.

[0022] The technical solution of this invention, through inter-frame differential and temporal differential calculation mechanisms, achieves accurate extraction of keyframes and local enhancement of dynamic targets, effectively highlighting motion regions and detailed features, and improving the expression efficiency of subsequent encoding. In the redundant frame processing stage, this invention utilizes differential coding and differentiated bitrate allocation strategies, combined with adjacent keyframe references, which reduces the amount of redundant data and achieves adaptive mapping between bitrate and content complexity, thereby improving compression flexibility. Furthermore, by performing quantization, pruning, and distillation operations on the deep compression model, this invention maintains high-precision compression performance while significantly reducing model computational complexity and storage overhead, meeting the real-time operation requirements of edge devices.

[0023] Overall, this invention balances compression efficiency and image quality under computing power constraints, and has high engineering application value and promotion significance.

[0024] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0025] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0026] Figure 1 A flowchart illustrating a video compression method based on a lightweight depth model, provided for an embodiment of the present invention;

[0027] Figure 2 A schematic diagram of a surveillance video compression device based on a lightweight depth model provided in an embodiment of the present invention;

[0028] Figure 3 A schematic diagram of the structure of an electronic device for implementing the monitoring video compression method based on a lightweight depth model according to an embodiment of the present invention. Detailed Implementation

[0029] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0030] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0031] Figure 1 This is a flowchart illustrating a surveillance video compression method based on a lightweight depth model, provided as an embodiment of the present invention. This embodiment is applicable to situations requiring efficient video compression and target area fidelity on low-computing-power devices. The method can be executed by a surveillance video compression device based on a lightweight depth model, which can be implemented in hardware and / or software and can be configured in an electronic device. Figure 1 As shown, the method specifically includes the following steps:

[0032] S1: Perform inter-frame difference and temporal difference calculation on the input surveillance video sequence, determine the original keyframe set by comparing the previous and next frames, and generate enhanced keyframe data within the original keyframe set through local region dynamic target detection and enhancement models.

[0033] S1.1: Determining the original set of keyframes.

[0034] S1.1.1: Monitoring video sequence The pixel difference matrix is ​​calculated frame by frame, and the set of difference matrices is saved as a time series. For a moment The data consists of grayscale or luminance frames. The pixel difference matrix is ​​calculated using the pixel-by-pixel difference between the current frame and the previous frame. Each element of the difference matrix is ​​stored as an 8-bit or 16-bit unsigned integer. In noisy or high dynamic range environments, it is recommended to use a 16-bit temporary cache followed by normalization and writing back to 8-bit storage. A circular buffer is used for sliding statistics in the storage structure. In memory-constrained scenarios, only the cumulative difference of each block can be stored instead of the complete pixel matrix to save space.

[0035] It should be noted that this invention not only saves the frame-by-frame difference values, but also forms a set of difference matrices and stores them in chronological order, thereby preserving the temporal change information in the video sequence. In this way, it can more accurately reflect the changes in inter-frame motion and local dynamics, avoid misjudgment caused by instantaneous noise, and improve the stability of key frame determination.

[0036] S1.1.2: Divide each difference matrix into blocks of fixed block size, obtain the energy of each block, and calculate the total frame energy. With the largest block energy percentage .

[0037] Specifically, block energy is represented by the accumulated difference of pixel values ​​within the block. Two implementation strategies can be selected: absolute difference accumulation or squared difference accumulation. It is recommended to use absolute difference accumulation to reduce computational overhead when computing power is extremely limited, while squared difference accumulation is used to amplify the contribution of significant changes in scenarios that need to be sensitive to large changes.

[0038] The total frame energy is obtained by summing the energy of all blocks in the frame. The maximum block energy is recorded as the highest value of a single block energy in the frame. The maximum block energy is compared with the total frame energy and the ratio is calculated to obtain the percentage of the maximum block energy. The higher the ratio, the more concentrated the motion in the frame is in a few blocks rather than being dispersed throughout the frame.

[0039] S1.1.3: Normalize the sum of frame energies with the median energy of the nearest sliding window of length L to obtain the normalized residual energy. .

[0040] It should be noted that the median was chosen because it is not sensitive to extreme values ​​and is more robust to noisy samples.

[0041] The setting of the sliding window length L follows the principle of time scale trade-off: for common videos with 25 to 30 frames per second, it is recommended to use 16, covering a time scale of about half a second to one second; for low frame rates, L can be appropriately reduced, and for high frame rates, L can be appropriately increased. The specific value is fine-tuned during the deployment phase based on the frame rate and target duration.

[0042] Furthermore, the degree of difference The calculation is as follows:

[0043]

[0044] in, and The weighting coefficient is used when the difference is... When less than or equal to the first threshold, Marked as candidate keyframes; when the difference When it is greater than or equal to the second threshold, Marked as a redundant frame.

[0045] Preferably, the first and second thresholds are set as follows:

[0046] In the sliding window Internal calculation of variability The median and the absolute deviation of the median ;

[0047] The first threshold is the degree of difference. The median minus k times the median absolute deviation (MAD); the second threshold is the degree of difference. The threshold is calculated by adding k times the median absolute deviation (MAD). Here, k is an adjustment factor used to control the width of the threshold range. A larger k value results in a wider threshold range and stricter keyframe selection; a smaller k value tightens the threshold range, making it easier to detect slight dynamic frames. An empirically recommended value is 2 to 3, with 3 being commonly used to balance false positives and false negatives.

[0048] S1.1.4: For each candidate keyframe, calculate the normalized residual energy difference with the preceding and following N frames. When the maximum residual energy difference is less than or equal to the preset consistency tolerance (which can be 0.15, i.e., 15% normalized energy fluctuation), and the corresponding maximum block energy percentage is... When the candidate keyframe is greater than or equal to a preset local motion percentage threshold, it is added to the original keyframe set.

[0049] As can be seen, this invention prevents transient, non-event-related fluctuations from being misjudged as keyframes through the consistency condition, and ensures that the selected frames have significant local target signals rather than global changes through the local energy condition. In practice, the above operations can be completed in a circular buffer of the differential residual set, and candidate frames that do not meet the local energy condition are eliminated early to save computational and memory resources.

[0050] S1.2: Enhance the generation of keyframe data.

[0051] In the original keyframe, the initial detection box of the target region is generated based on the block region with the highest energy. To avoid missing the crop and ensure integrity, the detection box is expanded outward by a predefined pixel value (e.g., 10~20 pixels) to form a local analysis region.

[0052] Specifically, for each original keyframe, the block energy distribution of that frame is read and the block with the highest energy or a set of several adjacent high-energy blocks is located. These high-energy block sets are then used to generate the initial target detection box.

[0053] It should be noted that conventional methods typically run detection on the entire frame, resulting in unnecessary computational overhead. This invention prioritizes energy-based selection of local regions to significantly reduce detection costs and improve response speed for small targets. The expanded local analysis region can undergo simple connected component merging to combine adjacent high-energy blocks into a continuous region, thus adapting to situations where targets may extend beyond the boundaries of a single block.

[0054] Furthermore, within the local analysis region, a lightweight detection algorithm (such as fast detection based on background subtraction, or the YOLO-tiny model) is invoked to extract the local target location.

[0055] Specifically, background difference gating is used as a lightweight pre-screening strategy in each local analysis region. Background difference is simple to compute and inexpensive in local regions. Lightweight depth detectors, such as YOLO-tiny or other micro-convolutional networks, are only called when the gating is deemed to be dynamically significant.

[0056] For example, the cumulative pixel difference of a local region within a short sliding window is statistically analyzed. If the cumulative value exceeds a preset threshold, the region is considered to have significant motion and enters the depth detection stage. When the depth detector is invoked, it uses pre-compressed model parameters and constrains the number of output boxes and non-maximum suppression thresholds to control inference overhead. The detector output includes target bounding boxes and confidence scores. It is recommended that the confidence threshold be set to 0.5 to 0.7. The above two-stage detection strategy retains the recognition capability of the depth model while minimizing the frequency of calling regular frames, thus making it suitable for continuous online deployment of low-computing-power monitoring devices.

[0057] Furthermore, adaptive sharpening or histogram equalization is applied to the detected local target locations to improve texture and edge contrast.

[0058] The adaptive sharpening process employs a non-linear unsharp-mask style. During enhancement, a local area is first lightly blurred to obtain a low-frequency image. Then, the low-frequency image is subtracted from the local original image to obtain the high-frequency components, which are then superimposed back onto the original image with a controllable gain.

[0059] Furthermore, by statistically analyzing local grayscale histograms, the brightness and contrast of local target locations are adjusted to approximate preset standard values ​​(e.g., the average brightness is within the range of 128±10).

[0060] Adaptive histogram equalization methods, such as Contrast-Limited Adaptive Histogram Equalization (CLAHE), are used to set a contrast limit threshold to avoid local oversaturation. Brightness normalization is then performed: the grayscale mean of the local region is statistically analyzed, and the gain is adjusted according to the target mean. The gain calculation is based on the target mean divided by the current mean, and the pixel values ​​are truncated after adjustment to maintain numerical validity.

[0061] The enhanced local target location is used to generate enhanced keyframes based on the original keyframe locations, and then stored in the enhanced keyframe set.

[0062] S2: Based on the enhanced keyframe data, the redundant frame set is differentially divided, and differential codes are assigned to the redundant frames according to the difference calculation results with the enhanced keyframes, thus obtaining a differentially coded frame set.

[0063] S2.1: For each redundant frame, select the nearest frame from the enhanced keyframes before and after the timeline as the reference keyframe, and perform pixel-by-pixel difference on the redundant frame and the reference keyframe to obtain the residual image.

[0064] Specifically, the keyframe selection rules include:

[0065] Before and after the redundant frame, find the enhanced keyframe with the smallest timestamp difference from the redundant frame in the enhanced keyframe set, obtaining forward candidate frames and backward candidate frames respectively. Calculate the time interval between the redundant frame and the two candidate keyframes, and select the one with the smaller interval as the reference keyframe. If the intervals before and after are equal, the forward keyframe is preferred because it is closer to the causal direction of motion in the time series, and can be used as a more stable prediction benchmark.

[0066] It should be noted that, compared to the conventional method of selecting only the previous frame, this invention allows for comparison between the previous and previous frames, avoiding deviation of the reference keyframe from the target at points of abrupt motion change, thereby improving the compression effect of the residual image.

[0067] S2.2: Calculate the mean square error of the residual image. And calculate the difference degree of each redundant frame. :

[0068]

[0069] Based on preset upper and lower bitrate limits and The target bitrate is calculated using a linear mapping of dissimilarity.

[0070]

[0071] in, and These represent the maximum and minimum differences in the redundancy of redundant frames in historical statistics. Initially, these differences can be estimated using the statistical results of the first 100 redundant frames. During operation, a sliding update method is used, updating the maximum and minimum values ​​with each new batch of redundant frames to avoid long-term offsets causing mapping failures. For extreme values, a percentile correction method is used, for example, taking the 5th percentile as... 95th percentile as This prevents accidental abnormal frames from distorting the mapping.

[0072] S2.3: Based on target bit rate Extract the corresponding differential coding strategy from the differential code rate mapping table.

[0073] The differential coding strategy includes the following dimensions: the quantization level of the residual image (e.g., quantization step size from small to large corresponds to different quality levels); the selection of entropy coding algorithm (e.g., Huffman coding is used at low bit rates, and adaptive binary arithmetic coding CABAC is used at high bit rates); and block-level adaptation (allowing small block coding at high bit rates and using large block merging to reduce the number of bits at low bit rates).

[0074] It should be noted that the mapping table is loaded during system initialization, and only table lookups are needed during operation, without the need for dynamic calculation of complex strategies, making it suitable for low-computing-power devices.

[0075] S3: Input the set of differentially encoded frames into the lightweight deep compression model, perform quantization, pruning and distillation operations on the model parameters, and output compressed video data.

[0076] S3.1: Based on the original lightweight deep compression model, the weight sparsity of each convolutional channel is calculated, and channels with weight sparsity higher than the preset sparsity threshold are deleted to obtain the pruned lightweight deep compression model.

[0077] S3.2: Using the original lightweight deep compression model as the teacher model and the pruned lightweight deep compression model as the student model, calculate the output results of the teacher model and the student model on the same input frame for the differentially encoded frame set.

[0078] That is, the set of differentially encoded frames is input frame by frame into the teacher model and the student model to obtain the output feature map or the final compressed reconstruction result, respectively.

[0079] S3.3: Subtract the output values ​​of the two frames one by one to obtain the difference of each pixel or feature point. Then square these differences one by one and sum them up to obtain the overall difference value.

[0080] Specifically, the outputs of the teacher model and the student model on the same frame are subtracted pixel by pixel; each difference is then squared; and the squared differences of all pixels in the frame are summed to obtain the overall difference value.

[0081] If the input contains multiple frames, the difference for each frame is calculated separately, and then the average of all frames is taken to obtain the overall difference. For example, for a 1920×1080 image frame, the sum of the squared differences of each pixel is assumed to be 1.2×10⁻⁶. 6 The difference of this frame is the sum of the differences divided by the number of pixels (2073600), which is approximately 0.58.

[0082] S3.4: Based on the difference value, adjust the weight parameters of the student model so that the difference between the output result and the output result of the teacher model is within a preset range, thus completing the distillation.

[0083] Specifically, if the output difference between the student model and the teacher model exceeds the upper limit (e.g., 0.3), the weights of the student model are adjusted through backpropagation to make its output closer to that of the teacher model.

[0084] During the adjustment process, priority is given to updating the weights of convolutional layers with larger errors, using a small learning rate (e.g., ...). Gradually correct the model to prevent it from failing.

[0085] When the difference between several consecutive batches consistently falls within the preset range, the distillation is considered complete.

[0086] S3.5: Map the floating-point parameters in the distilled student model to 8-bit integers according to their distribution range to complete the quantization.

[0087] Specifically, after distillation, the floating-point parameters in the student model are statistically analyzed to find the minimum and maximum values ​​of the parameters in each layer. This range is then uniformly mapped to 8-bit integers (0~255), which reduces storage overhead and ensures improved computational efficiency during inference.

[0088] S3.6: Input the set of differentially coded frames into a lightweight deep compression model that has undergone quantization, pruning, and distillation operations, and output a compressed video data stream.

[0089] S4: Map and compare the compressed video data with the original keyframe set, correct the distortion of the target area during the compression process, generate a compressed and corrected video, and store the compressed and corrected video.

[0090] S4.1: Map and compare the compressed video data with the original keyframe set.

[0091] The timestamps of each frame in the compressed video data are mapped to the timestamps in the original keyframe set to generate a frame-level mapping table. This means that for each compressed frame, the closest keyframe in time is found, ensuring that frames at the same time point are accurately aligned. If there are missing frames in the compressed video, timestamps can be inserted using linear interpolation to maintain the alignment.

[0092] Furthermore, based on the frame-level mapping table, the pixel regions of the compressed video frames and the original keyframes are compared frame by frame, and the pixel similarity value of each local region is calculated. When the pixel similarity is lower than the set pixel similarity threshold, the corresponding local region is marked as a compression distortion region, and a region index table containing the coordinates and range of the compression distortion region is generated.

[0093] The pixel similarity value is calculated using the Local Structural Similarity Index (SSIM) or the average pixel difference.

[0094] Specifically, by default, the local area can be consistent with the block area of ​​S1, which facilitates unified mapping; in scenarios that are sensitive to details, smaller blocks can be used for the target area to improve detection accuracy.

[0095] S4.2: Generate compressed and corrected video.

[0096] Based on the region index table, pixel blocks at the same positions in the original keyframe set replace the compression distortion areas in the compressed video frames, maintaining the sharpness of the target area. It should be noted that this operation is performed because the original keyframes are uncompressed, ensuring the sharpness of the area, especially the target area.

[0097] Interpolation is applied to the edges of the replaced area to ensure a natural blend between the replaced and original parts, avoiding abrupt stitching marks. For example, bilinear interpolation or Gaussian blur is applied only to a 3-5 pixel wide area at the boundary, allowing for a smooth transition of pixel values ​​between the inside and outside of the area. This ensures a natural blend of the replaced area, making the artificial repair invisible.

[0098] The frame set that has undergone region replacement and edge smoothing is repackaged into a video stream, generating a corrected video file, and stored in the local storage unit of the monitoring device.

[0099] Specifically, all corrected frames are repackaged into a video stream in chronological order.

[0100] The output video file format can be common compression formats such as H.264 or H.265 to ensure compatibility. Finally, the compressed and corrected video file is written to the local storage unit of the monitoring device, such as an embedded memory card or local hard drive.

[0101] A copy of the regional index table can be retained as additional metadata during storage for subsequent tracing or quality analysis.

[0102] Figure 2 This is a schematic diagram of a surveillance video compression device based on a lightweight depth model, provided as an embodiment of the present invention. Figure 2 As shown, the device includes:

[0103] The enhanced keyframe data generation module 210 is used to perform inter-frame difference and temporal difference calculation on the input surveillance video sequence, determine the original keyframe set by combining the comparison of previous and subsequent frames, and generate enhanced keyframe data within the original keyframe set through local region dynamic target detection and enhancement model.

[0104] The differential coding frame set determination module 220 is used to perform differential analysis on the redundant frame set based on the enhanced key frame data, and assign differential codes to the redundant frames according to the difference degree calculation result with the enhanced key frame to obtain the differential coding frame set.

[0105] The compressed video data output module 230 is used to input the set of differentially coded frames into the lightweight deep compression model, perform quantization, pruning and distillation operations on the model parameters, and output compressed video data.

[0106] The compressed and corrected video generation module 240 is used to map and compare the compressed video data with the original keyframe set, correct the distortion of the target area during the compression process, generate the compressed and corrected video, and store the compressed and corrected video.

[0107] In some optional embodiments, the enhanced keyframe data generation module 210 includes determining the original keyframe set, specifically for:

[0108] Surveillance video sequence The pixel difference matrix is ​​calculated frame by frame, and the set of difference matrices is saved as a time series. For a moment Grayscale or luminance frames;

[0109] Each difference matrix is ​​divided into blocks of fixed block size to obtain the energy of each block, and the total frame energy is calculated. With the largest block energy percentage ;

[0110] Normalize the sum of frame energies to the median energy of the nearest sliding window of length L to obtain the normalized residual energy. ;

[0111] Difference The calculation is as follows:

[0112]

[0113] in, and These are weighting coefficients;

[0114] When the difference When less than or equal to the first threshold, Marked as candidate keyframes; when the difference When it is greater than or equal to the second threshold, Marked as redundant frame;

[0115] For each candidate keyframe, calculate the normalized residual energy difference with the preceding and following N frames. When the maximum residual energy difference is less than or equal to the preset consistency tolerance, and the corresponding maximum block energy percentage is reached... When the candidate keyframe is greater than or equal to a preset local motion percentage threshold, it is added to the original keyframe set.

[0116] In some optional embodiments, the enhanced keyframe data generation module 210 includes an enhanced keyframe data generation submodule, specifically used for:

[0117] In the original keyframe, the initial detection box of the target region is generated based on the block region with the highest energy, and the predefined pixel values ​​are expanded outward to form a local analysis region.

[0118] Within the local analysis region, a lightweight detection algorithm is invoked to extract the local target location;

[0119] Adaptive sharpening or histogram equalization is applied to the detected local target locations to improve texture and edge contrast;

[0120] By statistically analyzing local grayscale histograms, the brightness and contrast of local target locations are adjusted to approximate preset standard values.

[0121] The enhanced local target location is used to generate enhanced keyframes based on the original keyframe locations, and then stored in the enhanced keyframe set.

[0122] In some optional embodiments, the differentially encoded frame set determination module 220 includes a submodule for differentially encoding redundant frames, specifically used for:

[0123] For each redundant frame, select the nearest frame from the enhanced keyframes before and after the timeline as the reference keyframe.

[0124] Perform pixel-by-pixel differencing on redundant frames and reference keyframes to obtain residual images;

[0125] Calculate the mean square error of the residual image. And calculate the difference degree of each redundant frame. :

[0126]

[0127] Based on preset upper and lower bitrate limits and The target bitrate is calculated using a linear mapping of dissimilarity.

[0128]

[0129] in, and These represent the maximum and minimum differences in redundancy among redundant frames in historical statistics.

[0130] Based on target bit rate Extract the corresponding differential coding strategy from the differential code rate mapping table.

[0131] In some alternative embodiments, the compressed video data output module 230 is specifically used for:

[0132] Based on the original lightweight deep compression model, the weight sparsity of each convolutional channel is calculated, and channels with weight sparsity higher than a preset sparsity threshold are deleted to obtain the pruned lightweight deep compression model.

[0133] Using the original lightweight deep compression model as the teacher model and the pruned lightweight deep compression model as the student model, the output results of the teacher model and the student model on the same input frame are calculated for the differentially encoded frame set.

[0134] Subtract the output values ​​of the two frame by frame to obtain the difference of each pixel or feature point, then square these differences one by one and sum them up to obtain the overall difference value.

[0135] Based on the difference value, adjust the weight parameters of the student model so that the difference between the output result and the output result of the teacher model is within a preset range, thus completing the distillation.

[0136] The floating-point parameters in the distilled student model are mapped to 8-bit integers according to their distribution range to complete the quantization.

[0137] In some alternative embodiments, the compressed video data output module 230 is specifically used to: input a set of differentially encoded frames into a lightweight deep compression model after quantization, pruning, and distillation operations, and output a compressed video data stream.

[0138] In some optional embodiments, the compressed and corrected video generation module 240 includes a comparison submodule, specifically used for:

[0139] The timestamp of each frame in the compressed video data is mapped to the timestamp in the original keyframe set to generate a frame-level mapping table;

[0140] Based on the frame-level mapping table, the pixel regions of the compressed video frames and the original keyframes are compared frame by frame. The pixel similarity value of each local region is calculated. When the pixel similarity is lower than the set pixel similarity threshold, the corresponding local region is marked as a compression distortion region, and a region index table containing the coordinates and range of the compression distortion region is generated.

[0141] The pixel similarity value is calculated using the Local Structural Similarity Index (SSIM) or the average pixel difference.

[0142] In some optional embodiments, the compression and correction video generation module 240 includes a compression and correction video generation submodule, specifically used for:

[0143] Based on the region index table, the compression distortion region in the compressed video frame is replaced with the pixel block at the same position in the original keyframe set;

[0144] Interpolation transition is performed on the edges of the replaced area to make the replaced area blend naturally with the original part;

[0145] The frame set that has undergone region replacement and edge smoothing is repackaged into a video stream, generating a corrected video file, and stored in the local storage unit of the monitoring device.

[0146] The surveillance video compression device based on a lightweight depth model provided in this embodiment of the invention can execute the surveillance video compression method based on a lightweight depth model provided in any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the method.

[0147] Figure 3This is a schematic diagram of an electronic device for implementing the lightweight depth model-based video compression method of this invention. The electronic device is intended to represent various forms of digital computers, such as laptops, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframes, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0148] like Figure 3 As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded from storage unit 18 into the RAM 13. The RAM 13 can also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0149] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0150] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as a surveillance video compression method based on a lightweight depth model.

[0151] In some embodiments, the surveillance video compression method based on a lightweight depth model can be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program can be loaded and / or installed on electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the surveillance video compression method based on a lightweight depth model described above can be performed. Alternatively, in other embodiments, processor 11 can be configured to perform the surveillance video compression method based on a lightweight depth model by any other suitable means (e.g., by means of firmware).

[0152] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0153] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0154] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0155] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0156] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or middleware components (e.g., application servers), or frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0157] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through a communication network. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0158] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0159] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A method for compressing surveillance video based on a lightweight depth model, characterized in that, include: The input surveillance video sequence is subjected to inter-frame difference and temporal difference calculation. The original key frame set is determined by comparing the previous and next frames. Enhanced key frame data is generated within the original key frame set through local region dynamic target detection and enhancement model. Based on the enhanced keyframe data, the redundant frame set is differentially divided, and differential codes are assigned to the redundant frames according to the difference degree calculation results with the enhanced keyframes, thus obtaining a differentially coded frame set. The differentially encoded frame set is input into the lightweight deep compression model, and quantization, pruning and distillation operations are performed on the model parameters to output compressed video data. The compressed video data is mapped and compared with the original keyframe set to correct the distortion of the target area during the compression process, generate a compressed and corrected video, and store the compressed and corrected video.

2. The video compression method based on a lightweight depth model as described in claim 1, characterized in that: The determination of the original keyframe set includes: Surveillance video sequence The pixel difference matrix is ​​calculated frame by frame, and the set of difference matrices is saved as a time series. For a moment Grayscale or luminance frames; Each difference matrix is ​​divided into blocks of fixed block size to obtain the energy of each block, and the total frame energy is calculated. With the largest block energy percentage ; Normalize the sum of frame energies to the median energy of the nearest sliding window of length L to obtain the normalized residual energy. ; Difference The calculation is as follows: in, and These are weighting coefficients; When the difference When less than or equal to the first threshold, Marked as candidate keyframes; when the difference When it is greater than or equal to the second threshold, Marked as redundant frame; For each candidate keyframe, calculate the normalized residual energy difference with the preceding and following N frames. When the maximum residual energy difference is less than or equal to the preset consistency tolerance, and the corresponding maximum block energy percentage is reached... When the candidate keyframe is greater than or equal to a preset local motion percentage threshold, it is added to the original keyframe set.

3. The video compression method based on a lightweight depth model as described in claim 2, characterized in that: The generation of the enhanced keyframe data includes: In the original keyframe, the initial detection box of the target region is generated based on the block region with the highest energy, and the predefined pixel values ​​are expanded outward to form a local analysis region. Within the local analysis region, a lightweight detection algorithm is invoked to extract the local target location; Adaptive sharpening or histogram equalization is applied to the detected local target locations to improve texture and edge contrast; By statistically analyzing local grayscale histograms, the brightness and contrast of local target locations are adjusted to approximate preset standard values. The enhanced local target location is used to generate enhanced keyframes based on the original keyframe locations, and then stored in the enhanced keyframe set.

4. The video compression method based on a lightweight depth model as described in claim 3, characterized in that: The process of differentially dividing redundant frames and assigning differential codes to them based on the difference calculation results with the enhanced key frames, resulting in a set of differentially coded frames, includes: For each redundant frame, select the nearest frame from the enhanced keyframes before and after the timeline as the reference keyframe. Perform pixel-by-pixel differencing on redundant frames and reference keyframes to obtain residual images; Calculate the mean square error of the residual image. And calculate the difference degree of each redundant frame. : Based on preset upper and lower bitrate limits and The target bitrate is calculated using a linear mapping of dissimilarity. in, and These represent the maximum and minimum differences in redundancy among redundant frames in historical statistics. Based on target bit rate Extract the corresponding differential coding strategy from the differential code rate mapping table.

5. The video compression method based on a lightweight depth model as described in claim 4, characterized in that: The quantization, pruning, and distillation operations performed on the model parameters include: Based on the original lightweight deep compression model, the weight sparsity of each convolutional channel is calculated, and channels with weight sparsity higher than a preset sparsity threshold are deleted to obtain the pruned lightweight deep compression model. Using the original lightweight deep compression model as the teacher model and the pruned lightweight deep compression model as the student model, the output results of the teacher model and the student model on the same input frame are calculated for the differentially encoded frame set. Subtract the output values ​​of the two frame by frame to obtain the difference of each pixel or feature point, then square these differences one by one and sum them up to obtain the overall difference value. Based on the difference value, adjust the weight parameters of the student model so that the difference between the output result and the output result of the teacher model is within a preset range, thus completing the distillation. The floating-point parameters in the distilled student model are mapped to 8-bit integers according to their distribution range to complete the quantization.

6. The video compression method based on a lightweight depth model as described in claim 5, characterized in that: The differentially encoded frame set is input into a lightweight deep compression model that has undergone quantization, pruning, and distillation operations, and the output is a compressed video data stream.

7. The video compression method based on a lightweight depth model as described in claim 6, characterized in that: The step of mapping and comparing the compressed video data with the original keyframe set includes: The timestamp of each frame in the compressed video data is mapped to the timestamp in the original keyframe set to generate a frame-level mapping table; Based on the frame-level mapping table, the pixel regions of the compressed video frames and the original keyframes are compared frame by frame. The pixel similarity value of each local region is calculated. When the pixel similarity is lower than the set pixel similarity threshold, the corresponding local region is marked as a compression distortion region, and a region index table containing the coordinates and range of the compression distortion region is generated. The pixel similarity value is calculated using the Local Structural Similarity Index (SSIM) or the average pixel difference.

8. The video compression method based on a lightweight depth model as described in claim 7, characterized in that: The generated compressed and corrected video includes: Based on the region index table, the compression distortion region in the compressed video frame is replaced with the pixel block at the same position in the original keyframe set; Interpolation transition is performed on the edges of the replaced area to make the replaced area blend naturally with the original part; The frame set that has undergone region replacement and edge smoothing is repackaged into a video stream, generating a corrected video file, and stored in the local storage unit of the monitoring device.

9. An electronic device, characterized in that, The electronic device includes: At least one processor; and a memory communicatively connected to the at least one processor; The memory stores a computer program that can be executed by the at least one processor, which is then executed by the at least one processor to enable the at least one processor to perform the surveillance video compression method based on a lightweight depth model as described in any one of claims 1-8.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that cause a processor to execute the video compression method based on a lightweight depth model as described in any one of claims 1-8.

Images