Big data hierarchical encryption transmission optimization method based on edge cloud collaboration

Abstract

The application provides a kind of big data hierarchical encryption transmission optimization method based on edge cloud cooperation, comprising: target identification is carried out to the monitoring video collected, and high-value monitoring area and area label are extracted;Integrating interest area, label, real-time network available bandwidth, edge device computing power load and cloud feedback strategy correction factor, dynamically classify video frame into key event frame, routine activity frame and static background frame;A hierarchical encryption strategy is used to generate an encrypted hierarchical video frame group, which is efficiently uploaded to the cloud according to the transmission strategy generated by the bandwidth and strategy;Complete video is recovered by residual decoding and picture reconstruction;Security feature abstract is extracted from key event frame and is chained and stored;At the same time, the reconstruction completeness is evaluated, and the strategy correction factor is fed back to the edge according to the strategy correction factor, to form a closed loop optimization;For solving the problem of high bandwidth consumption, insufficient security and poor real-time performance in massive data transmission of monitoring video.

Description

Technical Field

[0001] This invention relates to the fields of data security and data communication technology, and in particular to a method for optimizing the hierarchical encrypted transmission of big data based on edge-cloud collaboration. Background Technology

[0002] In the field of intelligent surveillance and video analytics, real-time transmission and cloud-based collaborative processing of high-definition video streams are crucial for realizing diverse business applications such as urban governance, traffic management, industrial inspection, and public safety management. With the widespread adoption of the Internet of Things (IoT) and sensing devices, surveillance systems continuously generate massive amounts of continuous video data across various scenarios, characterized by large data volumes, high native bitrates, and diverse real-time requirements. This presents significant challenges to achieving low-latency video transmission, accurate extraction of key information, and cross-domain data fusion applications, placing extremely high demands on the efficiency, intelligence, and reliability of video data transmission links.

[0003] However, existing surveillance video data transmission and processing architectures still face common technical bottlenecks when dealing with the aforementioned high-throughput data streams across multiple scenarios. Mainstream solutions typically employ full cloud upload of the video stream or simple frame-by-frame compression based on fixed rules, lacking real-time perception and dynamic evaluation capabilities of the video content's value. This results in the transmission link being occupied by a large amount of repetitive and invalid background footage, while truly valuable information streams may experience significant delays due to network congestion. Furthermore, traditional processing methods often use uniform encoding parameters and security strategies for video frames, failing to adapt compression intensity and implement tiered security protection based on content importance and business urgency. This makes it difficult to achieve the optimal balance between information importance and transmission efficiency under the constraints of limited computing resources and network bandwidth at the edge. Summary of the Invention

[0004] (a) Technical problems to be solved To address the shortcomings of existing technologies, this invention provides a big data hierarchical encryption transmission optimization method based on edge-cloud collaboration. It has the advantages of video content value perception, hierarchical processing strategy linkage, and cloud quality feedback optimization. It solves the problems of high bandwidth consumption, large transmission delay of key events, and low overall system efficiency caused by the lack of intelligent data hierarchical, adaptive encoding encryption, and edge-cloud collaborative optimization closed loop in monitoring scenarios with massive video data, limited network bandwidth, and limited edge computing resources.

[0005] (II) Technical Solution To achieve the above objectives, the present invention provides the following technical solution: This invention provides a method for optimizing hierarchical encrypted transmission of big data based on edge-cloud collaboration, applied at the edge, and includes the following steps: Target identification is performed on the acquired surveillance video to obtain high-value surveillance areas and area tags; Based on the high-value monitoring area, area label, real-time available network bandwidth, edge device computing power load, and policy correction factor sent from the cloud, the monitoring video is dynamically divided into key event frames, regular activity frames, and static background frames. The key event frames are encrypted at level one, the regular activity frames are encrypted with inter-frame residual coding and level two encryption, the static background frames are extracted with frame features, and the frames are then assembled into encrypted hierarchical video frame groups. A tiered transmission strategy is generated based on the available network bandwidth and the strategy correction factor, and the encrypted tiered video frame group is transmitted to the cloud according to the tiered transmission strategy.

[0006] According to a preferred embodiment of the present invention, the step of performing target identification on the acquired surveillance video to obtain high-value surveillance areas and area tags includes: The acquired surveillance video is split into a video frame sequence, and the video frame sequence is subjected to image denoising and image enhancement to obtain an enhanced video frame sequence. Target detection is performed on the enhanced video frame sequence to obtain target detection boxes and target object semantics; Non-maximum suppression and confidence filtering are applied to the target detection box to obtain a standard detection box, and the standard object semantics corresponding to the standard detection box are selected from the target object semantics. The enhanced video frame sequence is divided into regions based on the standard detection box to obtain target regions. Region labels corresponding to the target regions are generated according to the standard object semantics. High-value monitoring regions are then selected from the target regions based on the region labels.

[0007] According to another preferred embodiment of the present invention, the step of dynamically dividing the surveillance video into key event frames, regular activity frames, and static background frames based on the high-value monitoring area, area tag, real-time acquired network available bandwidth, edge device computing power load, and cloud-sent policy correction factor includes: Interest region mapping information is generated based on the high-value monitoring area and the area label, and the importance of each enhanced video frame in the enhanced video frame sequence is scored based on the interest region mapping information to obtain a frame importance sequence. The available network bandwidth and edge device computing load are normalized and weighted summed to obtain the resource occupancy, and the resource surplus is calculated based on the resource occupancy. The resource sufficiency is scored and weighted using the strategy correction factor to obtain a dynamic scoring threshold. The dynamic scoring threshold is compared with the frame importance sequence, and each enhanced video frame in the enhanced video frame sequence is divided into three-level frame types according to the comparison result, resulting in key event frames, regular activity frames, and static background frames.

[0008] According to another preferred embodiment of the present invention, the step of performing inter-frame residual coding and secondary encryption on the regular active frame includes: The regular active frames are divided into active frame sequence groups according to the time sequence and the time continuity between frames, and the active frame sequence in the active frame sequence group is selected one by one as the target active frame sequence. Select target active frames in the target active frame sequence one by one as reference active frames, and take the enhanced video frame in the enhanced video frame sequence that is before and closest to the reference active frame as the reference event frame. Perform block-level motion estimation on the reference active frame and the reference event frame to obtain motion vectors. Motion compensation and residual calculation are performed based on the motion vector, the reference active frame, and the reference event frame to obtain the residual pixel block sequence corresponding to the regular active frame; A quantization adjustment factor is calculated based on the strategy correction factor and resource redundancy. The residual pixel block sequence is then transformed, encoded, and quantized based on the quantization adjustment factor to obtain a quantized residual data sequence. The quantized residual data sequence and the corresponding motion vector are entropy encoded to obtain a residual encoded bit stream, and the residual encoded bit stream is then subjected to secondary encryption.

[0009] According to another preferred embodiment of the present invention, the step of performing motion compensation and residual calculation based on the motion vector, the reference active frame, and the reference event frame to obtain the residual pixel block sequence corresponding to the regular active frame includes: Based on the motion vector, motion compensation prediction is performed on the reference event frame to obtain the predicted activity frame; Pixel difference operations are performed between the predicted active frame and the reference active frame to obtain an inter-frame residual image; The inter-frame residual image is scanned and reconstructed using the motion vectors to obtain residual data blocks; All residual data blocks corresponding to the regular activity frame are constructed into a residual pixel block sequence.

[0010] According to another preferred embodiment of the present invention, the step of generating a hierarchical transmission policy based on the available network bandwidth and the policy correction factor includes: The network status level is mapped based on the available network bandwidth and a preset bandwidth threshold; The bandwidth allocation weight is extracted from the policy correction factor, and the encrypted hierarchical video frame group is hierarchically allocated bitrate based on the network status level and the bandwidth allocation weight to obtain hierarchical bitrate quota group. Priority and packet loss policy configurations are performed for the encrypted hierarchical video frame groups, and a hierarchical transmission policy is generated in combination with the hierarchical bitrate quota groups.

[0011] This invention provides a method for optimizing hierarchical encrypted transmission of big data based on edge-cloud collaboration, applied in the cloud, and includes the following steps: Receive the encrypted hierarchical video frame group uploaded by the edge terminal, perform bidirectional certificate verification and parallel decryption on the encrypted hierarchical video frame group to obtain the decrypted hierarchical video frame group. The decrypted hierarchical video frame group is subjected to residual decoding and image reconstruction to obtain a reconstructed surveillance video. Security feature digests are extracted from the key event frames of the reconstructed surveillance video and the security feature digests are uploaded to the blockchain. The reconstructed surveillance video is reconstructed and analyzed to obtain the reconstruction completeness. A strategy correction factor is generated based on the reconstruction completeness and then sent to the edge device.

[0012] According to another preferred embodiment of the present invention, the step of performing residual decoding and image reconstruction on the decrypted hierarchical video frame group to obtain the reconstructed surveillance video includes: The header file of the decrypted hierarchical video frame group is parsed to obtain key event frames, residual coded bitstreams and background frame features; The residual encoded bitstream is entropy decoded to obtain a quantized residual data sequence and a corresponding motion vector. The quantized residual data sequence is dequantized, and the motion vector is combined with the dequantized residual data sequence to perform an inverse transform operation to obtain a residual pixel block sequence. Based on the motion vector and key event frames, motion compensation prediction and residual superposition are performed on the residual pixel block sequence to obtain the reconstructed active frame; Based on the reconstructed activity frames and key event frames, the background frame features are upsampled at multiple levels to obtain the reconstructed background frame. The key event frames, reconstructed activity frames, and reconstructed background frames are then stitched together in timestamp order to form a reconstructed surveillance video.

[0013] To achieve at least one of the above-mentioned objectives, the present invention further provides a big data hierarchical encrypted transmission optimization system based on edge-cloud collaboration. The system is applied at the edge and includes a target recognition module, a video segmentation module, a hierarchical encryption module, and a hierarchical transmission module, wherein: The target recognition module performs target recognition on the acquired surveillance video to obtain high-value monitoring areas and area tags; The video segmentation module dynamically divides the monitoring video into key event frames, regular activity frames, and static background frames based on the high-value monitoring area, area label, real-time available network bandwidth, edge device computing power load, and policy correction factor sent from the cloud. The hierarchical encryption module performs first-level encryption on the key event frames, performs inter-frame residual coding and second-level encryption on the regular activity frames, extracts frame features from the static background frames, and aggregates them into an encrypted hierarchical video frame group. The hierarchical transmission module generates a hierarchical transmission strategy based on the available network bandwidth and the strategy correction factor, and transmits the encrypted hierarchical video frame group to the cloud according to the hierarchical transmission strategy.

[0014] This invention provides a big data hierarchical encrypted transmission optimization system based on edge-cloud collaboration. The system is applied in the cloud and includes a parallel decryption module, a screen reconstruction module, and a reconstruction analysis module, wherein: The parallel decryption module receives the encrypted hierarchical video frame group uploaded from the edge terminal, performs bidirectional certificate verification and parallel decryption on the encrypted hierarchical video frame group, and obtains the decrypted hierarchical video frame group. The image reconstruction module performs residual decoding and image reconstruction on the decrypted hierarchical video frame group to obtain a reconstructed surveillance video. It extracts a security feature digest from the key event frames of the reconstructed surveillance video and puts the security feature digest on the blockchain. The reconstruction analysis module performs reconstruction analysis on the reconstructed surveillance video to obtain the reconstruction completeness, generates a strategy correction factor based on the reconstruction completeness, and sends the strategy correction factor to the edge terminal.

[0015] (III) Beneficial Effects Compared with existing technologies, this invention provides a method and system for optimizing hierarchical encrypted transmission of big data based on edge-cloud collaboration, which has the following beneficial effects: This edge-cloud collaborative big data hierarchical encryption transmission optimization method automatically identifies key targets in the monitoring scene by performing video frame parsing, image preprocessing, and target detection on the monitoring video, generating corresponding high-value monitoring areas and area tags. By distinguishing areas containing key targets such as people, vehicles, or abnormal behavior from background areas, it provides a reliable semantic basis for subsequent video data hierarchical classification, encoding compression, and hierarchical encryption. By integrating video content features and system resource status, it achieves adaptive dynamic hierarchical classification of monitoring video frames. By using high-value monitoring areas and area tags to construct region of interest mapping information and calculating the importance score of video frames, it identifies high-value video frames containing key targets such as people, vehicles, or abnormal behavior. Combining real-time acquired network available bandwidth and edge device computing load, through data normalization and weighted correction, it transforms complex network fluctuations and hardware burdens into a single dynamic scoring threshold, achieving accurate filtering of the value of monitoring images. Through this hierarchical mechanism that combines content awareness and resource awareness, it can reduce the processing and transmission of irrelevant background data while ensuring the complete transmission of key information, thereby reducing network bandwidth consumption and edge computing pressure.

[0016] This edge-cloud collaborative big data hierarchical encryption transmission optimization method achieves synergistic optimization of video data compression efficiency and security protection capabilities by adopting differentiated processing strategies for different types of video frames. For critical event frames, a full-data encryption method is used to fully protect the key target area and its surrounding pixel information, thereby ensuring the security of important event data. For regular activity frames, inter-frame residuals are generated through block-level motion estimation and motion compensation prediction, and high-efficiency compression is achieved by combining discrete cosine transform, quantization processing, and entropy coding. At the same time, the quantization intensity is adaptively adjusted according to the strategy correction factor and resource surplus to improve the compression rate when resources are scarce, thereby reducing the transmission bit rate and computational burden. After encoding, the residual bit stream is subjected to secondary encryption to balance security and computational efficiency. While ensuring the secure transmission of critical information, the method significantly reduces the size of video data and improves the overall bandwidth utilization efficiency and edge computing efficiency of the system.

[0017] This edge-cloud collaborative big data hierarchical encrypted transmission optimization method determines the network status level by acquiring the available network bandwidth in real time and combining it with a preset bandwidth threshold. Based on this, it extracts bandwidth allocation weights according to a policy correction factor and performs hierarchical bitrate allocation for key event frames, regular activity frames, and static background frames, thereby constructing hierarchical bitrate quota groups. At the same time, it configures differentiated transmission priority scheduling strategies and packet loss control strategies for different types of video frames, so that key event frames can still obtain high bandwidth guarantees and be transmitted first even under network congestion, while regular activity frames and static background frames dynamically adjust transmission bandwidth and packet loss strategies according to network status. Through this hierarchical transmission mechanism, the real-time transmission of important event information can be prioritized when network resources are limited, while reducing the network bandwidth occupation of non-critical video data, improving the stability of video data transmission and bandwidth utilization efficiency, and significantly improving the transmission efficiency and system reliability of surveillance video in complex network environments. Attached Figure Description

[0018] Figure 1 The diagram shows a flowchart of the proposed method for optimizing the hierarchical encrypted transmission of big data based on edge-cloud collaboration when applied to the edge.

[0019] Figure 2 The diagram shows a flowchart of the proposed big data hierarchical encrypted transmission optimization method based on edge-cloud collaboration when applied to the cloud.

[0020] Figure 3 The diagram shown is a structural diagram of the big data hierarchical encrypted transmission optimization system based on edge-cloud collaboration of the present invention when applied to the edge.

[0021] Figure 4 The diagram shown is a structural diagram of the big data hierarchical encrypted transmission optimization system based on edge-cloud collaboration of the present invention when applied to the cloud. Detailed Implementation

[0022] The following description is intended to disclose the present invention and enable those skilled in the art to implement it. The preferred embodiments described below are merely examples, and other obvious modifications will occur to those skilled in the art. The basic principles of the invention defined in the following description can be applied to other embodiments, modifications, improvements, equivalents, and other technical solutions that do not depart from the spirit and scope of the invention.

[0023] It is understood that the term "a" should be understood as "at least one" or "one or more", that is, in one embodiment, the number of an element can be one, while in another embodiment, the number of the element can be multiple, and the term "a" should not be understood as a limitation on the number.

[0024] Example 1: Please combine Figure 1 This invention discloses a hierarchical encrypted transmission optimization method for big data based on edge-cloud collaboration. The method is applied at the edge and includes the following steps: S1. Perform target identification on the acquired surveillance video to obtain high-value surveillance areas and area labels.

[0025] The surveillance video refers to video acquired by edge video acquisition devices deployed in the surveillance scene. These devices are used to monitor and record personnel activities, vehicle traffic, and environmental conditions in real time, thereby achieving security protection, event evidence collection, and abnormal behavior detection. For example, they can be used in scenarios such as park surveillance, urban road surveillance, and shopping mall surveillance. The Region of Interest (ROI) refers to an image area in the surveillance video that contains important target objects, such as the areas corresponding to people and vehicles. The region labels refer to the labels obtained after dividing the surveillance video into regions and semantically classifying them. For example, region labels include personnel areas, vehicle areas, suspicious items, abnormal behavior, and background areas.

[0026] Specifically, the step of performing target identification on the acquired surveillance video to obtain high-value surveillance areas and area tags includes: The acquired surveillance video is split into a video frame sequence, and the video frame sequence is subjected to image denoising and image enhancement to obtain an enhanced video frame sequence. Target detection is performed on the enhanced video frame sequence to obtain target detection boxes and target object semantics; Non-maximum suppression and confidence filtering are applied to the target detection box to obtain a standard detection box, and the standard object semantics corresponding to the standard detection box are selected from the target object semantics. The enhanced video frame sequence is divided into regions based on the standard detection box to obtain target regions. Region labels corresponding to the target regions are generated according to the standard object semantics. High-value monitoring regions are then selected from the target regions based on the region labels.

[0027] The process involves using FFmpeg or OpenCV libraries to split the surveillance video into a sequence of video frames; employing nonlocal mean filtering or median filtering algorithms for image denoising; and enhancing the image through resolution unification, brightness normalization, or Gamma filtering. Lightweight YOLO models or MobileNet-SSD models can be used for target detection to reduce computational overhead at the edges. Region segmentation can be achieved using edge filtering or watershed algorithms, with standard bounding boxes providing initial localization. Edge filtering or watershed algorithms are used to achieve finer pixel-level contour segmentation. The target regions are the various image regions in the enhanced video frame sequence after region segmentation. Selecting high-value surveillance regions refers to identifying target regions labeled as personnel areas, vehicle areas, suspicious items, and abnormal behaviors as high-value surveillance regions.

[0028] By performing video frame parsing, image preprocessing, and target detection on surveillance videos, the system automatically identifies key targets in the monitored scene and generates corresponding high-value monitoring areas and area labels. Optimizing the detection results through non-maximum suppression and confidence filtering effectively reduces duplicate detection boxes and low-confidence targets, thereby improving the accuracy and stability of target recognition. By distinguishing areas containing key targets such as people, vehicles, or abnormal behavior from background areas, the system provides a reliable semantic basis for subsequent video data grading, encoding compression, and tiered encryption, achieving semantic understanding of the surveillance video content and enabling the system to prioritize the processing and transmission of data containing important information.

[0029] S2. Based on the high-value monitoring area, area label, real-time available network bandwidth, edge device computing load, and policy correction factor sent from the cloud, the monitoring video is dynamically divided into key event frames, regular activity frames, and static background frames.

[0030] Specifically, the available network bandwidth can be determined through data flow information from the Transmission Control Protocol (TCP) between the edge and the cloud. For example, the available network bandwidth can be obtained by using statistics on the TCP congestion window, round-trip time, and packet loss rate. Available bandwidth refers to the effective network bandwidth used for transmitting video data between the edge and the cloud. The real-time available network bandwidth is affected by factors such as real-time signal quality, network congestion, link quality, and cross-network scheduling between the edge and the cloud. The computing load of the edge device can be obtained in real time by using the system interface or hardware monitoring program of the edge device to read data such as CPU utilization, GPU utilization, and free memory. The policy correction factor is a dynamic parameter package fed back by the cloud in real time, including bandwidth allocation weight and hierarchical decision weight, which is used to adjust the hierarchical decision and data transmission strategy of the edge device online.

[0031] Specifically, the step of dynamically dividing the surveillance video into key event frames, regular activity frames, and static background frames based on the high-value monitoring area, area tag, real-time acquired network available bandwidth, edge device computing power load, and policy correction factor sent from the cloud includes: Interest region mapping information is generated based on the high-value monitoring area and the area label, and the importance of each enhanced video frame in the enhanced video frame sequence is scored based on the interest region mapping information to obtain a frame importance sequence. The available network bandwidth and edge device computing load are normalized and weighted summed to obtain the resource occupancy, and the resource surplus is calculated based on the resource occupancy. The resource sufficiency is scored and weighted using the strategy correction factor to obtain a dynamic scoring threshold. The dynamic scoring threshold is compared with the frame importance sequence, and each enhanced video frame in the enhanced video frame sequence is divided into three-level frame types according to the comparison result, resulting in key event frames, regular activity frames, and static background frames.

[0032] The generation of region of interest mapping information refers to generating corresponding structural information for high-value monitoring regions and region labels in each enhanced video frame. For example, the structure is {frame number | number of high-value monitoring regions | region label | area percentage of region of interest}. The importance scoring refers to scoring each enhanced video frame according to a preset weight based on the number of high-value monitoring regions, region labels, and area percentage of region of interest. For example, the weight of personnel regions is 0.8, the weight of vehicle regions is 0.6, and the weight of abnormal behavior is 1.0. The weights of each category in the number of high-value monitoring regions, region labels, and area percentage of region of interest are weighted and summed to obtain the frame importance. The data normalization... This refers to normalizing the available network bandwidth and edge device computing load to a preset numerical range for easier standardization, such as normalizing 20Mbps of available network bandwidth to 0.4. The sum of resource surplus and resource occupancy is 1. The threshold score refers to mapping the resource surplus to the corresponding importance threshold range. For example, when the resource surplus quantification index is 0.6, the corresponding importance threshold range is: 0-0.3 for static background frames, 0.3-0.7 for regular activity frames, and 0.7-1 for critical event frames. Weighted correction refers to correcting the importance threshold range according to the hierarchical decision weights in the strategy correction factor. The corresponding three-level frame type classification is shown in the table below. By integrating video content features with system resource status, adaptive dynamic grading of surveillance video frames is achieved. Interest region mapping information is constructed using high-value monitoring areas and region tags, and the importance score of video frames is calculated accordingly. This identifies high-value video frames containing key targets such as people, vehicles, or abnormal behavior. Combining real-time acquired network bandwidth and edge device computing load, data normalization and weighted correction transform complex network fluctuations and hardware burdens into a single dynamic scoring threshold, enabling precise filtering of the value of surveillance footage. This content-aware and resource-aware grading mechanism reduces the processing and transmission of irrelevant background data while ensuring the complete transmission of critical information, thereby reducing network bandwidth consumption and edge computing pressure. It also provides a foundation for subsequent video grading encoding, grading encryption, and transmission optimization.

[0033] S3. Perform first-level encryption on the key event frames, perform inter-frame residual coding and second-level encryption on the regular activity frames, extract frame features from the static background frames, and aggregate them into an encrypted hierarchical video frame group.

[0034] The first-level encryption refers to converting key event frames into pixel data and performing full SM4-CTR encryption on the pixel data. The frame feature extraction refers to performing multi-level downsampling on the static background frame and concatenating the region label and timestamp corresponding to the static background frame as metadata with the multi-level downsampled static background frame to obtain background frame features.

[0035] Specifically, the inter-frame residual coding and secondary encryption of the regular activity frames include: The regular active frames are divided into active frame sequence groups according to the time sequence and the time continuity between frames, and the active frame sequence in the active frame sequence group is selected one by one as the target active frame sequence. Select target active frames in the target active frame sequence one by one as reference active frames, and take the enhanced video frame in the enhanced video frame sequence that is before and closest to the reference active frame as the reference event frame. Perform block-level motion estimation on the reference active frame and the reference event frame to obtain motion vectors. Motion compensation and residual calculation are performed based on the motion vector, the reference active frame, and the reference event frame to obtain the residual pixel block sequence corresponding to the regular active frame; A quantization adjustment factor is calculated based on the strategy correction factor and resource redundancy. The residual pixel block sequence is then transformed, encoded, and quantized based on the quantization adjustment factor to obtain a quantized residual data sequence. The quantized residual data sequence and the corresponding motion vector are entropy encoded to obtain a residual encoded bit stream, and the residual encoded bit stream is then subjected to secondary encryption.

[0036] In this context, each active frame sequence in the active frame sequence group corresponds to a consecutive active frame sequence segment in the target active frame sequence. When only a key event frame exists in the enhanced video frame sequence before the reference active frame, the nearest enhanced video frame is selected as the reference event frame. If a regular active frame is closer than the key event frame, the corresponding regular active frame is used as the reference event frame. The block-level motion estimation refers to dividing the reference active frame and the reference event frame into several non-overlapping pixel blocks and performing block matching based on the minimum pixel difference criterion, using the displacement bias between the center points of the two most matched pixel blocks as the motion vector. The quantization adjustment factor is a weighted sum of the strategy correction factor and resource redundancy. The transform coding can be discrete cosine transform. The quantization processing refers to adjusting the quantization compensation during quantization processing through the quantization adjustment factor. When resources are scarce, the adjustment factor increases, the quantization step size increases accordingly, and more subtle residuals are forced to zero, thereby significantly reducing the bit rate. Context-based adaptive binary arithmetic coding can be used. Entropy coding is performed using Coding (CABAC) or adaptive arithmetic coding, and the secondary encryption refers to the use of non-full SM4-CTR encryption.

[0037] Specifically, the step of performing motion compensation and residual calculation based on the motion vector, the reference active frame, and the reference event frame to obtain the residual pixel block sequence corresponding to the regular active frame includes: Based on the motion vector, motion compensation prediction is performed on the reference event frame to obtain the predicted activity frame; Pixel difference operations are performed between the predicted active frame and the reference active frame to obtain an inter-frame residual image; The inter-frame residual image is scanned and reconstructed using the motion vectors to obtain residual data blocks; All residual data blocks corresponding to the regular activity frame are constructed into a residual pixel block sequence.

[0038] The motion compensation refers to reconstructing the corresponding pixel blocks of the reference event frame by displacement based on motion vectors, thereby forming a predicted activity frame; the scan recombination refers to rearranging the two-dimensional residual blocks into a one-dimensional residual coefficient sequence according to a preset scan path, which can be performed by a zigzag scan.

[0039] By employing differentiated processing strategies for different types of video frames, a synergistic optimization of video data compression efficiency and security protection capabilities is achieved. For critical event frames, a full-scale encryption method is used to fully protect the key target area and its surrounding pixel information, thereby ensuring the security of important event data. For regular activity frames, inter-frame residuals are generated through block-level motion estimation and motion compensation prediction, and combined with discrete cosine transform, quantization processing, and entropy coding to achieve efficient compression. At the same time, the quantization intensity is adaptively adjusted according to the strategy correction factor and resource surplus to improve the compression rate when resources are scarce, thereby reducing the transmission bit rate and computational burden. After encoding, the residual bit stream is subjected to secondary encryption to balance security and computational efficiency. While ensuring the secure transmission of critical information, the system significantly reduces the size of video data and improves the overall bandwidth utilization efficiency and edge computing efficiency.

[0040] S4. Generate a hierarchical transmission strategy based on the available network bandwidth and the strategy correction factor, and transmit the encrypted hierarchical video frame group to the cloud according to the hierarchical transmission strategy.

[0041] The step of generating a tiered transmission strategy based on the available network bandwidth and the strategy correction factor includes: The network status level is mapped based on the available network bandwidth and a preset bandwidth threshold; The bandwidth allocation weight is extracted from the policy correction factor, and the encrypted hierarchical video frame group is hierarchically allocated bitrate based on the network status level and the bandwidth allocation weight to obtain hierarchical bitrate quota group. Priority and packet loss policy configurations are performed for the encrypted hierarchical video frame groups, and a hierarchical transmission policy is generated in combination with the hierarchical bitrate quota groups.

[0042] The mapping of network status levels refers to mapping different levels of network status based on the proportion of available network bandwidth within the bandwidth threshold. For example, the network status levels may include congested status, normal status, and relaxed status, with different network status levels corresponding to different bandwidth allocation strategies. For example, as shown in the table above, when the network status level is congested, 55% bandwidth is reserved for critical event frames, 40% for regular activity frames, and 5% for static background frames; when the network status level is normal, 50% bandwidth is reserved for critical event frames, 40% for regular activity frames, and 10% for static background frames; when the network status level is relaxed, 30% bandwidth is reserved for critical event frames, 50% for regular activity frames, and 20% for static background frames. The bandwidth allocation weight refers to the weight assigned to each of the three types of video frames (critical event frames, regular activity frames, and static background frames) after encryption, and the sum of the weights for these three types of video frames is 1. Tiered bitrate allocation refers to... The bandwidth allocation strategy uses weighted calculations to weight the bandwidth proportion of each type of video frame in the bandwidth allocation strategy. Priority configuration refers to assigning different transmission priorities to critical event frames, regular activity frames, and static background frames. For example, critical event frames have the highest transmission priority and are subject to strict priority scheduling to ensure low latency; regular activity frames have a medium transmission priority and are subject to weighted fair queue scheduling, allowing the use of idle bandwidth within a certain range; static background frames have the lowest transmission priority and are transmitted only when there is remaining bandwidth. The packet loss strategy, for example, does not allow packet loss for critical event frames, while static background frames are allowed to be directly discarded when bandwidth is insufficient.

[0043] By acquiring real-time available network bandwidth and combining it with preset bandwidth thresholds to determine network status levels, and then extracting bandwidth allocation weights based on policy correction factors, hierarchical bitrate allocation is performed on key event frames, regular activity frames, and static background frames to construct hierarchical bitrate quota groups. Simultaneously, differentiated transmission priority scheduling strategies and packet loss control strategies are configured for different types of video frames. This ensures that key event frames receive high bandwidth guarantees and are transmitted with priority even under network congestion, while regular activity frames and static background frames have their transmission bandwidth and packet loss strategies dynamically adjusted according to network status. Through this hierarchical transmission mechanism, real-time transmission of important event information can be prioritized when network resources are limited, while reducing the bandwidth consumption of non-critical video data. This improves the stability and bandwidth utilization efficiency of video data transmission, significantly enhancing the transmission efficiency and system reliability of surveillance video in complex network environments.

[0044] Please combine Figure 2 This invention discloses a hierarchical encrypted transmission optimization method for big data based on edge-cloud collaboration. The method is applied in the cloud and includes the following steps: S21. Receive the encrypted hierarchical video frame group uploaded by the edge terminal, perform bidirectional certificate verification and parallel decryption on the encrypted hierarchical video frame group to obtain the decrypted hierarchical video frame group.

[0045] The two-way certificate verification refers to the process where, when communicating and transmitting data between the edge and the cloud, the edge needs to verify the cloud's digital certificate, and the cloud also needs to verify the edge's digital certificate. This can be achieved by the edge and the cloud each sending a CA-signed digital certificate. The parallel decryption refers to the use of the cloud's multi-core processor to simultaneously decrypt multiple data blocks in the encrypted hierarchical video frame group. The decryption algorithm can be SM4-CTR decryption.

[0046] S22. Perform residual decoding and image reconstruction on the decrypted hierarchical video frame group to obtain the reconstructed surveillance video. Extract the security feature summary from the key event frames of the reconstructed surveillance video and put the security feature summary on the blockchain.

[0047] In detail, the residual decoding and image reconstruction of the decrypted hierarchical video frame group to obtain the reconstructed surveillance video includes: The header file of the decrypted hierarchical video frame group is parsed to obtain key event frames, residual coded bitstreams and background frame features; The residual encoded bitstream is entropy decoded to obtain a quantized residual data sequence and a corresponding motion vector. The quantized residual data sequence is dequantized, and the motion vector is combined with the dequantized residual data sequence to perform an inverse transform operation to obtain a residual pixel block sequence. Based on the motion vector and key event frames, motion compensation prediction and residual superposition are performed on the residual pixel block sequence to obtain the reconstructed active frame; Based on the reconstructed activity frames and key event frames, the background frame features are upsampled at multiple levels to obtain the reconstructed background frame. The key event frames, reconstructed activity frames, and reconstructed background frames are then stitched together in timestamp order to form a reconstructed surveillance video.

[0048] Among them, entropy decoding, inverse quantization, and inverse transform operation are the inverse steps of entropy encoding, quantization, and transform encoding, respectively; multi-level upsampling refers to selecting the video frame closest to the feature timestamp of the background frame to be reconstructed from the reconstructed activity frame and key event frame as the reference frame, and performing multi-level upsampling and texture compensation on the background frame feature based on the reference frame; extracting security feature digest refers to using algorithms such as SHA-256 to perform hash digest calculation on the key event frame to generate security feature digest; blockchain on-chain refers to constructing a data structure containing security feature digest, edge device unique identifier, timestamp, and regional label, signing the data structure with a private key in the cloud, and broadcasting the signed data structure to the blockchain network.

[0049] By parsing the header files of the decrypted hierarchical video frame groups, key event frames, residual encoded bitstreams, and background frame features are separated. The residual pixel block sequence is then recovered through inverse processes such as entropy decoding, inverse quantization, and inverse transformation. Simultaneously, motion compensation prediction is performed on the key event frames using motion vectors, and the residuals are superimposed to reconstruct the content of the active frames. By using the key event frames and the reconstructed active frames as reference frames, the background frame features are upsampled and recovered at multiple levels to reconstruct the background image. Finally, the key event frames, reconstructed active frames, and reconstructed background frames are stitched together according to the timestamp order to generate a complete reconstructed surveillance video. By extracting security feature summaries from the key event frames in the reconstructed surveillance video and storing them on the blockchain, the reliable recording and tamper-proof preservation of key event information can be achieved.

[0050] S23. Perform reconstruction analysis on the reconstructed surveillance video to obtain the reconstruction completeness, generate a strategy correction factor based on the reconstruction completeness, and send the strategy correction factor to the edge terminal.

[0051] Specifically, the process of reconstructing and analyzing the reconstructed surveillance video to obtain the reconstruction completeness includes: A compliance analysis is performed on the residual encoded bitstream corresponding to the reconstructed surveillance video to obtain the decoding health score; Smoothness analysis is performed on the motion vectors of the reconstructed active frames in the reconstructed surveillance video to obtain the reconstruction smoothness. Edge block effect analysis was performed on the reconstructed surveillance video to obtain the quantized distortion. The reconstruction integrity is obtained by weighting the decoding health, reconstruction smoothness, and quantization distortion.

[0052] The compliance analysis refers to the proportion of grammatically valid codewords and the proportion of complete sequence parameter sets in the statistical residual encoded bitstream; the smoothness analysis refers to the calculation of the smoothness of motion vectors between consecutive adjacent regular activity frames, which can be determined by calculating the standard deviation of the adjacent differences of motion vectors; the edge block effect analysis refers to the statistical analysis of the reconstructed surveillance video in the discrete cosine transform domain, and the calculation of the proportion of zero coefficients and the energy distribution of AC coefficients in the reconstructed surveillance video in the discrete cosine transform domain to obtain the quantization distortion.

[0053] In detail, a strategy correction factor can be generated using a preset decision tree model or a multilayer perceptron model. The strategy correction factor includes a hierarchical decision weight for adjusting the hierarchical decision at the edge and a bandwidth allocation weight for adjusting the data transmission strategy. For example, when the reconstruction completeness is less than a preset completeness threshold, the hierarchical decision weight between key event frames and regular activity frames is increased, and the bandwidth allocation weight between key event frames and regular activity frames is enhanced.

[0054] By performing multi-dimensional quality analysis on the reconstructed surveillance video, an objective assessment of the reliability of video reconstruction is achieved. Compliance analysis of the residual encoded bitstream yields decoding health, effectively detecting bitstream corruption or decoding anomalies during transmission. Smoothness analysis of motion vectors between adjacent reconstructed active frames determines reconstruction smoothness, identifying abrupt changes in motion prediction between video frames and reflecting the temporal continuity of the video. Frequency domain block effect analysis of the reconstructed surveillance video yields quantization distortion, assessing the degree of quantization distortion generated during video compression. A strategy correction factor is generated based on the reconstruction integrity, dynamically adjusting the video frame grading strategy and bandwidth allocation strategy at the edge. This enables the system to adaptively optimize front-end encoding and transmission strategies based on cloud reconstruction quality, improving the reliability of critical event information transmission and overall video transmission efficiency.

[0055] Example 2: Please combine Figure 3 This invention discloses a big data hierarchical encrypted transmission optimization system based on edge-cloud collaboration. The system is applied at the edge and includes a target recognition module, a video segmentation module, a hierarchical encryption module, and a hierarchical transmission module, wherein: The target recognition module performs target recognition on the acquired surveillance video to obtain high-value monitoring areas and area tags; The video segmentation module dynamically divides the monitoring video into key event frames, regular activity frames, and static background frames based on the high-value monitoring area, area label, real-time available network bandwidth, edge device computing power load, and policy correction factor sent from the cloud. The hierarchical encryption module performs first-level encryption on the key event frames, performs inter-frame residual coding and second-level encryption on the regular activity frames, extracts frame features from the static background frames, and aggregates them into an encrypted hierarchical video frame group. The hierarchical transmission module generates a hierarchical transmission strategy based on the available network bandwidth and the strategy correction factor, and transmits the encrypted hierarchical video frame group to the cloud according to the hierarchical transmission strategy.

[0056] Please combine Figure 4 This invention discloses a big data hierarchical encrypted transmission optimization system based on edge-cloud collaboration. The system is applied in the cloud and includes a parallel decryption module, a screen reconstruction module, and a reconstruction analysis module, wherein: The parallel decryption module receives the encrypted hierarchical video frame group uploaded from the edge terminal, performs bidirectional certificate verification and parallel decryption on the encrypted hierarchical video frame group, and obtains the decrypted hierarchical video frame group. The image reconstruction module performs residual decoding and image reconstruction on the decrypted hierarchical video frame group to obtain a reconstructed surveillance video. It extracts a security feature digest from the key event frames of the reconstructed surveillance video and puts the security feature digest on the blockchain. The reconstruction analysis module performs reconstruction analysis on the reconstructed surveillance video to obtain the reconstruction completeness, generates a strategy correction factor based on the reconstruction completeness, and sends the strategy correction factor to the edge terminal.

[0057] The processes described above with reference to the flowcharts in the embodiments disclosed in this invention can be implemented as computer software programs. The embodiments disclosed in this invention include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication component, and / or installed from a removable medium. When the computer program is executed by a central processing unit (CPU), it performs the functions defined in the methods of this application. It should be noted that the computer-readable medium described above in this application can be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wire segments, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this application, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in connection with an instruction execution system, apparatus, or device. In this application, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can also be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on a computer-readable medium may be transmitted using any suitable medium, including but not limited to: wireless segments, wire segments, optical fibers, RF, etc., or any suitable combination thereof.

[0058] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0059] Those skilled in the art should understand that the embodiments of the present invention described above and shown in the accompanying drawings are merely examples and do not limit the present invention. The purpose of the present invention has been fully and effectively achieved. The functions and structural principles of the present invention have been shown and explained in the embodiments. Without departing from the stated principles, the implementation of the present invention may have any variations or modifications.

Claims

1. A method for optimizing hierarchical encrypted transmission of big data based on edge-cloud collaboration, characterized in that, The method is applied to the edge end and includes: Target identification is performed on the acquired surveillance video to obtain high-value surveillance areas and area tags; Based on the high-value monitoring area, area label, real-time available network bandwidth, edge device computing power load, and policy correction factor sent from the cloud, the monitoring video is dynamically divided into key event frames, regular activity frames, and static background frames. The key event frames are encrypted at level one, the regular activity frames are encrypted with inter-frame residual coding and level two encryption, the static background frames are extracted with frame features, and the frames are then assembled into encrypted hierarchical video frame groups. A tiered transmission strategy is generated based on the available network bandwidth and the strategy correction factor, and the encrypted tiered video frame group is transmitted to the cloud according to the tiered transmission strategy.

2. The method for optimizing hierarchical encrypted transmission of big data based on edge-cloud collaboration according to claim 1, characterized in that, The step of performing target identification on the acquired surveillance video to obtain high-value surveillance areas and area tags includes: The acquired surveillance video is split into a video frame sequence, and the video frame sequence is subjected to image denoising and image enhancement to obtain an enhanced video frame sequence. Target detection is performed on the enhanced video frame sequence to obtain target detection boxes and target object semantics; Non-maximum suppression and confidence filtering are applied to the target detection box to obtain a standard detection box, and the standard object semantics corresponding to the standard detection box are selected from the target object semantics. The enhanced video frame sequence is divided into regions based on the standard detection box to obtain target regions. Region labels corresponding to the target regions are generated according to the standard object semantics. High-value monitoring regions are then selected from the target regions based on the region labels.

3. The method for optimizing hierarchical encrypted transmission of big data based on edge-cloud collaboration according to claim 1, characterized in that, The process of dynamically dividing the surveillance video into key event frames, regular activity frames, and static background frames based on the high-value monitoring area, area tag, real-time available network bandwidth, edge device computing load, and cloud-sent policy correction factor includes: Interest region mapping information is generated based on the high-value monitoring area and the area label, and the importance of each enhanced video frame in the enhanced video frame sequence is scored based on the interest region mapping information to obtain a frame importance sequence. The available network bandwidth and edge device computing load are normalized and weighted summed to obtain the resource occupancy, and the resource surplus is calculated based on the resource occupancy. The resource sufficiency is scored and weighted using the strategy correction factor to obtain a dynamic scoring threshold. The dynamic scoring threshold is compared with the frame importance sequence, and each enhanced video frame in the enhanced video frame sequence is divided into three-level frame types according to the comparison result, resulting in key event frames, regular activity frames, and static background frames.

4. The method for optimizing hierarchical encrypted transmission of big data based on edge-cloud collaboration according to claim 3, characterized in that, The process of performing inter-frame residual coding and secondary encryption on the regular activity frames includes: The regular active frames are divided into active frame sequence groups according to the time sequence and the time continuity between frames, and the active frame sequence in the active frame sequence group is selected one by one as the target active frame sequence. Select target active frames in the target active frame sequence one by one as reference active frames, and take the enhanced video frame in the enhanced video frame sequence that is before and closest to the reference active frame as the reference event frame. Perform block-level motion estimation on the reference active frame and the reference event frame to obtain motion vectors. Motion compensation and residual calculation are performed based on the motion vector, the reference active frame, and the reference event frame to obtain the residual pixel block sequence corresponding to the regular active frame; Based on the strategy correction factor and resource surplus, a quantization adjustment factor is calculated. Based on the quantization adjustment factor, the residual pixel block sequence is transformed, encoded and quantized to obtain a quantized residual data sequence. The quantized residual data sequence and the corresponding motion vector are entropy encoded to obtain a residual encoded bit stream, and the residual encoded bit stream is then subjected to secondary encryption.

5. The method for optimizing hierarchical encrypted transmission of big data based on edge-cloud collaboration according to claim 4, characterized in that, Motion compensation and residual calculations are performed based on the motion vector, the reference active frame, and the reference event frame to obtain the residual pixel block sequence corresponding to the regular active frame, including: Based on the motion vector, motion compensation prediction is performed on the reference event frame to obtain the predicted activity frame; Pixel difference operations are performed between the predicted active frame and the reference active frame to obtain an inter-frame residual image; The inter-frame residual image is scanned and reconstructed using the motion vectors to obtain residual data blocks; All residual data blocks corresponding to the regular activity frame are constructed into a residual pixel block sequence.

6. The method for optimizing hierarchical encrypted transmission of big data based on edge-cloud collaboration according to claim 5, characterized in that, The generation of a tiered transmission strategy based on the available network bandwidth and the policy correction factor includes: The network status level is mapped based on the available network bandwidth and a preset bandwidth threshold; The bandwidth allocation weight is extracted from the policy correction factor, and the encrypted hierarchical video frame group is hierarchically allocated bitrate based on the network status level and the bandwidth allocation weight to obtain hierarchical bitrate quota group. Priority and packet loss policy configurations are performed for the encrypted hierarchical video frame groups, and a hierarchical transmission policy is generated in combination with the hierarchical bitrate quota groups.

7. A method for optimizing hierarchical encrypted transmission of big data based on edge-cloud collaboration, characterized in that, The method is applied in the cloud and includes: Receive the encrypted hierarchical video frame group uploaded from the edge terminal, perform bidirectional certificate verification and parallel decryption on the encrypted hierarchical video frame group to obtain the decrypted hierarchical video frame group. The decrypted hierarchical video frame group is subjected to residual decoding and image reconstruction to obtain a reconstructed surveillance video. Security feature digests are extracted from the key event frames of the reconstructed surveillance video and the security feature digests are uploaded to the blockchain. The reconstructed surveillance video is reconstructed and analyzed to obtain the reconstruction completeness. A strategy correction factor is generated based on the reconstruction completeness and then sent to the edge device.

8. The method for optimizing hierarchical encrypted transmission of big data based on edge-cloud collaboration according to claim 7, characterized in that, The process of performing residual decoding and image reconstruction on the decrypted hierarchical video frame group to obtain the reconstructed surveillance video includes: The header file of the decrypted hierarchical video frame group is parsed to obtain key event frames, residual coded bitstreams and background frame features; The residual encoded bitstream is entropy decoded to obtain a quantized residual data sequence and a corresponding motion vector. The quantized residual data sequence is dequantized, and the motion vector is combined with the dequantized residual data sequence to perform an inverse transform operation to obtain a residual pixel block sequence. Based on the motion vector and key event frames, motion compensation prediction and residual superposition are performed on the residual pixel block sequence to obtain the reconstructed active frame; Based on the reconstructed activity frames and key event frames, the background frame features are upsampled at multiple levels to obtain the reconstructed background frame. The key event frames, reconstructed activity frames, and reconstructed background frames are then stitched together in timestamp order to form a reconstructed surveillance video.

9. A hierarchical encrypted transmission optimization system for big data based on edge-cloud collaboration, characterized in that, The system is applied at the edge and includes a target recognition module, a video segmentation module, a hierarchical encryption module, and a hierarchical transmission module, wherein: The target recognition module performs target recognition on the acquired surveillance video to obtain high-value monitoring areas and area tags; The video segmentation module dynamically divides the monitoring video into key event frames, regular activity frames, and static background frames based on the high-value monitoring area, area label, real-time available network bandwidth, edge device computing power load, and policy correction factor sent from the cloud. The hierarchical encryption module performs first-level encryption on the key event frames, performs inter-frame residual coding and second-level encryption on the regular activity frames, extracts frame features from the static background frames, and aggregates them into an encrypted hierarchical video frame group. The hierarchical transmission module generates a hierarchical transmission strategy based on the available network bandwidth and the strategy correction factor, and transmits the encrypted hierarchical video frame group to the cloud according to the hierarchical transmission strategy.

10. A hierarchical encrypted transmission optimization system for big data based on edge-cloud collaboration, characterized in that, The system is applied in the cloud and includes a parallel decryption module, a screen reconstruction module, and a reconstruction analysis module, wherein: The parallel decryption module receives the encrypted hierarchical video frame group uploaded from the edge terminal, performs bidirectional certificate verification and parallel decryption on the encrypted hierarchical video frame group, and obtains the decrypted hierarchical video frame group. The image reconstruction module performs residual decoding and image reconstruction on the decrypted hierarchical video frame group to obtain a reconstructed surveillance video. It extracts a security feature digest from the key event frames of the reconstructed surveillance video and puts the security feature digest on the blockchain. The reconstruction analysis module performs reconstruction analysis on the reconstructed surveillance video to obtain the reconstruction completeness, generates a strategy correction factor based on the reconstruction completeness, and sends the strategy correction factor to the edge terminal.

Images