Internet of Things video monitoring data lightweight processing method based on artificial intelligence and edge computing

Abstract

The invention discloses an Internet of Things video monitoring data lightweight processing method based on artificial intelligence and edge computing. The problems that an existing monitoring system is high in bandwidth occupation, insufficient in edge computing power, poor in video compression and detection collaboration and the like are solved. The method comprises the steps that an edge end collects a video, and noise self-adaptive suppression and computing power-bandwidth cooperative resolution adaptation preprocessing are carried out on the video; the format is converted into an improved YUV4: 2: 0 format, and AV1 coding compression is optimized; edge target detection is realized in combination with a lightweight CNN, and key frames are screened in a linkage manner; the cloud end carries out space-time joint up-sampling frame insertion and quality restoration on the key frame; the terminal generates a multi-resolution video stream, and constructs an intelligent data slice and a lightweight AI analysis interface. The method reduces resource consumption, improves video quality and detection precision, adapts to multiple scenes and terminals, and is suitable for various Internet of Things monitoring scenes.

Description

Technical Field

[0001] This invention belongs to the interdisciplinary field of IoT video surveillance and intelligent data processing, specifically relating to a lightweight processing method for IoT video surveillance data that integrates artificial intelligence and edge computing technologies. It aims to address core technical pain points in IoT monitoring systems, such as high bandwidth consumption due to massive video data transmission, low processing efficiency due to limited computing power of edge devices, and poor coordination between video compression and target detection, thereby achieving efficient lightweight processing and high-quality application of monitoring data. Background Technology

[0002] With the rapid popularization of IoT technology, video surveillance systems have become core sensing infrastructure in fields such as industrial production, public safety, and smart cities. Deployment scale continues to expand, the number of surveillance cameras is exploding, and the resulting video data volume is also rising sharply. However, the current IoT video surveillance data processing system faces significant technical bottlenecks, making it difficult to meet practical application needs. On the one hand, traditional monitoring systems mostly adopt an edge acquisition and cloud-centralized processing architecture. The edge is only responsible for the acquisition and transmission of raw video data, without preprocessing or optimizing the data. Massive amounts of raw video streams directly occupy limited IoT communication bandwidth, easily causing network congestion and significantly increasing the data reception and storage pressure on the cloud. Especially in industrial scenarios, the dynamic noise generated by equipment operation further increases the video data volume, exacerbating bandwidth consumption. On the other hand, existing video compression technologies mostly use fixed bitrate and quantization parameter strategies, without adaptive optimization based on the content characteristics of the surveillance video. This results in compressed videos either losing dynamic target details or still having high redundancy in static background data, making it difficult to achieve a balance between high compression ratio and high quality preservation. Meanwhile, edge computing devices typically have limited computing resources. Traditional target detection models are large and time-consuming to infer, making real-time detection at the edge impossible. If the detection task is entirely delegated to the cloud, data transmission delays will affect the real-time performance of monitoring. Furthermore, the lack of coordination between detection results and video encoding processes prevents the optimization of encoding strategies based on detection information to further reduce data volume. In addition, when the cloud processes sparse keyframes transmitted from the edge, traditional upsampling and quality restoration methods are insufficient to compensate for missing information between frames, easily leading to problems such as blurry images and color distortion, resulting in insufficient video quality acquired by terminal applications. Moreover, terminal devices are diverse, with significant differences in hardware resolution and network conditions. Existing systems lack dynamic adaptation mechanisms for different terminals and cannot provide highly adaptable video streams and intelligent analysis interfaces for various terminals, thus hindering the realization of the practical value of IoT monitoring data.

[0003] Therefore, this invention proposes a lightweight processing method and system for IoT video surveillance data based on artificial intelligence and edge computing to solve the above problems. Summary of the Invention

[0004] The purpose of this section is to outline some aspects of embodiments of the present invention and to briefly describe some preferred embodiments. Simplifications or omissions may be made in this section, as well as in the abstract and title of this application, to avoid obscuring the purpose of these documents; however, such simplifications or omissions should not be construed as limiting the scope of the invention.

[0005] In view of the problems existing in the above and / or prior art, the present invention is proposed.

[0006] Therefore, the purpose of this invention is to overcome the shortcomings of the prior art and provide a lightweight processing method for IoT video surveillance data based on artificial intelligence and edge computing. The method comprises the following steps: acquiring raw RGB video data at the edge, and sequentially performing preprocessing including multi-scene noise adaptive suppression and computing power-bandwidth collaborative resolution adaptation; converting the preprocessed RGB data to an improved YUV4:2:0 format, completing color space adaptive conversion and adaptive weighted chromaticity downsampling; performing optimized AV1 encoding compression on the YUV4:2:0 data, including content-aware variable bitrate control and improved CfL quantization parameter optimization; achieving lightweight target detection at the edge based on a convolutional neural network, completing the collaborative input construction of bitstream features and detection models, model structure optimization and quantization training, and keyframe selection linked to detection results and encoding; receiving keyframes and detection results at the cloud, performing collaborative preprocessing, improved CNN spatiotemporal joint upsampling frame interpolation, and luminance-chromaticity joint quality restoration; and adapting and restoring the video stream at the terminal, generating multi-resolution video streams, constructing intelligent data slices and indexes, and designing a lightweight AI analysis interface that supports invocation.

[0007] Preferably, the edge-end video data acquisition and preprocessing steps specifically include: multi-scene adaptive noise suppression, and pixel point calculation. 5×5 neighborhood gray mean With variance Construct a dynamic model of the filter kernel standard deviation: ( Constructing a three-dimensional Gaussian filter kernel: For three consecutive frames of spatiotemporal joint convolution, PSNR is improved by 3.2-4.5dB; Computing power-bandwidth coordinated resolution adaptation: Defining content complexity: Combined with the computing power load of edge devices With bandwidth An improved bilinear interpolation method is used:

[0008]

[0009] ( (For texture weights), dynamically adapting to resolution, reducing bandwidth usage by 25% to 70%.

[0010] Preferably, the color encoding conversion step based on the improved YUV4:2:0 specifically includes:

[0011] In the adaptive RGB-YUV color space conversion, pixel points are extracted. 3×3 neighborhood RGB mean Construct a transformation model:

[0012]

[0013] in, , , Color reproduction accuracy is improved by 8% to 12%;

[0014] In the adaptive weighted YUV4:2:0 chromaticity downsampling, the calculation Component edge strength Divide the edge / flat area, for Downsampling:

[0015]

[0016] in, Data compression is 50%, and edge color error is reduced by 30% to 45%.

[0017] Preferably, the video encoding and compression step based on optimized AV1 specifically includes content-aware variable bitrate control and AV1 quantization parameter optimization with improved CfL, wherein:

[0018] In the content-aware variable bitrate control:

[0019] Computational space complexity: ,

[0020] Time complexity: ,

[0021] Building a bitrate model: , ( The bitrate is dynamically adjusted, reducing fluctuations by 15% to 22%.

[0022] In the optimization of the AV1 quantization parameters of the improved CfL:

[0023] Constructing a luminance-chromaticity prediction model: , Solving using the least squares method, and adjusting the quantization parameters based on the error: , ( , This reduces colorimetric measurement error by 25% to 35%.

[0024] Preferably, the lightweight edge target detection step based on convolutional neural networks specifically includes:

[0025] In the collaborative input of the bitstream features and the detection model, lightweight decoding of AV1 keyframes is performed to extract encoded features and construct a matrix. Component normalization: , Input fusion: The amount of input data is reduced by 40% to 55%;

[0026] In the lightweight CNN optimization and quantization, depthwise separable convolution is used: Then, an attention mechanism was introduced: INT8 Quantization KL Divergence Calibration: The model size is 15-20MB, and the inference speed is 30-40fps.

[0027] In the linked keyframe filtering: the proportion of the target region is calculated. By dynamically adjusting the sampling interval, the amount of data transmitted is reduced by 60% to 75%.

[0028] Preferably, the cloud-based spatiotemporal joint upsampling and quality restoration steps specifically include: cloud-based collaborative preprocessing, improved CNN spatiotemporal upsampling, and luminance-chrominance joint restoration.

[0029] In the cloud-based collaborative preprocessing, keyframes are decoded, and optical flow vectors are calculated. ;

[0030] In the improved CNN spatiotemporal upsampling, a U-Net structure is constructed, and temporal attention is introduced: Loss function: , ), achieving reconstruction from 6fps to 24fps;

[0031] In the aforementioned luminance-chrominance joint restoration, chrominance correction is performed as follows:

[0032]

[0033] Component enhancement:

[0034]

[0035] in, PSNR improved by 1.2-2.0 dB.

[0036] Preferably, the terminal data adaptation and application invocation steps specifically include: multi-resolution stream generation, intelligent data slicing, and lightweight AI interface.

[0037] In the multi-resolution stream generation, a Laplacian pyramid is constructed:

[0038]

[0039] Reconstructed resolution: Dynamically adapts to bandwidth, reducing bandwidth usage by 30% to 70%;

[0040] In the intelligent data slicing, the slice target density is calculated: By building an index tree, data transfer volume is reduced by 65% ​​to 80%.

[0041] In the lightweight AI interface, the PCA compression feature is: ( Terminal classification probability: Transmission latency is 100-200ms, and processing speed is 25-30fps.

[0042] This invention provides a lightweight processing method for IoT video surveillance data based on artificial intelligence and edge computing, which has the following beneficial effects:

[0043] 1. By preprocessing at the edge, optimizing AV1 encoding and key frame filtering, bandwidth usage and data transmission volume are significantly reduced. The lightweight detection model is adapted to edge computing power, and invalid processing is reduced in the cloud, significantly reducing system bandwidth, computing power and storage consumption.

[0044] 2. Multi-stage technology improves the peak signal-to-noise ratio and color reproduction accuracy of video, reduces quantization error, and improves target detection accuracy, avoiding missed detections, achieving a balance between lightweight processing and high quality and high precision.

[0045] 3. Full-process adaptation to multiple monitoring scenarios, terminal generation of multi-resolution video streams adapts to different hardware and bandwidth, lightweight AI interface supports local intelligent analysis, meeting diverse application needs. Attached Figure Description

[0046] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the structures shown in these drawings without creative effort.

[0047] Figure 1 This diagram illustrates a lightweight processing method for IoT video surveillance data based on artificial intelligence and edge computing.

[0048] Figure 2This is a flowchart of S100, the edge-end video data acquisition and preprocessing process in this invention;

[0049] Figure 3 This is a flowchart of S200, the color encoding conversion based on the improved YUV4:2:0, in this invention;

[0050] Figure 4 This is a flowchart of S300, video encoding compression based on optimized AV1, in this invention;

[0051] Figure 5 This is a flowchart of S400, a lightweight edge target detection based on a convolutional neural network, in this invention.

[0052] Figure 6 This is a flowchart of the S500, cloud-based spatiotemporal joint upsampling and quality restoration process in this invention;

[0053] Figure 7 This is a flowchart of S600, terminal data adaptation, and application invocation in this invention;

[0054] The objectives, features, and advantages of this invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0055] The technical solutions of the present invention will now be clearly and completely described with reference to the accompanying drawings of the embodiments of the present invention. It should be understood that the specific embodiments described herein are for illustration and explanation only and are not intended to limit the present invention.

[0056] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the examples in the specification.

[0057] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0058] Secondly, the term "an embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in different places throughout this specification does not necessarily refer to the same embodiment, nor is it a single embodiment or an embodiment selectively excluded from other embodiments.

[0059] This invention proposes a lightweight processing method for IoT video surveillance data based on artificial intelligence and edge computing, which includes the following steps:

[0060] S100, edge-end video data acquisition and preprocessing;

[0061] S200, based on improved YUV4:2:0 color encoding conversion;

[0062] S300, video encoding and compression based on optimized AV1;

[0063] S400, lightweight edge target detection based on convolutional neural networks;

[0064] S500, cloud-based spatiotemporal joint upsampling and quality restoration;

[0065] S600, terminal data adaptation and application invocation;

[0066] In a preferred embodiment of the present invention, in S100, edge video data acquisition and preprocessing, S110, multi-scene noise adaptive suppression is performed first, followed by S120, computing power-bandwidth collaborative resolution adaptation.

[0067] In S110, multi-scene adaptive noise suppression, after acquiring raw RGB format video data through an IoT monitoring camera, a multi-parameter adaptive filtering algorithm is designed by integrating dynamic filtering and statistical feature analysis techniques. First, the parameters of each pixel are calculated. 5x5 neighborhood gray average With grayscale variance: Construct the standard deviation of the filter kernel Dynamic adjustment model: ,in To adapt the weighting coefficients, this model automatically increases the standard deviation of the filter kernel in high-noise regions to enhance noise reduction, and decreases the standard deviation in low-noise, high-detail regions to preserve texture information. Subsequently, a three-dimensional Gaussian filter kernel is constructed, considering inter-frame correlation over time; the filtering formula is then extended as follows: in For time dimension parameters, Using the current frame timestamp, this filter kernel performs spatiotemporal joint convolution operations on three consecutive frames of video data. Compared to traditional single-frame filtering, this can improve the peak signal-to-noise ratio (PSNR) of video frames by 3.2-4.5dB and structural similarity by 0.02-0.05, effectively balancing the contradiction between noise suppression and detail preservation, and adapting to the dynamic noise characteristics in industrial scenarios.

[0068] In the S120 section on computing power-bandwidth collaborative resolution adaptation, a two-stage dynamic resolution adjustment strategy based on real-time resource status is designed, taking into account the fluctuations in computing power of edge computing devices and changes in IoT communication bandwidth. First, the complexity of video frame content is defined. The calculation is performed by combining intra-frame edge density and texture richness. ,in For video frame size, For pixels Edge detection results For pixels Texture intensity; simultaneously collect real-time computing load data from edge devices. Based on communication bandwidth, construct a resolution adaptation decision model: when and or At that time, improved bilinear interpolation was used to downsample the original resolution to 1280x720, and the interpolation formula was optimized as follows:

[0069] in These are weighting coefficients based on neighborhood texture; when and and When, it adapts to a 1920x1080 resolution; when and and At that time, the original resolution is preserved.

[0070] In a preferred embodiment of the present invention, in S200, the color encoding conversion based on the improved YUV4:2:0, S210, the adaptive conversion of RGB-YUV color space is performed first, and then S220, the adaptive weighted YUV4:2:0 chromaticity downsampling is performed.

[0071] In the S210 RGB-YUV color space adaptive conversion, following the preprocessed RGB format video data from S100, a color space conversion algorithm incorporating pixel neighborhood features is designed to address potential color deviations and brightness fluctuations that may still exist after preprocessing. First, each pixel is extracted... 3x3 neighborhood RGB component mean A dynamic conversion coefficient model is constructed to balance local color consistency, and the conversion formula is optimized as follows:

[0072]

[0073] in, The average brightness value of the neighborhood, ranging from 0 to 255, is used to dynamically adjust the brightness component. To improve contrast and avoid the image being too dark or too bright after preprocessing; , , Neighborhood Component variance is used to suppress noise propagation in the chroma component. This conversion model improves color reproduction accuracy by 8%–12% compared to traditional fixed-coefficient conversion, and increases the PSNR of the converted video frames by 2.5–3.8 dB, ensuring that key visual information is not lost during color space conversion of the preprocessed data.

[0074] In the S220 adaptive weighted YUV 4:2:0 chroma downsampling, considering the differences in sensitivity to chroma information in different regions, a content-based weighted downsampling strategy is designed for the YUV data converted by S210 to avoid the detail blurring caused by traditional uniform downsampling. First, the Sobel operator is used to calculate... edge strength of the component Divide the pixel region into edge regions ( , (Edge threshold, value 50-80) and flat regions; then... The components are downsampled using 2x2 interlaced subsampling, and the downsampling formula is optimized as follows:

[0075]

[0076] in, As weighting coefficients, pixels in edge regions are due to Larger pixels have higher weights, which automatically increase the weight to ensure accurate preservation of chroma information at the edges; pixels in flat areas have a weight close to 1 to ensure downsampling efficiency. This downsampling strategy achieves a 50% reduction in data volume, reduces chroma error in edge areas by 30%–45%, and reduces block artifacts caused by chroma distortion by 20%–30% in subsequent video encoding. It effectively connects with the preprocessing results of the S100 and provides high-quality YUV data for subsequent video encoding and compression.

[0077] In a preferred embodiment of the present invention, step S300, video encoding compression based on optimized AV1, includes the following steps:

[0078] S310, Content-Aware Variable Bit Rate Control;

[0079] S320, improved AV1 quantization parameter optimization for CfL;

[0080] In the S310 content-aware variable bitrate control, the YUV4:2:0 format video data output from S200 is received. To address the differences in video content complexity under different monitoring scenarios, a dynamic AV1 encoding bitrate adjustment strategy incorporating spatiotemporal features is designed. First, the spatiotemporal complexity features of the video frames are extracted, including spatial complexity... pass Intra-frame gradient entropy calculation of components: ,in for Probability distribution of component gradient values; time complexity Calculated using the average optical flow field values ​​of adjacent frames: ,in Each pixel The horizontal and vertical optical flow vectors are solved using the LK optical flow algorithm. A rate control model is constructed based on time and space complexity. ,in These are the weighting coefficients. The bitrate of the previous frame. This is the index for the current frame. The model dynamically adjusts the bitrate based on the video content, automatically increasing the bitrate to 1210500Kbps in fast-moving scenes to ensure clear motion details; and reducing the bitrate to 810000Kbps in static background scenes to reduce bandwidth usage. Compared to fixed bitrate control, bitrate fluctuation is reduced by 15%–22%, while the PSNR remains stable above 42d8, effectively connecting with the high-quality YUV data output by the S200 and avoiding image quality loss caused by improper bitrate adaptation.

[0081] In the optimization of AV1 quantization parameters in S320 and the improved CfL, the luminance-chrominance correlation of the YUV data after S200 conversion is addressed by optimizing the luminance-chrominance prediction (CHL) algorithm for AV1 encoding to reduce chrominance distortion during quantization. First, a luminance-chrominance joint prediction model is constructed, based on... The local statistical characteristics of the components are used to correct the chromaticity prediction parameters, and the prediction formula is optimized as follows: ,in This is the predicted chromaticity value. To reconstruct the brightness value, This represents the average local brightness. Solve using the least squares method: Subsequently, the quantization parameter (QP) is dynamically adjusted based on the prediction error. The quantization step size model is as follows: ,in The basic quantization parameter (value 20-30). For colorimetric prediction error, The coefficients were adjusted. This optimization reduced the quantization error of the chroma component by 25% to 35%, improved the chroma PSNR of the encoded video by 1.8 to 2.5 dB, and improved the encoding speed by 10% to 15% due to the optimization of the CfL algorithm complexity. It effectively utilizes the chroma details preserved by S200 to provide a high compression ratio and high quality AV1 bitstream for subsequent edge data transmission.

[0082] In a preferred embodiment of the present invention, in step S400, lightweight edge target detection based on convolutional neural network, step S410 is first performed, where AV1 bitstream features and detection model are collaboratively constructed; step S420 is performed, where the structure of the lightweight CNN model is optimized and quantized; and step S430 is performed, where the detection results and AV1 encoding are linked and keyframes are selected.

[0083] In the collaborative input construction of the S410 and AV1 bitstream features and the detection model, the optimized AV1 encoded bitstream output by S300 is used. Considering the limited computing power at the edge and the need to avoid repeated decoding and encoding loss, a strategy for directly extracting bitstream features and fusing them with the detection model input is designed. First, keyframes in the AV1 bitstream are decoded using lightweight methods, extracting only the keyframes. The encoding features of the components, such as intra-frame prediction mode and quantization parameters, are used to construct a feature matrix. (For feature dimensions, take 8-12); and at the same time... The component images are scaled to 320x320 resolution and then normalized. ,in These are the global mean and standard deviation of the Y component after S200 transformation, respectively. Then, the feature matrix... With normalized image The channels are concatenated and fused, and then used as input to the convolutional neural network. The fusion formula is as follows: ,in For convolution operations, The kernel is 3x3. This fusion strategy avoids the computational cost of full decoding, reducing the amount of input data by 40% to 55%. At the same time, the introduction of encoded features improves the model's target detection accuracy for blurry, low-bitrate frames by 10% to 15%, effectively connecting with the encoding results of S300.

[0084] In the structural optimization and quantization training of the S420 and lightweight CNN models, to address the computational limitations at the edge, MobileNetV3 is used as the base model. Combining the miniaturization approach used in railway foreign object detection, the network structure and training strategy are optimized. Firstly, depthwise separable convolutions are used instead of standard convolutions in the feature extraction layer, resulting in a reduction in computational cost. The formula is as follows: ,in The kernel size is [size]. These represent the number of input and output channels, respectively. This replacement reduces the number of model parameters by 60%–70% compared to standard convolution computation. Secondly, an attention mechanism is introduced in the classification layer, with attention weights calculated as follows: ,in For channel The feature response values ​​are used to enhance the weights of the target features. During model training, INT8 quantization is used, and the quantization parameters are calibrated using KL divergence. ,in Output distribution for floating-point model, To quantize the model output distribution, the accuracy loss after quantization is ensured to be less than 3%. The optimized model size is compressed to 15.20MB, and the inference speed on edge devices is improved to 30-40fps, meeting the requirements of real-time detection.

[0085] The S430, detection results, and AV1 encoding are linked for keyframe filtering. Combining the bitrate control of S300 with the detection results, the keyframe sampling strategy is dynamically adjusted to reduce invalid data transmission. First, the target detection results output by the model are evaluated for reliability, retaining targets with a reliability score > 0.7, and the target region proportion is calculated. ,in For the target quantity, For the first The area of ​​the bounding rectangle of each target. Then, a keyframe determination model is constructed: when the target proportion exceeds 5% or a high-confidence target exists, the current frame is marked as a keyframe, with a sampling interval of 1–2 seconds; when… At this time, the sampling interval is extended to 20-25 seconds. After key frame screening, only key frames are fully encoded and transmitted, while non-key frames are only transmitted with detection results and encoding parameters. The amount of data transmitted is further reduced by 60%-75%. At the same time, by linking detection results and encoding parameters, the system avoids target omissions caused by low bit rate in S300, ensuring the efficiency and accuracy of edge data processing.

[0086] In a preferred embodiment of the present invention, in S500, cloud-based spatiotemporal joint upsampling and quality restoration, S510 is first performed, which involves cloud-based collaborative reception and preprocessing of edge keyframes and detection information; then S520 is performed, which involves spatiotemporal joint upsampling and frame interpolation based on improved CNN; and finally S530 is performed, which involves quality restoration based on luminance-chrominance joint optimization.

[0087] In the cloud-based collaborative reception and preprocessing of S510, edge keyframes, and detection information, the cloud receives the edge keyframes and target detection results output by S400. First, the cloud efficiently decodes the keyframes, extracting YUV4:2:0 format data. Simultaneously, it parses the target region information in the detection results and constructs a "frame data-detection mask" correlation matrix. To address the potential keyframe sampling sparsity issue due to lightweight processing at the edge, the motion vectors between adjacent keyframes are calculated, and an optical flow estimation algorithm is used to solve for inter-frame pixel displacement and optical flow vectors. Obtained by minimizing the energy function: ,in For adjacent keyframes, The regularization coefficient (ranging from 0.01 to 0.05) is used to constrain the smoothness of optical flow. Through this processing, the cloud can accurately capture the target's motion trajectory, providing motion constraints for subsequent spatiotemporal upsampling. At the same time, the associated detection mask can mark the target area, ensuring that the repair process prioritizes the preservation of target details and effectively connects with the lightweight detection results of the 5400.

[0088] In the S520 spatiotemporal joint upsampling interpolation based on improved CNN, a convolutional neural network interpolation model fusing spatiotemporal features is designed for the sparse keyframes preprocessed by S510 to solve the edge blurring problem that easily occurs in traditional optical flow interpolation. First, an improved U-Net structure is constructed. At the encoding end, dilated convolution is used to extract multi-scale spatiotemporal features, with dilation rates set to 1, 2, and 4 to expand the receptive field. At the decoding end, a temporal attention module is introduced to assign weights to feature maps at different time steps. The attention weights are calculated as follows: ,in For adjacent frame feature maps, cosine

[0089] Similarity function. Interpolated image. Through feature fusion and convolution generation, an improved Smooth-L1 loss function is adopted, while perceptual loss is introduced to improve visual quality: ,in , This is the feature extraction function for the ReLU4_4 layer of the pre-trained VGG-19 network. The weighting coefficients are 0.3-0.5. This model can reconstruct sparse keyframes from 6fps to dense keyframes from 24fps, improving the PSNR of the interpolated image to 41.5-42.8dB, while maintaining the SSIM of the target region above 0.98. This effectively compensates for the time redundancy problem caused by sparse sampling at the edges.

[0090] In the S530 quality restoration based on luma-chroma joint optimization, to address the potential chroma distortion (such as residual YUV conversion errors at the edges) and luma unevenness issues after S520 frame interpolation, a luma-chroma joint restoration model is designed, incorporating the CfL concept of AV1 encoding. First, the restoration frame is calculated... Local brightness mean of the component With variance Construct a color correction model:

[0091]

[0092] in The interpolated chroma components (U / N) are the chroma components. These are adaptive coefficients (ranging from 0.1 to 0.2 and 0.05 to 0.1 respectively), used to adjust chromaticity based on luminance distribution. Simultaneously, for... The component performs adaptive edge enhancement, extracting edge information using the Laplacian operator. The enhancement formula is as follows: ,in The edge enhancement factor (value 0.2-0.4) is used to improve the sharpness of target edges. The PSNR of the repaired video frame is improved by 1.2-2.0dB compared to the interpolated result, and the chroma distortion rate is reduced by 25%-35%. The final output is a high-quality video stream that meets the needs of IoT monitoring, providing reliable data support for terminal applications (such as real-time monitoring and historical playback).

[0093] In a preferred embodiment of the present invention, in S600, terminal data adaptation and application invocation, firstly in S610, dynamic generation and adaptation of multi-resolution video streams, then in S620, intelligent data slicing and indexing based on target detection results, and finally in S630, terminal adaptation and invocation of lightweight A1 analysis interface.

[0094] In the dynamic generation and adaptation of the S610 multi-resolution video stream, taking over the high-quality repaired video stream output by S500, a multi-resolution generation strategy based on the Laplacian pyramid is designed to address the hardware resolution differences and bandwidth fluctuations of IoT monitoring terminals (such as local displays in smart power distribution cabinets, large screens in remote monitoring centers, and mobile apps). Firstly, the highest resolution video frame repaired by S500 (e.g., 1920x1080) is used as the bottom layer of the pyramid. The upper pyramid image is generated by Gaussian downsampling. The downsampling formula is: ,in A 5x5 Gaussian kernel (standard deviation 1.2) is used. The pyramid levels are defined (values ​​1-3, corresponding to resolutions of 720P, 480P, and 360P); subsequently, the residual images of each pyramid level are calculated. ,in This is a bilinear upsampling operation. The terminal determines the upsampling frequency based on its hardware parameters (such as screen resolution) and real-time bandwidth. Reconstruct the target resolution frame from the residual image: ,when Automatically switch to 360P stream. 480P / 720P streams are called at the same time. The strategy dynamically reduces terminal data transmission bandwidth usage by 30% to 70%, while residual reconstruction ensures that the low-resolution stream retains key details after S500 repair (such as device status indicators and target outlines), adapting to different terminal hardware and network conditions.

[0095] S620, Intelligent Data Slicing and Indexing Based on Target Detection Results. Combining the target detection information (target coordinates, category confidence) output from the S400 edge terminal with the video frames repaired by the S500, a target-related data slicing and indexing mechanism is designed to meet the terminal's application requirement of "retrieving target segments on demand" (e.g., only viewing video segments containing abnormal targets during fault recovery). First, a target-frame association matrix is ​​constructed ( For the number of target categories, (the number of video frames), when the... Frame exists in the first Class target confidence hour, Otherwise, it is 0; then the video stream is sliced ​​according to time windows (e.g., 30s / window), and the target density of each slice is calculated: ,in For the number of slice frames, For the first The bounding rectangular region of the target in frame 1. An index tree is built based on standard density. When the terminal queries, it quickly locates the slice containing the target through the index tree, downloading only the corresponding slice data, reducing data transmission volume by 65% ​​to 80% compared to the complete video stream. At the same time, the target area enhancement markers after S500 repair are embedded in the slices, allowing the terminal to directly locate the target position during loading, improving troubleshooting efficiency.

[0096] Terminal adaptation and invocation of S630 and lightweight A1 analysis interface. To support localized intelligent analysis by terminals (such as equipment status identification of intelligent power distribution cabinets and abnormal behavior judgment of monitoring terminals), a lightweight AL model interface and adaptation scheme are designed based on the video data repaired by the S500. First, feature extraction is performed on the video frames output by the S500, and the MobileNetV2 network is used to extract the feature vector of the target area. Reduce data volume through feature compression: ,in To reduce the dimensionality, the feature vector volume is compressed to 2.5% of its original size. The compressed features and S400 detection results (target coordinates, category) are then encapsulated into JSON format interface data. The interface call protocol uses the MOTT lightweight communication protocol, with transmission latency controlled within 12000ms. The terminal loads a lightweight analysis model (such as an SVM-based device status classification model), inputs the interface data, and calculates the classification probability. ,in For the first The model parameters for the category are used to perform a secondary determination of the target category localized on the terminal, such as distinguishing between "normal devices" and "abnormal devices". This interface is adapted to the computing power of edge terminals, with a model inference speed of 25-30fps. At the same time, it reuses the repair results of S500 and the detection information of S400 to ensure the accuracy and real-time performance of terminal intelligent analysis.

[0097] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A lightweight processing method for IoT video surveillance data based on artificial intelligence and edge computing, characterized in that, Includes the following steps: Raw RGB video data is acquired at the edge and preprocessed sequentially by multi-scene adaptive noise suppression and computing power-bandwidth collaborative resolution adaptation. The preprocessed RGB data is converted to an improved YUV4:2:0 format to complete adaptive color space conversion and adaptive weighted chromaticity downsampling. Optimized AV1 encoding compression was performed on YUV4:2:0 data, including content-aware variable bitrate control and improved CfL quantization parameter optimization; lightweight edge target detection was achieved based on convolutional neural networks, and the collaborative input construction of bitstream features and detection model was completed, the model structure was optimized and quantized training was performed, and the key frame selection of detection results and encoding linkage was performed. The cloud receives keyframes and detection results, performs collaborative preprocessing, improves CNN spatiotemporal joint upsampling and frame interpolation, and performs luminance-chrominance joint quality restoration; the terminal adapts and restores the video stream, generates multi-resolution video streams, builds intelligent data slicing and indexing, designs a lightweight AI analysis interface and supports its use.

2. The method according to claim 1, characterized in that, The edge-end video data acquisition and preprocessing steps specifically include: multi-scene adaptive noise suppression, and pixel point calculation. 5×5 neighborhood gray mean With variance Construct a dynamic model of the filter kernel standard deviation: ( Constructing a three-dimensional Gaussian filter kernel: For three consecutive frames of spatiotemporal joint convolution, PSNR is improved by 3.2-4.5dB; Computing power-bandwidth coordinated resolution adaptation: Defining content complexity: Combined with the computing power load of edge devices With bandwidth An improved bilinear interpolation method is used: ( (For texture weights), dynamically adapting to resolution, reducing bandwidth usage by 25% to 70%.

3. The method according to claim 1, characterized in that, The color encoding conversion steps based on the improved YUV4:2:0 specifically include: In the adaptive RGB-YUV color space conversion, pixel points are extracted. 3×3 neighborhood RGB mean Construct a transformation model: in, , , Color reproduction accuracy is improved by 8% to 12%; In the adaptive weighted YUV4:2:0 chromaticity downsampling, the calculation Component edge strength Divide the edge / flat area, for Downsampling: in, Data compression is 50%, and edge color error is reduced by 30% to 45%.

4. The method according to claim 1, characterized in that, The video encoding and compression steps based on optimized AV1 specifically include content-aware variable bitrate control and AV1 quantization parameter optimization with improved CfL, wherein: In the content-aware variable bitrate control: Computational space complexity: , Time complexity: , Building a bitrate model: , Dynamically adjust the bitrate, reducing fluctuations by 15% to 22%; In the optimization of the AV1 quantization parameters of the improved CfL: Constructing a luminance-chromaticity prediction model: , Solving using the least squares method, and adjusting the quantization parameters based on the error: , , The colorimetric measurement error is reduced by 25% to 35%.

5. The method according to claim 1, characterized in that, The lightweight edge target detection step based on convolutional neural networks specifically includes: In the collaborative input of the bitstream features and the detection model, lightweight decoding of AV1 keyframes is performed to extract encoded features and construct a matrix. Component normalization: , Input fusion: The amount of input data is reduced by 40% to 55%; In the lightweight CNN optimization and quantization, depthwise separable convolution is used: Then, an attention mechanism was introduced: INT8 Quantization KL Divergence Calibration: The model size is 15-20MB, and the inference speed is 30-40fps. In the linked keyframe filtering: the proportion of the target region is calculated. By dynamically adjusting the sampling interval, the amount of data transmitted is reduced by 60% to 75%.

6. The method according to claim 1, characterized in that, The cloud-based spatiotemporal joint upsampling and quality restoration steps specifically include: cloud-based collaborative preprocessing, improved CNN spatiotemporal upsampling, and luminance-chrominance joint restoration. In the cloud-based collaborative preprocessing, keyframes are decoded, and optical flow vectors are calculated. ； In the improved CNN spatiotemporal upsampling, a U-Net structure is constructed, and temporal attention is introduced: Loss function: , ), achieving reconstruction from 6fps to 24fps; In the aforementioned luminance-chrominance joint restoration, chrominance correction is performed as follows: Component enhancement: in, PSNR improved by 1.2-2.0 dB.

7. The method according to claim 1, characterized in that, The terminal data adaptation and application invocation steps specifically include: multi-resolution stream generation, intelligent data slicing, and lightweight AI interface. In the multi-resolution stream generation, a Laplacian pyramid is constructed: Reconstructed resolution: Dynamically adapts to bandwidth, reducing bandwidth usage by 30% to 70%; In the intelligent data slicing, the slice target density is calculated: By building an index tree, data transfer volume is reduced by 65% ​​to 80%. In the lightweight AI interface, the PCA compression feature is as follows: ( Terminal classification probability: Transmission latency is 100-200ms, and processing speed is 25-30fps.

Images