Image data transmission system and method based on multi-scale compression and time series analysis

Abstract

The present invention relates to the field of communication network technology, and proposes an image data transmission system and method based on multi-scale compression and timing analysis. The method comprises: S1: a transmitting end pre-processes the transmitted video data to obtain pre-processed video frames; S2: a CNN and a Transformer-based hierarchical attention model are used to extract features from the video frames; S3: information between video frames is modeled through local motion compensation and global motion compensation to obtain multi-scale information, and the multi-scale information is processed to obtain compressed video data; S4: a long short-term memory network model is established to select a timing to send the compressed video data to a video receiver, and the receiver uses the same neural network parameters as when the transmitting end compressed the video data to restore the video data. The method of the present invention utilizes a solution that transmits a smaller amount of video data and dynamically adapts to network changes during transmission to achieve efficient transmission of image data.

Description

Technical Field

[0001] The present invention relates to the field of communication network technology, and in particular to an image data transmission system and method based on multi-scale compression and time series analysis. Background Art

[0002] With the rapid development of Internet technology, video image data transmission has become an important part of network communication. Efficient video transmission systems are crucial for applications such as real-time monitoring, distance education, and video conferencing. Image and video transmission includes the following steps: (1) Source coding: compressing the original image data. The purpose of compression is to reduce the amount of data required for transmission. The core role is the efficiency of the encoding algorithm; (2) Data transmission: sending data through a network (such as a local area network, the Internet, or a wireless network). Different transmission strategies determine the integrity and efficiency of the transmitted data; (3) Decoding and recovery: restoring the transmitted compressed image data, correcting errors, and converting it into the original image data. The most critical factor is the accuracy of the restored original image data.

[0003] Due to the increasing resolution of image data and limited transmission bandwidth, existing video data transmission methods have many defects. For example, the current image and video transmission system has the following problems: (1) Existing video data compression and encoding algorithms can lead to over-compression problems, which in turn leads to a significant decrease in video quality, especially in low-bandwidth environments, which often results in a large loss of details. (2) Existing compression algorithms often cannot balance the compression rate and the computational complexity of decompression. Too low a compression rate will result in a large amount of transmission cost, and too high a compression rate will result in excessive decompression complexity, which in turn limits the application performance on resource-constrained devices. (3) Current video data transmission often cannot fully adapt to the dynamic changes in network conditions, which may cause video transmission interruption or quality problems when the network conditions are unstable. (4) The existing video decoding process is inefficient, and errors in the process may be amplified by defects in the compression algorithm, resulting in a decrease in image quality.

[0004] For example, the Chinese invention patent with announcement number CN117808907A discloses an image compression sensing reconstruction method and system based on a hybrid Transformer and CNN. The hybrid Transformer and CNN image compression sensing reconstruction model is constructed according to the sampling rate, including a sampling subnet, an initial reconstruction subnet and a depth reconstruction subnet; the sampling subnet is used to extract features of the image to obtain measurement values; the initial reconstruction subnet is used for initial reconstruction from the measurement values ​​to the original signal; the depth reconstruction subnet is used to perform depth reconstruction based on the image after initial reconstruction; the constructed image compression sensing reconstruction model is trained according to the set network loss function and training data; and image compression sensing reconstruction is performed based on the trained image compression sensing reconstruction model.

[0005] For example, the Chinese invention patent with announcement number CN116029953A discloses a reference-free image quality assessment method based on self-supervised learning and Transformer. It uses a supervised contrastive learning method to perform self-supervised training on the large-scale unlabeled image quality assessment dataset KADIS, thereby learning the degradation characteristics of distorted images with different distortion types and levels; combining the characteristics of CNN and Transformer to design a multi-stream backbone network structure to extract the distortion features corresponding to the image; and designing a block attention module to fuse the degradation features and distortion features to obtain the final distorted image quality features for regression prediction of the image objective score.

[0006] However, the above method has certain limitations in balancing compression efficiency and video quality in complex scenarios. The present invention uses deep learning technology to achieve higher compression rates and efficiency, while better preserving the original features of the video and reducing compression distortion. Summary of the Invention

[0007] The first objective of the present invention is to provide an efficient video data compression algorithm that balances compression rate and compression efficiency. Traditional video compression techniques often use hand-crafted algorithms to predict and encode differences between video frames. However, the present invention utilizes deep learning technology to not only retain the original features to a great extent during compression while achieving a high compression rate and saving bandwidth during transmission, but also to restore the original data with minimal distortion during compression recovery. To achieve this objective, the following technical solutions are proposed:

[0008] An image data transmission method based on multi-scale compression and time series analysis includes the following steps:

[0009] S1: The sending end preprocesses the transmitted video data to obtain preprocessed video frames;

[0010] S2: extracting features from the video frame using a CNN and a Transformer-based hierarchical attention model to obtain an output feature map, and reducing the loss of semantic information of the video frame by performing multi-channel encoding on the output feature map;

[0011] S3: Modeling information between video frames through local motion compensation and global motion compensation to obtain multi-scale information, and processing the multi-scale information to obtain compressed video data;

[0012] S4: By establishing a long short-term memory network model, a timing is selected to send the compressed video data to a video receiver, and the receiver uses the same neural network parameters as those used for compression at the sending end to restore the video data.

[0013] Preferably, the pretreatment specifically comprises the following steps:

[0014] The wavelet denoising method is used to denoise the noisy image data; the pixel points of the low-resolution image data are increased by up-sampling method, and the high-resolution image data resolution is guaranteed by Lanczos resampling technology.

[0015] Preferably, the CNN processing specifically includes the following steps:

[0016] Perform feature extraction on the video frame using a convolutional layer of size 3×H×W to obtain an output value of an output feature map of the video frame at a current position;

[0017] The output value is sent to the MaxPooling layer of size N×H×W for pooling to obtain a pooled representation;

[0018] The pooled representation is input into a residual layer consisting of two activation functions ReLu and two convolutional layers of size N×k×1, and the gradient disappearance is reduced by linking the image frame to the output value to extract key visual features.

[0019] Preferably, the establishment of the hierarchical attention model specifically includes the following steps:

[0020] Divide the output feature map into multiple non-overlapping independent windows;

[0021] Independently calculating the self-attention score and the correlation calculation of the image pixels for the independent window;

[0022] The semantic information of the video frame is obtained by interactively expressing the correlation between the independently calculated self-attention scores of the independent windows and the image pixels.

[0023] Preferably, the multi-channel encoding process specifically includes the following steps:

[0024] Connect the output feature map directly to the output of the multi-channel encoder;

[0025] Pass the output feature map through N ResBlock modules and connect it to the output of the multi-channel encoder;

[0026] The output feature map is passed through Swin Transformer and an activation function Sigmoid, and connected to the output of the multi-channel encoder. At the same time, fast convolution and deconvolution strategies are used to reduce computational complexity.

[0027] Preferably, the calculation formula of the fast convolution and deconvolution strategy is:

[0028] V=A T GWGT ⊙B T XBA

[0029] Where ⊙ represents the Hadamard product, V is the m×m output block, W is the k×k weight, X is the p×p input block, and A, B, and G are transformation matrices.

[0030] Preferably, the local motion compensation process specifically includes the following steps:

[0031] extracting multi-scale information from a previous frame of the video frame;

[0032] Aligning the multi-scale information and estimating the motion vector using the Lucas-Kanade optical flow algorithm;

[0033] The multi-scale information is feature-deformed using the motion vector to form a local context corresponding to the video frame, thereby obtaining a locally compensated video frame.

[0034] Preferably, the global motion compensation process specifically includes the following steps:

[0035] An intermediate feature is calculated based on the multi-scale information, a similarity score is calculated between the multi-scale information and the intermediate feature, and then the similarity score is concatenated with the attention score to generate a global motion compensation code.

[0036] Preferably, the establishment of the long short-term memory network model specifically includes the following steps:

[0037] Capturing network traffic data and labeling the traffic data with time-related tags;

[0038] Pre-processing the flow data to determine the seasonal period of the flow data, and then dividing the flow data into N small segments, each segment including a complete seasonal period, to obtain a flow data set;

[0039] Using a clustering algorithm to identify the traffic data set into K different groups;

[0040] For the K different groups, a prediction model is trained respectively using a long short-term memory network model.

[0041] Based on the same concept, an image data transmission system based on multi-scale compression and time series analysis is also proposed, which includes a video pre-processing module, a feature extraction module, a transform coding module, a motion compensation module, a decoding and post-processing module;

[0042] The video preprocessing module obtains preprocessed video frames by performing denoising, resolution adjustment and color space conversion on the video frames;

[0043] The feature extraction module receives the video frame, analyzes the video frame using the CNN and Transformer model architecture, and extracts key visual features;

[0044] The transform coding module receives the visual features for compression;

[0045] The motion compensation module receives the compressed visual features and predicts the changes between the video frames through motion vector estimation and attention splicing;

[0046] The decoding and post-processing module restores the compressed visual features into image frame data.

[0047] The video preprocessing module primarily optimizes video data in preparation for the subsequent compression process. This preprocessing includes denoising, resolution adjustment, and color space conversion. Furthermore, to minimize the quality issues associated with segmented frames, the present invention sorts frames or selectively discards certain frames to reduce redundancy and improve compression efficiency.

[0048] The feature extraction module uses a convolutional neural network (CNN) and Transformer model architecture to analyze video frames and extract key visual features. These features are then used in the subsequent data compression process. It can identify important objects and areas in the video and preserve them more during compression, thereby ensuring video quality.

[0049] The transform coding module uses the codec model to concentrate the high-energy information of the features extracted in the previous step, which not only makes the data compression more efficient but also reduces the discarded key information.

[0050] The motion compensation module designs a new method for estimating motion between adjacent frames to predict changes between frames. The purpose is to encode only the changed parts, thereby reducing the encoding and transmission of some unchanged redundant parts.

[0051] The decoding and post-processing modules use the same methods for decoding as for encoding and also include sharpening, deblocking, and color correction measures that help improve the visual quality of the decompressed video.

[0052] Compared with the prior art, the present invention has the following beneficial effects:

[0053] The method of the present invention utilizes deep learning technology to not only preserve the original features to a great extent during compression and achieve high compression rates, thus saving bandwidth during transmission, but also to restore the original data with minimal distortion during compression recovery algorithms. This approach improves efficiency through efficient client-server data transmission, while also leveraging machine learning to select transmission times with the most unobstructed network conditions, minimizing data loss and avoiding retransmission overhead. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Figure 1 is a flow chart of Example 1 of the present invention;

[0055] Figure 2 is a diagram of the overall system structure of Example 1 of the present invention;

[0056] Figure 3 This is a flow chart of the video preprocessing module in the efficient compression solution adopted in Example 1 of the present invention;

[0057] Figure 4 Detailed implementation details and model architecture diagram of the compression module of Example 1 of the present invention;

[0058] Figure 5 This is a flowchart of predicting the optimal time for network communication based on user traffic data according to embodiment 1 of the present invention. DETAILED DESCRIPTION

[0059] The present invention is further described in detail below in conjunction with test examples and specific implementation methods. However, this should not be understood as limiting the scope of the present invention to the following embodiments, and all technologies implemented based on the content of the present invention fall within the scope of the present invention.

[0060] Example 1

[0061] A method for image data transmission based on multi-scale compression and time series analysis, the flow chart is as follows Figure 1 As shown, the following steps are included:

[0062] S1: The sending end preprocesses the transmitted video data to obtain preprocessed video frames;

[0063] S2: extracting features from the video frame using a CNN and a Transformer-based hierarchical attention model to obtain an output feature map, and reducing the loss of semantic information of the video frame by performing multi-channel encoding on the output feature map;

[0064] S3: Modeling information between video frames through local motion compensation and global motion compensation to obtain multi-scale information, and processing the multi-scale information to obtain compressed video data;

[0065] S4: By establishing a long short-term memory network model, a timing is selected to send the compressed video data to a video receiver, and the receiver uses the same neural network parameters as those used for compression at the sending end to restore the video data.

[0066] Based on the same concept, an image data transmission system based on multi-scale compression and time series analysis is also proposed, which is characterized by including a video preprocessing module, a feature extraction module, a transform coding module, a motion compensation module, a decoding and post-processing module;

[0067] The video preprocessing module is used for the transmitting end to preprocess the transmitted video data to obtain preprocessed video frames;

[0068] The feature extraction module receives and extracts features from the video frame using a CNN and a Transformer-based hierarchical attention model to obtain an output feature map;

[0069] The transform coding module is used to reduce the loss of semantic information of the video frame to obtain an output feature map;

[0070] The motion compensation module receives the compressed video frames to model information between the video frames to obtain compressed video data;

[0071] The decoding and post-processing module restores the compressed video data to original video data.

[0072] like Figure 2 As shown in the figure, the efficient transmission of video in this system is divided into three stages. The first stage is that the sender compresses the video to be transmitted, which includes preprocessing steps such as framing the input video and adjusting the resolution; the image features of the segmented frames are extracted to further generate a compressed representation, and the information between different frames is compensated to eliminate redundant joint contextual semantic information. In the second stage, after generating the compressed representation, the efficient UDP transmission scheme between the sender and the receiver is unified, and then the neural network is used to monitor the traffic over a period of time and then send it at the appropriate time. In the third stage, the receiver uses the same neural network parameters as when compressing the sender to restore the compressed representation.

[0073] As a specific embodiment, in step S1 , because the quality of the transmitted video varies, pre-processing the video materials with different problems is crucial for the subsequent compression and decompression process.

[0074] First, the noisy video material is subjected to denoising. The present invention adopts the wavelet denoising method, as shown in formula (1), where ψ a,b (t) is the wavelet function, which uses Fourier transform to project the segmented frame into the frequency domain, a is the scale parameter, b is the position parameter, and f(t) is the input image frame. After the transformation, noise removal is performed based on the relationship between the wavelet coefficient and the threshold, as shown in formula (2), where d ij is the wavelet coefficient, and λ is the threshold. Finally, according to formula (3), the inverse wavelet transform is performed to restore the original noise-removed data.

[0075] W f (a,b)=∫f(t)ψ a,b (t)dt (1)

[0076]

[0077] f(t)=∑ a,b W f (a,b)ψ a,b (t) (3)

[0078] Secondly, for videos with too low resolution, upsampling is used to increase the number of pixels, and for images with too high resolution, Lanczos resampling technology is used to maintain image quality when adjusting the resolution. In the present invention, OpenCV and FFmpeg are combined for image preprocessing.

[0079] like Figure 3 As shown, videos with excessive noise are first denoised to prevent the noise from being mistaken for video details during compression and subsequently amplified. The goal of adjusting the resolution is to ensure that the image detection window size learned by the model parameters is within a certain range, thereby restoring higher clarity during reconstruction.

[0080] As a specific embodiment, CNN is used for feature extraction of the initial frame: the preprocessed image frame Xt is first fed into a convolutional layer (Conv) of size 3×H×W for feature extraction, where H and W are the height and width of the frame. The calculation process is shown in formula (4), where * represents the convolution operation, (x, y) represents the position in the output feature map, and (m, n) represents the area covered by the convolution kernel in the input feature map. The convolution kernel h slides over the input feature map f. At each position, the area of ​​the input image covered by the convolution kernel is multiplied element-by-element by the convolution kernel elements. These products are then summed to obtain the value of the output feature map at the current position.

[0081] g(x,y)=(f*h)(x,y)=∑ m ∑ n f(m,n)·h(xm,yn) (4)

[0082] The output of the convolutional layer is then fed into a MaxPooling layer of size N×H×W for pooling. This paper uses the maximum pooling method, which uses the point with the largest pixel in an N×N pooling window as the pooled representation. The purpose is to reduce the feature dimension and thus enhance the generalization and robustness of the model. This is expressed as shown in Formula (5), where f is the output of Conv.

[0083]

[0084] Finally, the pooled representation is input into a residual layer, linking the input to the output. The purpose is to reduce the problem of vanishing gradients and improve training speed and convergence. It is specifically composed of two activation functions ReLu (RectifiedLinearUnit) and two convolutional layers of size N×k×1. The mathematical expression of ReLu is shown in formula (6). When the input x is greater than 0, the function outputs x. When the input x is less than or equal to 0, the function outputs 0. In ResBlock, two ReLu functions are used. The first ReLU is placed after the convolution layer, and the second is used before adding the residual of the input. This helps the network learn effective features while maintaining gradient flow.

[0085] f(x) = max(0, x) (6) Transformer is used for feature extraction of the initial frame: Considering the high cost of computing the entire graph and the unclear semantic information, the present invention uses a Transformer-based hierarchical attention model Swin Transformer and improves it. Its calculation principle and the calculation process adopted by the present invention are shown in the following steps:

[0086] a) Window Partitioning: The core of this Transformer architecture is the self-attention mechanism of a moving window. This mechanism is based on the local correlation of image data, which is reflected in the fact that adjacent pixels in image frames often exhibit similar color, brightness, and texture, and that a physical object in the image is continuous within a certain spatial range. Based on this idea, this method partitions the feature map extracted by the CNN in the previous step, key is to divide it into multiple non-overlapping small windows.

[0087] b) Self-attention calculation: Next, the self-attention score is calculated independently in each window. The calculation process is shown in formula (7), where Q, K, and V are query, key, and value matrices respectively, and d k It is the dimension of the key. In the present invention, Q, K, and V specifically represent the input feature map obtained by linear transformation, which is achieved by multiplying Q, K, and V by the weight matrix W plus the bias b.

[0088]

[0089] c) Multi-scale information integration: In the first step, the entire image is divided into independent windows. In the second step, self-attention and the correlation of image pixels within each window are independently calculated. In this step, the semantic information of the image can only be expressed by the interaction of information within different windows.

[0090] In the feature extraction module of the previous step, CNN and Transformer are used to extract the semantic information of the input image frame. However, even if the ResBlock residual module is used, there is still a risk of model overfitting and low computational efficiency. Therefore, a variation coding module is used in the present invention to solve this problem.

[0091] Its specific implementation includes three steps: the first step is to directly connect the initial feature map (CNN extraction) to the output, in order to reduce the loss of original semantic information due to the model's over-complexity during the training process; the second step is to pass the initial feature map through N ResBlock modules and connect it to the output, in order to extract the unchanged semantic information of the original feature map; the third step is to pass the initial feature map through the Swin Transformer and an activation function Sigmoid and connect it to the output. The purpose of this is mentioned in the description of the second module. The Sigmoid activation function used is shown in formula (8).

[0092]

[0093] Fast convolution strategy: In order to reduce computational complexity and thus improve compression efficiency, a new fast convolution and deconvolution strategy is proposed in this invention, the mathematical expression of which is shown in formula (9). Where X, W, and V represent the input block of size p×p, the weight of k×k, and the output block of m×m, respectively. ⊙ represents the Hadamard product, and A, B, and G are transformation matrices, which can be adjusted as hyperparameters during model training. Compared with calculating each element in the output feature map separately, the fast algorithm uses structural similarity to generate the output block. This method significantly reduces the computational complexity of convolution and deconvolution.

[0094] V=A T GWG T ⊙B T XBA (9)

[0095] Furthermore, to reduce redundancy between video frames and effectively lower data transmission and storage requirements, the present invention proposes a new motion compensation algorithm. By combining local motion context compensation with global context motion compensation, the semantic information between videos can be fully integrated. The specific implementation principle of the invention is as follows:

[0096] Video Local Motion Compensation: This method uses the multi-scale characteristics of time propagation to perform local motion compensation, which is performed first when the second module compresses the data. Local compensation focuses on small-scale motion in the video, such as slight movement of an object or slight camera shake.

[0097] a) First, for the features extracted from the current frame in the second module, multi-scale information will also be extracted from the previous frame of the current frame, so that the model can capture image information from different resolution levels, thereby improving the robustness of the model. The present invention uses Laplacian pyramid and Harris corner detection technology to obtain multi-scale information

[0098] b) Then the detected multi-scale information is aligned, and the present invention uses the Lucas-Kanade optical flow algorithm to estimate the motion vector These vectors describe the movement paths of features between consecutive frames.

[0099] c) Deform the extracted multi-scale features through motion vectors to form the local context corresponding to the current frame

[0100] Finally, after these three steps, the current frame after local compensation can be obtained It is expressed as shown in formula (10), x t The representation of the current frame is extracted. In this way, the network can well understand how to perform conditional encoding and also prepare for decoding recovery.

[0101]

[0102] Global motion compensation: In the second module, we have mentioned the use of Swin Transformer for feature extraction. At the same time, we can use the characteristics of its global attention mechanism to perform global motion compensation. For a given compressed frame with multi-scale information Calculate the intermediate feature y 1t ,y 2t .Will and The similarity score is calculated and then concatenated with the attention score to generate the global motion compensation code. The similarity calculation formula and the expression of global motion compensation are shown in formula (11) and formula (12).

[0103]

[0104] In the second module, the extracted features are obtained, and after variable coding and motion compensation, the compressed image data is obtained. In this module, the compressed image data is restored to the image frame data. In the present invention, this is specifically implemented through the ResBlock and Deconvolution (DeConv) modules. It should be pointed out that the deconvolution here uses the fast algorithm proposed in the third module. Figure 3 In the , the image data to be compressed is divided into frames. Figure 4 The input is Figure 3 Output.

[0105] In an optional implementation, the video to be transmitted is input into the system, and the system first performs preliminary cleaning on the video based on the video resolution, color points, noise, etc. to obtain a video representation that is conducive to compression while preserving the fidelity, and then adjusts the resolution to a uniform size for the model. Figure 3 As shown, the feature extraction of the image frame is started, and the features of the image are represented by dimensionality reduction. The purpose is to compress it so that its transmission is more efficient. The deep learning technology is used for compression, which is more accurate and has higher compression efficiency than the traditional manual extraction of vital signs.

[0106] like Figure 3 The feature compression module combines CNN and Transformer in feature extraction. When using CNN, the present invention utilizes fast convolution and deconvolution strategies. Its calculation formula is mentioned in the technical solution. It is improved in implementation as shown in formula (13). The only difference is the addition of the mask matrix M. Its purpose is to reduce the computational complexity in the actual process and reduce the expression of the dimensions that are insufficient for convolution calculation.

[0107] V=A T [(M⊙GWG T )⊙(B T XB)]A (13) CNN extracts image edge and texture features, but its ability to understand the relationship between video frames and time series is insufficient. Therefore, Transformer is added during feature extraction to extract dynamic features and enhance the features extracted by CNN. Since the traditional Transformer architecture requires the entire image as input, which greatly increases the computational complexity, the moving window mechanism of SwinTransformer is used to calculate the attention score only within each small window, which is very fast. The information between different windows is then integrated to obtain a compressed representation. In the example, the number and size of the attention mechanism can be adjusted to achieve the best compression effect and balance.

[0108] like Figure 3 The transform coding module shown in the example constructs three channels. The first channel directly connects the input and output extracted by the CNN. This is similar to the concept of the ResBlock residual layer and prevents the problems of vanishing and exploding gradients during compression model training. The second channel associates the SwinTransformer with the output through an activation function, capturing dynamic sequence relationships. Finally, the third channel combines the invariant semantic representations learned by multiple ResBlocks to learn more complex feature representations.

[0109] The present invention proposes global motion compensation and local motion compensation to model the information between two or more frames. The application of global and local motion compensation technology in the patent example shows how to optimize learning video compression by combining flow networks and cross-attention mechanisms. Local motion compensation is mainly implemented through flow networks, which is used to accurately estimate and compress motion information between reference frames and current frames. This mainly involves capturing local redundancy between frames at the multi-scale feature level. Global motion compensation uses a cross-attention mechanism and is performed within the feature domain to capture long-range inter-frame similarities, which helps to understand and encode large-scale motion and global correlation. In addition, in order to deal with the high complexity problem of traditional cross-attention, the present invention divides the softmax operation into two independent operations, thereby reducing the complexity from quadratic to linear, making it possible to process high-resolution videos. This combination of local and global motion compensation significantly improves compression efficiency and video quality, proving its effectiveness in learning video compression.

[0110] The second objective of the present invention is to provide a technology based on a custom UDP protocol and machine learning. On the one hand, it utilizes efficient client-server data transmission to improve efficiency, and on the other hand, it uses machine learning to select the most unobstructed network time for transmission to reduce data loss and thus avoid the overhead of retransmission. Ultimately, it provides a robust video data transmission method:

[0111] 1) Using enhanced UDP transport protocol to customize the structure of data packets between the customized server and the client, the server dynamically selects error control and retransmission strategies based on the bandwidth between the server and the client.

[0112] 2) Machine learning-based network traffic prediction technology: Data transmission should be reduced during network congestion, and accelerated during network traffic. The geographic location and network traffic of clients and servers are expected to be regular. This invention uses machine learning technology to detect and capture network traffic and predict future network traffic to achieve efficient data transmission to the recipient. The specific implementation is shown below.

[0113] The detection software monitors the traffic distribution of the sender within a certain area and time, and then performs unsupervised classification on the detected data.

[0114] a) First, the collected traffic is labeled with time-related tags;

[0115] b) Perform data preprocessing to determine the seasonal period of the data, and then divide the collected traffic data into small segments of length N, each segment including a complete seasonal period;

[0116] c) Use a clustering algorithm to identify the data set in the previous step as K different groups. Specifically, the present invention uses the K-means clustering algorithm and uses DTW as the distance metric. The specific calculation formula is shown in formula (14), where x and x ' are two time series, d is the Euclidean distance, and π is the set of all possible time series alignment paths;

[0117] d) Train a prediction model for each population, using 70% of each population as the training set, 20% as the validation set, and 10% as the test set. This paper uses a long short-term memory network model, which includes an input gate, a forget gate, a cell state, and an output gate. The specific settings are as follows: The LSTM input is 24-hour traffic monitoring data, with 48 stacking layers. Feature engineering is used to distinguish weekend and weekday traffic, and only features with a Pearson coefficient greater than 0.9 are used.

[0118]

[0119] Specifically, such as Figure 2 As shown, network traffic data is first clustered using unsupervised clustering techniques to identify and separate data groups with similar temporal characteristics. Subsequently, a specialized long short-term memory (LSTM) network model is trained for each data group. These models use traffic data from the past 24 hours to predict network traffic for the next hour. This technology not only improves prediction accuracy but also dynamically adjusts based on real-time data, ensuring that the model maintains optimal performance in diverse network environments. Furthermore, by continuously collecting new traffic data and retraining the model, this strategy enables the system to adapt to new or unknown traffic patterns, allowing it to continue to operate effectively in a dynamically changing network environment.

[0120] The image data transmission method based on multi-scale compression and time series analysis of the present invention is described in detail below with reference to Example 2.

[0121] Example 2

[0122] The proposed multi-scale compression and timing analysis image data transmission algorithm is compared with several existing commonly used algorithms, mainly measuring their performance in compression efficiency, video quality, computational complexity and network adaptability. The comparison results are shown in Table 1.

[0123] The YouTube-8M dataset is used in the experiments, which contains a variety of high-resolution video clips and is suitable for evaluating the performance of video compression algorithms.

[0124] To compare the performance of different algorithms, the experiment uses the following common quantitative indicators:

[0125] Compression ratio (%): Measures the ratio of the compressed file size to the original file size. The higher the better.

[0126] PSNR (dB): Peak signal-to-noise ratio, used to evaluate the quality of compressed video. The higher the PSNR, the better the quality.

[0127] SSIM: Structural Similarity, quantifies the visual similarity between the compressed image and the original image, with 1 being perfect similarity.

[0128] Latency (ms): The time delay caused by video compression and transmission. The lower the better.

[0129] Computational complexity (GFLOPs): The number of floating-point operations consumed during the calculation process. The lower the complexity, the higher the calculation efficiency.

[0130] Comparison algorithms:

[0131] H.265 (HEVC): Achieves higher compression efficiency by utilizing advanced intra-frame and inter-frame prediction algorithms to reduce redundant data.

[0132] CNN+RNN compression algorithm based on deep learning: CNN is used to extract spatial features of video frames, while RNN is good at processing time series information and is particularly suitable for modeling associations between video frames.

[0133] Swin Transformer: An image and video compression method based on the self-attention mechanism. It can extract spatial and temporal features of video frames in a multi-scale layered manner, thereby effectively compressing videos.

[0134] Table 1 Comparison of compression efficiency of three different algorithms

[0135]

[0136] Through comparative analysis, the multi-scale compression and timing analysis algorithm of the present invention demonstrates significant advantages in multiple key performance indicators.

[0137] First, the algorithm achieves a compression rate of 38%-42%, a significant improvement over traditional H.265 (25%-30%) and CNN+RNN (30%-35%). In terms of video quality, the PSNR reaches 42dB-46dB, and the SSIM value remains above 0.95, significantly better than H.265's 35dB-37dB and SSIM of 0.9-0.94.

[0138] Secondly, with the help of timing analysis technology, the present invention can dynamically adapt to network changes and control the delay to below 50ms, which is significantly lower than high computational complexity algorithms such as CNN+RNN and SwinTransformer.

[0139] In addition, by optimizing the convolution and deconvolution strategies, the computational complexity of this algorithm is controlled at around 2.5GFLOPs, which is lower than the 5.0GFLOPs of the Swin Transformer. This ensures low computational overhead while achieving efficient compression.

[0140] In summary, the algorithm of the present invention has all-round advantages in terms of compression rate, video quality, delay and computational complexity, and is particularly suitable for video transmission tasks with different resolutions in dynamic network environments.

[0141] Finally, it should be noted that the embodiments described in detail above are only the best practices of the invention and cannot be used to limit the scope of rights of the invention. Equivalent replacement of the technical solutions recorded in the aforementioned embodiments does not deviate the essence of the corresponding technical solutions from the scope of the technical solutions of the embodiments of the invention, and they should all be included in the scope of the claims and description of the invention.

Claims

1. An image data transmission method based on multi-scale compression and time series analysis, characterized in that: The following steps are involved: S1: The sending end preprocesses the transmitted video data to obtain preprocessed video frames; S2: extracting features from the video frame using a CNN and a Transformer-based hierarchical attention model to obtain an output feature map, and reducing the loss of semantic information of the video frame by performing multi-channel encoding on the output feature map; S3: Modeling information between video frames through local motion compensation and global motion compensation to obtain multi-scale information, and processing the multi-scale information to obtain compressed video data; S4: By establishing a long short-term memory network model, a timing is selected to send the compressed video data to a video receiver, and the receiver uses the same neural network parameters as those used for compression at the sending end to restore the video data.

2. The image data transmission method based on multi-scale compression and time series analysis according to claim 1, characterized in that: The pretreatment specifically comprises the following steps: The wavelet denoising method is used to denoise the noisy image data; the pixel points of the low-resolution image data are increased by up-sampling method, and the high-resolution image data resolution is guaranteed by Lanczos resampling technology.

3. The image data transmission method based on multi-scale compression and time series analysis according to claim 1, characterized in that: The CNN described in S2 specifically includes the following steps: S31: Use a convolutional layer with a size of 3×H×W to extract features from the video frame to obtain an output value of an output feature map of the video frame at a current position; S32: Send the output value to a MaxPooling layer of size N×H×W for pooling to obtain a pooled representation; S33: The pooled representation is input into a residual layer consisting of two activation functions ReLu and two convolutional layers of size N×k×1, and gradient vanishing is reduced by linking the video frame to the output value to extract key visual features.

4. The image data transmission method based on multi-scale compression and time series analysis according to claim 3, characterized in that: The establishment of the hierarchical attention model specifically includes the following steps: Divide the output feature map into multiple non-overlapping independent windows; Independently calculating the self-attention score and the correlation calculation of the image pixels for the independent window; The semantic information of the video frame is obtained by interactively expressing the correlation between the independently calculated self-attention scores of the independent windows and the image pixels.

5. The image data transmission method based on multi-scale compression and time series analysis according to claim 1, characterized in that: The multi-channel encoding process specifically includes the following steps: Connect the output feature map directly to the output of the multi-channel encoder; Pass the output feature map through N ResBlock modules and connect it to the output of the multi-channel encoder; The output feature map is passed through Swin Transformer and an activation function Sigmoid, and connected to the output of the multi-channel encoder. At the same time, fast convolution and deconvolution strategies are used to reduce computational complexity.

6. The image data transmission method based on multi-scale compression and time series analysis according to claim 5, characterized in that: The calculation formula of the fast convolution and deconvolution strategy is: V=A T GWG T ⊙B T XBA Where ⊙ represents the Hadamard product, V is the m×m output block, W is the k×k weight, X is the p×p input block, and A, B, and G are transformation matrices.

7. The image data transmission method based on multi-scale compression and time series analysis according to claim 1, characterized in that: The local motion compensation process specifically includes the following steps: extracting multi-scale information from a previous frame of the video frame; Aligning the multi-scale information and estimating the motion vector using the Lucas-Kanade optical flow algorithm; The multi-scale information is feature-deformed using the motion vector to form a local context corresponding to the video frame, thereby obtaining a locally compensated video frame.

8. The image data transmission method based on multi-scale compression and time series analysis according to claim 7, characterized in that: The global motion compensation process specifically includes the following steps: An intermediate feature is calculated based on the multi-scale information, a similarity score is calculated between the multi-scale information and the intermediate feature, and then the similarity score is concatenated with the attention score to generate a global motion compensation code.

9. The image data transmission method based on multi-scale compression and time series analysis according to claim 1, characterized in that: The establishment of the long short-term memory network model specifically includes the following steps: Capturing network traffic data and labeling the traffic data with time-related tags; Pre-processing the flow data to determine the seasonal period of the flow data, and then dividing the flow data into N small segments, each segment including a complete seasonal period, to obtain a flow data set; Using a clustering algorithm to identify the traffic data set into K different groups; For the K different groups, a prediction model is trained respectively using a long short-term memory network model.

10. An image data transmission system based on multi-scale compression and time series analysis, characterized in that: The image data transmission system is used to perform the image data transmission method based on multi-scale compression and timing analysis according to any one of claims 1 to 9, comprising a video preprocessing module, a feature extraction module, a transform coding module, a motion compensation module, a decoding and post-processing module; The video preprocessing module is used for the transmitting end to preprocess the transmitted video data to obtain preprocessed video frames; The feature extraction module receives and extracts features from the video frame using a CNN and a Transformer-based hierarchical attention model to obtain an output feature map; The transform coding module is used to reduce the loss of semantic information of the video frame to obtain an output feature map; The motion compensation module receives the compressed video frames to model information between the video frames to obtain compressed video data; The decoding and post-processing module restores the compressed video data to original video data.

Images