Image data transmission system and method based on multi-scale compression and time sequence analysis

Abstract

The invention relates to the technical field of communication networks, and provides an image data transmission system and method based on multi-scale compression and time sequence analysis, and the method comprises the steps: S1, carrying out the preprocessing of transmitted video data through a transmitting end, and obtaining a preprocessed video frame; s2, performing feature extraction on the video frame by using a CNN (Convolutional Neural Network) and a hierarchical attention model based on Transform; s3, modeling information between the video frames through local motion compensation and global motion compensation to obtain multi-scale information, and processing the multi-scale information to obtain compressed video data; and S4, the compressed video data is sent to a video receiving party by establishing a long-short term memory network model and selecting opportunity, and the receiving party recovers the video data by using neural network parameters which are the same as those of a sending end during compression. According to the method disclosed by the invention, the efficient transmission of image data is realized by utilizing a scheme of transmitting a smaller data volume of a video and dynamically adapting to network change during transmission.

Description

Technical Field

[0001] The present invention relates to the field of communication network technology, and in particular to an image data transmission system and method based on multi-scale compression and time series analysis. Background Art

[0002] With the rapid development of Internet technology, video image data transmission has become an important part of network communication. Efficient video transmission systems are essential for applications such as real-time monitoring, distance education, and video conferencing. Image video transmission includes the following steps: (1) Source coding: compressing the original image data. The purpose of compression is to reduce the amount of data required for transmission. The core role is the efficiency of the encoding algorithm; (2) Data transmission: sending data through a network (such as a local area network, the Internet, or a wireless network). Different transmission strategies determine the integrity and efficiency of the transmitted data; (3) Decoding and recovery: restoring the transmitted compressed image data, correcting errors, and converting it into the original image data. The most critical factor is the accuracy of the restored original image data.

[0003] Due to the increasing resolution of image data and limited transmission bandwidth, existing video data transmission methods have many defects. For example, the current image and video transmission system has the following problems: (1) The existing video data compression and encoding algorithm may lead to over-compression, which in turn leads to a significant decrease in video quality, especially in low-bandwidth environments, which often results in a large loss of details. (2) Existing compression algorithms often fail to balance the compression rate and the computational complexity during decompression. Too low a compression rate will result in a large amount of transmission cost, while too high a compression rate will result in excessive decompression complexity, which in turn limits the application performance on resource-constrained devices. (3) Current video data transmission often fails to fully adapt to dynamic changes in network conditions, which may cause video transmission interruptions or quality problems when the network conditions are unstable. (4) The existing video decoding process is inefficient, and errors in the process may be amplified due to defects in the compression algorithm, resulting in a decrease in image quality.

[0004] For example, the Chinese invention patent with announcement number CN117808907A discloses a method and system for image compressed sensing reconstruction based on a hybrid Transformer and CNN, which constructs an image compressed sensing reconstruction model of a hybrid Transformer and CNN according to the sampling rate, including a sampling subnet, an initial reconstruction subnet and a deep reconstruction subnet; the sampling subnet is used to extract features of the image to obtain a measurement value; the initial reconstruction subnet is used for initial reconstruction from the measurement value to the original signal; the deep reconstruction subnet is used for deep reconstruction based on the image after initial reconstruction; the constructed image compressed sensing reconstruction model is trained according to the set network loss function and training data; and image compressed sensing reconstruction is performed based on the trained image compressed sensing reconstruction model.

[0005] For example, the Chinese invention patent with announcement number CN116029953A discloses a reference-free image quality assessment method based on self-supervised learning and Transformer. It uses a supervised contrastive learning method to perform self-supervised training on the large-scale unlabeled image quality assessment dataset KADIS, thereby learning the degradation characteristics of distorted images with different distortion types and levels; combining the characteristics of CNN and Transformer to design a multi-stream backbone network structure to extract the distortion features corresponding to the image; and designing a block attention module to fuse the degradation features and distortion features to obtain the final distorted image quality features for regression prediction of the image objective score.

[0006] However, the above method has certain limitations in balancing compression efficiency and video quality in complex scenes. The present invention uses deep learning technology to achieve higher compression rate and compression efficiency, while better retaining the original features of the video and reducing compression distortion. Summary of the invention

[0007] The first purpose of the present invention is to provide an efficient video data compression algorithm that can balance the relationship between compression rate and compression efficiency. In traditional video compression technology, manually designed algorithms are often used to predict and encode the differences between video frames. The present invention uses deep learning technology to not only retain the original features to a great extent during compression, but also has a high compression rate and saves bandwidth during transmission. In the compression recovery algorithm, the original data can also be restored with a smaller distortion rate. In order to achieve the above purpose, the following technical solutions are proposed:

[0008] An image data transmission method based on multi-scale compression and time series analysis comprises the following steps:

[0009] S1: The transmitting end preprocesses the transmitted video data to obtain a preprocessed video frame;

[0010] S2: extracting features of the video frame using CNN and a Transformer-based hierarchical attention model to obtain an output feature map, and reducing the loss of semantic information of the video frame by performing multi-channel encoding on the output feature map;

[0011] S3: Modeling information between video frames through local motion compensation and global motion compensation to obtain multi-scale information, and processing the multi-scale information to obtain compressed video data;

[0012] S4: By establishing a long short-term memory network model, a timing is selected to send the compressed video data to a video receiver, and the receiver uses the same neural network parameters as those used for compression by the sender to restore the video data.

[0013] Preferably, the pretreatment specifically comprises the following steps:

[0014] The wavelet denoising method is used to denoise the noisy image data; the pixel points of the low-resolution image data are increased by up-sampling, and the resolution of the high-resolution image data is guaranteed by Lanczos resampling technology.

[0015] Preferably, the CNN processing specifically includes the following steps:

[0016] Use a convolutional layer of size 3x H x W to extract features from the video frame to obtain an output value of an output feature map of the video frame at a current position;

[0017] Send the output value to a MaxPooling layer of size N x H x W for pooling to obtain a pooled representation;

[0018] The pooled representation is input into a residual layer consisting of two activation functions ReLu and two convolutional layers of size N xkx 1, which reduces gradient vanishing and extracts key visual features by linking the image frame to the output value.

[0019] Preferably, the establishment of the hierarchical attention model specifically includes the following steps:

[0020] Divide the output feature map into multiple non-overlapping independent windows;

[0021] Independently calculating the self-attention score and the correlation calculation of the image pixels for the independent window;

[0022] The semantic information of the video frame is obtained by interactively expressing the correlation between the independently calculated self-attention scores of the independent windows and the image pixels.

[0023] Preferably, the multi-channel encoding process specifically includes the following steps:

[0024] Connect the output feature map directly to the output of the multi-channel encoder;

[0025] Pass the output feature map through N ResBlock modules and connect it to the output of the multi-channel encoder;

[0026] The output feature map is passed through Swin Transformer and an activation function Sigmoid, and connected to the output of the multi-channel coder. At the same time, fast convolution and deconvolution strategies are used to reduce the computational complexity.

[0027] Preferably, the calculation formula of the fast convolution and deconvolution strategy is:

[0028] V=AT GWG T ⊙B T XBA

[0029] Where ⊙ represents the Hadamard product, V is the m×m output block, W is the k×k weight, X is the p×p input block, and A, B, and G are transformation matrices.

[0030] Preferably, the local motion compensation process specifically includes the following steps:

[0031] Extracting multi-scale information from a previous frame of the video frame;

[0032] Aligning the multi-scale information and estimating the motion vector using the Lucas-Kanade optical flow algorithm;

[0033] The multi-scale information is feature deformed by using the motion vector to form a local context of the corresponding video frame, thereby obtaining a video frame after local compensation.

[0034] Preferably, the global compensation process specifically includes the following steps:

[0035] An intermediate feature is calculated based on the multi-scale information, a similarity score is calculated between the multi-scale information and the intermediate feature, and then the similarity score is concatenated with the attention score to generate a global motion compensation code.

[0036] Preferably, the establishment of the long short-term memory network model specifically includes the following steps:

[0037] Capturing network traffic data and labeling the traffic data with time-related tags;

[0038] The seasonal period of the flow data is determined by preprocessing the flow data, and then the flow data is divided into N small segments, each segment including a complete seasonal period, to obtain a flow data set;

[0039] Using a clustering algorithm to identify the traffic data set into K different groups;

[0040] For the K different groups, a prediction model is trained respectively using a long short-term memory network.

[0041] Based on the same concept, an image data transmission system based on multi-scale compression and timing analysis is also proposed, including a video preprocessing module, a feature extraction module, a transform coding module, a motion compensation module, a decoding and post-processing module;

[0042] The video preprocessing module obtains preprocessed video frames by performing denoising, adjusting resolution and color space conversion on the video frames;

[0043] The feature extraction module receives the video frame, analyzes the video frame using the CNN and Transformer model architecture, and extracts key visual features;

[0044] The transform coding module receives the visual features for compression;

[0045] The motion compensation module receives the compressed visual features and predicts the changes between the video frames through motion vector estimation and attention splicing;

[0046] The decoding and post-processing module restores the compressed visual features into image frame data.

[0047] The main function of the video preprocessing module is to optimize the video data and prepare for the subsequent compression process. The preprocessing process includes denoising, adjusting resolution, and color space conversion. In addition, in order to reduce the quality of the segmented frames that cannot be guaranteed, the frames are sorted or some frames are selectively discarded to reduce redundancy, thereby improving compression efficiency.

[0048] The feature extraction module uses convolutional neural networks (CNN) and Transformer model architecture to analyze video frames and extract key visual features. These features are used in the subsequent data compression process. It can identify important objects and areas in the video and make these areas more retained during compression, thereby ensuring video quality.

[0049] The transform coding module uses the codec model to display the high-energy information of the features extracted in the previous step, which not only makes the data compression more efficient but also reduces the discarded key information.

[0050] The motion compensation module designs a new method for estimating motion between adjacent frames to predict changes between frames, with the aim of encoding only the changed parts, thereby reducing the encoding and transmission of some unchanged redundant parts.

[0051] The decoding and post-processing module uses the same method for decoding as encoding, and also includes sharpening, deblocking, and color correction measures, which help improve the visual quality of the decompressed video.

[0052] Compared with the prior art, the present invention has the following beneficial effects:

[0053] The method of the present invention utilizes deep learning technology to not only retain the original features to a great extent during compression and have a high compression rate to save bandwidth during transmission, but also can restore the original data with a small distortion rate during compression recovery algorithm. On the one hand, it utilizes efficient client and server data transmission to improve efficiency, and on the other hand, it utilizes machine learning to select the most unobstructed network time for transmission to reduce data loss and avoid retransmission overhead. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Figure 1 is a flow chart of Embodiment 1 of the present invention;

[0055] Figure 2 is an overall system structure diagram of Embodiment 1 of the present invention;

[0056] Figure 3 is a flow chart of a video preprocessing module in the efficient compression solution adopted in Embodiment 1 of the present invention;

[0057] Figure 4 It is the implementation details and model architecture diagram of the compression module of Example 1 of the present invention;

[0058] Figure 5 This is a flow chart of predicting the optimal time for network communication based on user traffic data according to Embodiment 1 of the present invention. DETAILED DESCRIPTION

[0059] The present invention is further described in detail below in conjunction with test examples and specific implementation methods. However, this should not be understood as the scope of the above subject matter of the present invention being limited to the following embodiments, and all technologies realized based on the content of the present invention belong to the scope of the present invention.

[0060] Example 1

[0061] A method for image data transmission based on multi-scale compression and timing analysis, the flow chart is as follows Figure 1 As shown, the following steps are included:

[0062] S1: The transmitting end preprocesses the transmitted video data to obtain a preprocessed video frame;

[0063] S2: extracting features of the video frame using CNN and a Transformer-based hierarchical attention model to obtain an output feature map, and reducing the loss of semantic information of the video frame by performing multi-channel encoding on the output feature map;

[0064] S3: Modeling information between video frames through local motion compensation and global motion compensation to obtain multi-scale information, and processing the multi-scale information to obtain compressed video data;

[0065] S4: By establishing a long short-term memory network model, a timing is selected to send the compressed video data to a video receiver, and the receiver uses the same neural network parameters as those used for compression by the sender to restore the video data.

[0066] Based on the same concept, an image data transmission system based on multi-scale compression and timing analysis is also proposed, which is characterized by including a video preprocessing module, a feature extraction module, a transform coding module, a motion compensation module, a decoding and post-processing module;

[0067] The video preprocessing module is used for the transmitting end to preprocess the transmitted video data to obtain the preprocessed video frame;

[0068] The feature extraction module receives and extracts features from the video frame using a CNN and a Transformer-based hierarchical attention model to obtain an output feature map;

[0069] The transform coding module is used to reduce the loss of semantic information of the video frame to obtain an output feature map;

[0070] The motion compensation module receives the compressed video frames to model the information between the video frames to obtain compressed video data;

[0071] The decoding and post-processing module restores the compressed video data to the original video data.

[0072] like Figure 2 As shown, in this system, efficient video transmission is divided into three stages. The first stage is that the sender compresses the video to be transmitted, which includes preprocessing steps such as framing the input video and adjusting the resolution; image feature extraction is performed on the segmented frames to further generate a compressed representation, and compensation representation is performed on the information between different frames to eliminate redundant joint contextual semantic information. In the second stage, after generating the compressed representation, the efficient UDP transmission scheme between the sender and the receiver is unified, and then the neural network is used to monitor the traffic within a period of time and then send it at the appropriate time. In the third stage, the receiver uses the same neural network parameters as when the sender compressed it to restore the compressed representation.

[0073] As a specific embodiment, in step S1, because the quality of the transmitted video is uneven, preprocessing the video materials with different problems is crucial for the subsequent compression and decompression process.

[0074] First, the noisy video material is denoised. The present invention adopts a wavelet denoising method, as shown in formula (1), where ψ a,b (t) is a wavelet function, which uses Fourier transform to project the segmented frame into the frequency domain, a is the scale parameter, b is the position parameter, and f(t) is the input image frame. After the transformation, noise removal is performed based on the relationship between the wavelet coefficient and the threshold, as shown in formula (2), where d ij is the wavelet coefficient, and λ is the threshold. Finally, according to formula (3), the inverse wavelet transform is performed to restore the original data without noise.

[0075] W f (a,b)=∫f(t)ψ a,b (t)dt (1)

[0076]

[0077] f(t)=∑ a,b W f (a,b)ψ a,b (t) (3)

[0078] Secondly, for videos with too low resolution, upsampling is used to increase the number of pixels, and for images with too high resolution, Lanczos resampling technology is used to maintain image quality when adjusting the resolution. In the present invention, OpenCV and FFmpeg are combined for image preprocessing.

[0079] like Figure 3 As shown in the figure, for videos with too much noise, they are first denoised to prevent the noise from being mistaken for video details in the compression model and thus further amplified. The purpose of adjusting the resolution is to make the image detection window size learned by the model parameters within a certain range, so that the clarity restored during reconstruction is higher.

[0080] As a specific embodiment, CNN is used for feature extraction of the initial frame: the preprocessed image frame Xt is first fed into a convolution layer (Conv) of size 3xHxW for feature extraction, where H and W are the height and width of the frame, and the calculation process is shown in formula (4), where * represents the convolution operation, (x, y) represents the position in the output feature map, and (m, n) represents the area covered by the convolution kernel in the input feature map. The convolution kernel h slides over the input feature map f, and at each position, the area of ​​the input image covered by the convolution kernel is multiplied element by element with the elements of the convolution kernel, and then these products are summed to obtain the value of the output feature map at the current position.

[0081] g(x,y)=(f*h)(x,y)=∑ m ∑ n f(m,n)·h(xm,yn) (4)

[0082] The output of the convolutional layer is then sent to a MaxPooling layer of size NxHxW for pooling. The present invention adopts the maximum pooling method, that is, the point with the largest pixel in an NxN pooling window is used as the pooling representation. Its purpose is to reduce the feature dimension and thus enhance the generalization ability and robustness of the model.

[0083] It is expressed as shown in formula (5), where f is the output of Conv.

[0084]

[0085] Finally, the pooled representation is input into a residual layer to link the input to the output, in order to reduce the problem of gradient vanishing and improve the training speed and convergence. It is specifically composed of two activation functions ReLu (RectifiedLinearUnit) and two convolutional layers of size N xkx 1. The mathematical expression of ReLu is shown in formula (6). When the input x is greater than 0, the function outputs x. When the input x is less than or equal to 0, the function outputs 0. In ResBlock, two ReLu functions are used. The first ReLU is placed after the convolutional layer, and the second one is used before adding the residual of the input. This can help the network learn effective features while maintaining the gradient flow.

[0086] f(x)=max(0,x) (6)

[0087] Transformer is used for feature extraction of the initial frame: Considering that the cost of calculating the entire graph is high and the semantic information is unclear, the present invention uses a Transformer-based hierarchical attention model Swin Transformer and improves it. The calculation principle and the calculation process adopted by the present invention are shown in the following steps:

[0088] a) Window division: The core of the Transformer architecture used in this invention is the self-attention mechanism of the moving window, which is based on the local correlation of image data, which is reflected in the fact that adjacent pixels in the image frame data often present similar colors, brightness and textures, and a certain physical part in the image is continuous within a certain spatial range. Based on this idea, this invention divides the feature map extracted by CNN in the previous step, and the key is to divide it into multiple non-overlapping small windows.

[0089] b) Self-attention calculation: Next, the self-attention score is calculated independently in each window. The calculation process is shown in formula (7), where Q, K, and V are query, key, and value matrices respectively, and d k It is the dimension of the key. In the present invention, Q, K, and V specifically represent the input feature map obtained by linear transformation, which is achieved by multiplying Q, K, and V by the weight matrix W plus the bias b.

[0090]

[0091] c) Multi-scale information integration: In the first step, it is proposed to divide the entire image into independent windows, and in the second step, the self-attention and the correlation of the image pixels in each window are calculated independently. In this step, the information in different windows interacts to express the semantic information of the image.

[0092] In the feature extraction module of the previous step, CNN and Transformer are used to extract the semantic information of the input image frame. However, even if the ResBlock residual module is used, there is still a risk of model overfitting and low computational efficiency. Therefore, a variation coding module is used in the present invention to solve this problem.

[0093] Its specific implementation includes three steps: the first step: directly connect the initial feature map (CNN extraction) to the output, the purpose is to reduce the loss of original semantic information due to the complexity of the model during the training process; the second step: pass the initial feature map through N ResBlock modules and connect it to the output, the purpose is to extract the unchanged semantic information of the original feature map; the third step: pass the initial feature map through Swin Transformer and an activation function Sigmoid and connect it to the output. The purpose is mentioned in the description of the second module. The Sigmoid activation function used is shown in formula (8).

[0094]

[0095] Fast convolution strategy: In order to reduce the computational complexity and thus improve the compression efficiency, a new fast convolution and deconvolution strategy is proposed in the present invention, and its mathematical expression is shown in formula (9). Wherein, X, W and V represent the input block of size p×p, the weight of k×k and the output block of m×m, respectively. ⊙ represents the Hadamard product, and A, B and G are transformation matrices, which can be adjusted as hyperparameters during the model training process. Compared with calculating each element in the output feature map separately, the fast algorithm uses structural similarity to generate output blocks. This method significantly reduces the computational complexity of convolution and deconvolution.

[0096] V=A T GWG T ⊙B T XBA (9)

[0097] Furthermore, in order to reduce the redundancy between video frames and effectively reduce the data transmission and storage requirements, the present invention proposes a new motion compensation algorithm. By combining local motion context compensation and global context motion compensation, the semantic information between videos can be fully combined. The specific implementation principle of the invention is as follows:

[0098] Video local motion compensation: The present invention uses the multi-scale features of time propagation to perform local motion compensation, and first performs local compensation when the second module compresses the data. Local compensation focuses on small-scale motion in the video, such as slight movement of an object or slight shaking of the camera.

[0099] a) First, for the features extracted from the current frame in the second module, multi-scale information will also be extracted from the previous frame of the current frame, so that the model can capture image information from different resolution levels, thereby improving the robustness of the model. The present invention uses Laplacian pyramid and Harris corner detection technology to obtain multi-scale information

[0100] b) Then the detected multi-scale information is aligned, and the present invention uses the Lucas-Kanade optical flow algorithm to estimate the motion vector These vectors describe the path that the features move between consecutive frames.

[0101] c) Deform the extracted multi-scale features through motion vectors to form a local context corresponding to the current frame c 1t ,

[0102] Finally, after these three steps, the current frame after local compensation can be obtained. It is expressed as shown in formula (10), x t The representation of the current frame is extracted. In this way, the network can well understand how to perform conditional encoding and also prepare for decoding recovery.

[0103]

[0104] Global Motion Compensation: In the second module, we have mentioned the use of Swin Transformer for feature extraction. At the same time, we can use the characteristics of its global attention mechanism to perform global motion compensation. For a given compressed frame with multi-scale information Calculate the intermediate feature y 1t ,y 2t .Will and The similarity score is calculated and then concatenated with the attention score to generate the global motion compensation code. The similarity calculation formula and the expression of global motion compensation are shown in formula (11) and formula (12).

[0105]

[0106] In the second module, the extracted features are obtained, and after the change coding and motion compensation, the compressed image data is obtained. In this module, the compressed image data is restored to the image frame data. In the present invention, it is specifically implemented through the ResBlock and deconvolution (DeConv) modules. It should be pointed out that the deconvolution here uses the fast algorithm proposed in the third module. Figure 3 In the , the image data to be compressed is divided into frames. Figure 4The input is Figure 3 Output.

[0107] In an optional implementation, the video to be transmitted is input into the system, and the system first performs preliminary cleaning on the video according to the video resolution, color points, noise, etc. to obtain a video representation that is conducive to compression while maintaining fidelity, and then adjusts the resolution to a uniform size for the model. Figure 3 As shown, feature extraction of the image frame is started, and the image features are represented by dimensionality reduction. The purpose is to compress it so that its transmission is more efficient. Deep learning technology is used for compression, which is more accurate and has higher compression efficiency than traditional manual extraction of vital signs.

[0108] like Figure 3 The feature compression module combines CNN and Transformer in feature extraction. When using CNN, the present invention utilizes fast convolution and deconvolution strategies. Its calculation formula has been mentioned in the technical solution. It is improved in implementation as shown in formula (13). The only difference is that the mask matrix M is added. Its purpose is to reduce the computational complexity in the actual process and reduce the expression of insufficient dimensions in the convolution calculation.

[0109] V=A T [(M⊙GWG T )⊙(B T XB)]A (13)

[0110] CNN extracts image edge and texture features, but its ability to understand the relationship between video frames and time series is insufficient. Therefore, Transformer is added during feature extraction to extract dynamic features and enhance the features extracted by CNN. Since the traditional Transformer architecture requires the entire image as input, which greatly increases the computational complexity, the moving window mechanism of SwinTransformer is used to calculate the attention score only in each small window, which is very fast. Then the information between different windows is integrated to obtain a compressed representation. In the example, the number and size of the attention mechanism can be adjusted to achieve the best compression effect and balance.

[0111] like Figure 3 The transform coding module shown in the example constructs three channels. The first channel directly connects the input extracted by CNN with the output, which is similar to the idea of ​​the ResBlock residual layer, preventing the gradient vanishing and gradient exploding problems during the training of the compression model. The second channel associates the Swintransformer with the output through the activation function to capture the dynamic sequence relationship. Finally, the third channel learns the invariant semantic representation of multiple ResBlocks to learn more complex feature representations.

[0112] The present invention proposes global motion compensation and local motion compensation to model the information between two or more frames. The application of global and local motion compensation technology in the patent example shows how to optimize learning video compression by combining flow networks and cross-attention mechanisms. Local motion compensation is mainly implemented through flow networks, which is used to accurately estimate and compress motion information between reference frames and current frames. This mainly involves capturing local redundancy between frames at the multi-scale feature level. Global motion compensation uses a cross-attention mechanism and is performed in the feature domain to capture long-range inter-frame similarities, which helps to understand and encode large-scale motion and global correlations. In addition, in order to deal with the high complexity of traditional cross-attention, the present invention divides the softmax operation into two independent operations, thereby reducing the complexity from quadratic to linear, making it possible to process high-resolution videos. This combination of local and global motion compensation significantly improves compression efficiency and video quality, proving its effectiveness in learning video compression.

[0113] The second purpose of the present invention is to provide a technology based on a custom UDP protocol and machine learning, which, on the one hand, utilizes efficient client and server data transmission to improve efficiency, and on the other hand, utilizes machine learning to select the most unobstructed network time for transmission to reduce data loss and thus avoid retransmission overhead, and ultimately provides a robust video data transmission method:

[0114] 1) The enhanced UDP transmission protocol is used to customize the structure of the data packet between the customized server and the client, and the server dynamically selects the error control and retransmission strategy according to the bandwidth between the server and the client.

[0115] 2) Network traffic prediction technology based on machine learning: When the network is congested, data should be sent to the network less, and when the network is unobstructed, data should be sent to the network quickly. The geographical location and network traffic of the client and the server should be regular. The present invention uses machine learning technology to detect and capture network traffic and predict network traffic for a period of time in the future to achieve efficient data transmission to the receiver. The specific implementation is as follows.

[0116] The detection software monitors the traffic distribution of the sender within a certain area and time, and then performs unsupervised classification on the detected data.

[0117] a) First, the collected traffic is labeled with time-related tags;

[0118] b) Perform data preprocessing to determine the data seasonal cycle, and then divide the collected flow data into small segments of length N, each segment including a complete seasonal cycle;

[0119] c) using a clustering algorithm to identify the data set in the previous step as K different groups. Specifically, the present invention uses a K-means clustering algorithm and uses DTW as a distance metric. The specific calculation formula is shown in formula (14), where x and x′ are two time series, d is the Euclidean distance, and π is the set of all possible time series alignment paths;

[0120] d) Train a prediction model for each group, using 70% of each group as a training set, 20% as a validation set, and 10% as a test set. The present invention uses a long short-term memory network to construct, which now includes an input gate, a forget gate, a unit state, and an output gate. The specific settings are as follows: the LSTM input is 24h traffic monitoring data, the number of stacking layers is 48, and feature engineering is used to distinguish weekend and weekday traffic, and only features with a Pearson coefficient greater than 0.9 are used.

[0121]

[0122] Specifically, Figure 2 As shown in the figure, the network traffic data is first clustered through unsupervised clustering technology to identify and separate data groups with similar time characteristics. Subsequently, a dedicated long short-term memory (LSTM) network model is trained for each data group. These models use the traffic data of the past 24 hours to predict the network traffic in the next hour. This technology not only improves the accuracy of the prediction, but also can be dynamically adjusted according to real-time data to ensure that the model can maintain optimal performance in different network environments. In addition, this strategy enables the system to adapt to new or unknown traffic patterns by continuously collecting new traffic data and retraining the model, so that it can continue to operate effectively in a dynamically changing network environment.

[0123] The image data transmission method based on multi-scale compression and time series analysis of the present invention is described in detail below in conjunction with Example 2.

[0124] Example 2

[0125] The present invention compares the proposed multi-scale compression and timing analysis image data transmission algorithm with several existing commonly used algorithms, mainly measuring their performance in compression efficiency, video quality, computational complexity and network adaptability. The comparison results are shown in Table 1.

[0126] The YouTube-8M dataset is used in the experiments, which contains diverse high-resolution video clips and is suitable for evaluating the performance of video compression algorithms.

[0127] In order to compare the performance of different algorithms, the experiment uses the following common quantitative indicators:

[0128] Compression ratio (%): Measures the ratio of the compressed file size to the original file size. The higher the better.

[0129] PSNR (dB): Peak signal-to-noise ratio, used to evaluate the quality of compressed video. The higher the PSNR, the better the quality.

[0130] SSIM: Structural Similarity, quantifies the visual similarity between the compressed image and the original image, with 1 being perfect similarity.

[0131] Delay (ms): The time delay caused by video compression and transmission. The lower the better.

[0132] Computational complexity (GFLOPs): The number of floating-point operations consumed during the calculation process. The lower the complexity, the higher the calculation efficiency.

[0133] Comparison algorithms:

[0134] H.265 (HEVC): Achieves higher compression efficiency by utilizing advanced intra-frame and inter-frame prediction algorithms to reduce redundant data.

[0135] CNN+RNN compression algorithm based on deep learning: CNN is used to extract the spatial features of video frames, while RNN is good at processing time series information and is particularly suitable for modeling the association between video frames.

[0136] Swin Transformer: An image and video compression method based on self-attention mechanism. It can extract spatial and temporal features of video frames in a multi-scale hierarchical manner, thereby effectively compressing videos.

[0137] Table 1 Comparison of compression efficiency of three different algorithms

[0138]

[0139] Through comparative analysis, the multi-scale compression and timing analysis algorithm of the present invention has demonstrated significant advantages in multiple key performance indicators.

[0140] First, the algorithm achieved a compression rate of 38%-42%, which is a significant improvement over traditional H.265 (25%-30%) and CNN+RNN (30%-35%). In terms of video quality, the PSNR value reached 42dB-46dB, and the SSIM value remained above 0.95, significantly better than H.265's 35dB-37dB and 0.9-0.94 SSIM performance.

[0141] Secondly, with the help of timing analysis technology, the present invention can dynamically adapt to network changes and control the delay to below 50ms, which is significantly lower than high computational complexity algorithms such as CNN+RNN and SwinTransformer.

[0142] In addition, by optimizing the convolution and deconvolution strategies, the computational complexity of this algorithm is controlled at around 2.5 GFLOPs, which is lower than the 5.0 GFLOPs of Swin Transformer, ensuring low computational overhead while achieving efficient compression.

[0143] In summary, the algorithm of the present invention has all-round advantages in terms of compression rate, video quality, delay and computational complexity, and is particularly suitable for video transmission tasks with different resolutions in a dynamic network environment.

[0144] Finally, it should be noted that the embodiments described in detail above are only the best practices of the invention and cannot be used to limit the scope of rights of the invention. Equivalent replacement of the technical solutions recorded in the aforementioned embodiments does not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the invention, and they should all be included in the scope of the claims and description of the invention.

Claims

1. An image data transmission method based on multi-scale compression and time series analysis, characterized in that: The following steps are involved: S1: The transmitting end preprocesses the transmitted video data to obtain a preprocessed video frame; S2: extracting features of the video frame using CNN and a Transformer-based hierarchical attention model to obtain an output feature map, and reducing the loss of semantic information of the video frame by performing multi-channel encoding on the output feature map; S3: Modeling information between video frames through local motion compensation and global motion compensation to obtain multi-scale information, and processing the multi-scale information to obtain compressed video data; S4: By establishing a long short-term memory network model, a timing is selected to send the compressed video data to a video receiver, and the receiver uses the same neural network parameters as those used for compression by the sender to restore the video data.

2. The image data transmission method based on multi-scale compression and time series analysis according to claim 1, characterized in that: The pretreatment specifically comprises the following steps: The wavelet denoising method is used to denoise the noisy image data; the pixel points of the low-resolution image data are increased by up-sampling, and the resolution of the high-resolution image data is guaranteed by Lanczos resampling technology.

3. The image data transmission method based on multi-scale compression and time series analysis according to claim 1, characterized in that: The CNN described in S2 specifically includes the following steps: S31: Use a convolutional layer with a size of 3x H x W to extract features from the video frame to obtain an output value of an output feature map of the video frame at a current position; S32: Send the output value to a MaxPooling layer of size NxHxW for pooling to obtain a pooled representation; S33: The pooled representation is input into a residual layer consisting of two activation functions ReLu and two convolutional layers of size Nxk x1, and the gradient vanishing is reduced by linking the video frame to the output value to extract key visual features.

4. The image data transmission method based on multi-scale compression and time series analysis according to claim 1, characterized in that: The establishment of the hierarchical attention model specifically includes the following steps: Divide the output feature map into multiple non-overlapping independent windows; Independently calculating the self-attention score and the correlation calculation of the image pixels for the independent window; The semantic information of the video frame is obtained by interactively expressing the correlation between the independently calculated self-attention scores of the independent windows and the image pixels.

5. The image data transmission method based on multi-scale compression and time series analysis according to claim 1, characterized in that: The multi-channel encoding process specifically comprises the following steps: Connect the output feature map directly to the output of the multi-channel encoder; The output feature map passes through N ResBlock modules and is connected to the output of the multi-channel encoder; The output feature map is passed through SwinTransformer and an activation function Sigmoid, and connected to the output of the multi-channel coder. At the same time, fast convolution and deconvolution strategies are used to reduce the computational complexity.

6. The image data transmission method based on multi-scale compression and time series analysis according to claim 5, characterized in that: The calculation formula of the fast convolution and deconvolution strategy is: V=A T GWG T ⊙B T XBA Where ⊙ represents the Hadamard product, V is the m×m output block, W is the k×k weight, X is the p×p input block, and A, B, and G are transformation matrices.

7. The method for transmitting image data based on multi-scale compression and time series analysis according to claim 1, characterized in that: The local motion compensation process specifically includes the following steps: Extracting multi-scale information from a previous frame of the video frame; Aligning the multi-scale information and estimating the motion vector using the Lucas-Kanade optical flow algorithm; The multi-scale information is feature deformed by using the motion vector to form a local context of the corresponding video frame, thereby obtaining a video frame after local compensation.

8. The image data transmission method based on multi-scale compression and time series analysis according to claim 1, characterized in that: The global compensation process specifically includes the following steps: An intermediate feature is calculated based on the multi-scale information, a similarity score is calculated between the multi-scale information and the intermediate feature, and then the similarity score is concatenated with the attention score to generate a global motion compensation code.

9. The image data transmission method based on multi-scale compression and time series analysis according to claim 1, characterized in that: The establishment of the long short-term memory network model specifically includes the following steps: Capturing network traffic data and labeling the traffic data with time-related tags; The seasonal period of the flow data is determined by preprocessing the flow data, and then the flow data is divided into N small segments, each segment including a complete seasonal period, to obtain a flow data set; Using a clustering algorithm to identify the traffic data set into K different groups; For the K different groups, a prediction model is trained respectively using a long short-term memory network.

10. An image data transmission system based on multi-scale compression and time series analysis, characterized in that: It includes video preprocessing module, feature extraction module, transform coding module, motion compensation module, decoding and post-processing module; The video preprocessing module is used for the transmitting end to preprocess the transmitted video data to obtain the preprocessed video frame; The feature extraction module receives and extracts features from the video frame using a CNN and a Transformer-based hierarchical attention model to obtain an output feature map; The transform coding module is used to reduce the loss of semantic information of the video frame to obtain an output feature map; The motion compensation module receives the compressed video frames to model the information between the video frames to obtain compressed video data; The decoding and post-processing module restores the compressed video data to the original video data.

Images