Video transmission method and system based on streaming processing

Abstract

The invention discloses a video transmission method and system based on streaming processing, and relates to the technical field of video transmission, and the method comprises the steps: collecting a video stream, carrying out the preprocessing, extracting a video frame, packaging the video frame into a picture stream, carrying out the data fragmentation, distributing an initialization path for fragmented data, and calculating an initial traffic proportion; and optimizing path selection and adjusting path flow distribution according to network feedback to obtain a path optimization result, and transmitting the fragmented data according to the path optimization result. Through multi-level feature extraction and key frame selection of video frames, fragmented packaging of video data is realized, a dynamic path optimization technology is combined, path selection and flow distribution are dynamically adjusted based on a Q learning algorithm and a luminescence firefly algorithm, the transmission efficiency is improved, video image processing and a dynamic network optimization technology are fully combined, and the video image processing efficiency is improved. And a brand new solution is provided for efficient and stable transmission of high-definition video streams.

Description

Technical Field

[0001] The present invention relates to the technical field of video transmission, and in particular to a video transmission method and system based on streaming processing. Background Art

[0002] With the rapid development of the Internet and multimedia technology, video transmission has become an important field in data communication. Among the traditional methods of video transmission, fixed-bitrate video streaming protocols (such as RTMP and HLS) are widely used for the transmission of real-time video streams. However, the fixed-bitrate method is difficult to adapt to complex network environments. When the network bandwidth fluctuates, it is easy to cause problems such as freezing and image quality degradation. In recent years, the introduction of streaming processing technology has provided a new solution for the real-time and stability of video transmission. Streaming processing can process data streams in slices, and combined with network dynamic adjustment algorithms, it makes video transmission more flexible and efficient. In order to further improve the transmission quality, academia and industry have proposed a variety of optimization strategies. For example, the transmission method based on adaptive streaming technology significantly improves the network utilization efficiency by dynamically adjusting the video quality and bit rate by real-time monitoring of the network status. At the same time, the application of deep learning technology makes the preprocessing and feature extraction of video frames more refined, providing support for complex tasks such as semantic recognition and scene detection. However, the performance of existing solutions in terms of efficient path optimization, dynamic traffic allocation and image quality optimization is still limited, and the adaptability to complex video content and non-ideal network environments is weak. Summary of the invention

[0003] In view of the above existing problems, the present invention is proposed.

[0004] Therefore, the present invention provides a video transmission method and system based on streaming processing, which solves the problem that the existing solutions still have limited performance in terms of efficient path optimization, dynamic traffic allocation and image quality optimization, and have weak adaptability to complex video content and non-ideal network environment.

[0005] In order to solve the above technical problems, the present invention provides the following technical solutions: In a first aspect, the present invention provides a video transmission method based on streaming processing, which comprises: Collect video streams, perform preprocessing, and extract video frames. Encapsulate the video frames into image streams and perform data slicing. Assign initialization paths to the slicing data and calculate the initial traffic ratio. Optimize path selection and adjust path traffic distribution according to network feedback to obtain path optimization results, and transmit shard data according to the path optimization results; After receiving the fragmented data, the fragmented data is reassembled into a video stream and the image quality is optimized, and the transmitted video data is stored.

[0006] As a preferred solution of the video transmission method based on streaming processing described in the present invention, wherein: the collecting of video streams and preprocessing and then extracting video frames refers to using a high-resolution camera device to capture a high-definition video stream in real time at a frame rate of 30fps, storing the collected video stream in a ring buffer in units of frames, marking the timestamps of the video frames and sorting them by the timestamps to form a frame sequence, and preprocessing the frame sequence; For each frame of image A, pixel values ​​are extracted from the RGB channels and color moments are calculated, including extracting the color mean C1, variance C2 and skewness C3, and combining C1, C2 and C3 to form a color feature vector; Convert each frame of image into a grayscale image, calculate the co-occurrence matrix of each pair of pixels based on the neighborhood relationship, calculate the texture features of each frame of image A, including contrast B1, homogeneity B2 and correlation B3, and combine B1, B2 and B3 to form a texture feature vector; Based on the grayscale image of texture analysis, the frequency of occurrence of each grayscale value is counted and normalized to the probability distribution p(f). Based on p(f), the Shannon entropy E of each frame image is calculated. f , measures the information complexity of the image; Load the pre-trained VGG16 model, freeze the convolutional layer weights and select Conv1 to Conv5 as the multi-layer feature output layer. Each layer extracts low-level structural features and high-level semantic features respectively. Input the frame sequence into the VGG16 model and extract features layer by layer. Calculate the gradient amplitude of the frame sequence and count the frequency distribution of the gradient amplitude to generate a multi-channel histogram H M ; Combine the multi-layer convolutional features of VGG16 with the HGMF-MC features to obtain the fused deep feature vector F, fuse the deep feature vector F with the color, texture, and information complexity features to obtain the final deep feature vector, and store the feature vectors of all frames as a feature vector sequence; Use convolutional neural network as the classification model, input the comprehensive feature vector of each frame into the classification model, calculate the classification probability, set the classification threshold D, if the classification probability is greater than or equal to the classification threshold D, then mark the current frame as a semantic scene frame, otherwise mark the current frame as a static scene frame, and mark the frame sequence as a semantic scene frame set and a static scene frame set; Calculate the gradient component and gradient amplitude of each frame in the semantic scene frame set, divide the gradient amplitude of each frame into multiple channels, and perform bucket statistics on the gradient amplitude of each channel to construct the histogram HGMF; Calculate the difference value of the histogram HGMF of each pair of adjacent frames one by one, calculate the mean and standard deviation of the differences between all frames, and set the threshold s according to the sum of the mean and standard deviation; Traverse the semantic scene frame set, select frames whose inter-frame difference values ​​are greater than the threshold s as key frames, and add a timestamp to each key frame; Perform 2D-DWT on each frame in the static scene frame set to extract low-frequency features S; Use the perceptual hashing method to generate a hash value for the low-frequency features of each frame, calculate the difference in hash values ​​of adjacent frames through the Hamming distance, calculate the mean and standard deviation of all hash value differences, and set the threshold g according to the sum of the mean and standard deviation; Traverse the hash value difference sequence, select the frames whose local extremum is greater than g as key frames, and add a timestamp to each key frame; Dynamically calculate the frame rate F based on the actual network bandwidth, number of windows and video content; The key frame set is sorted by timestamp, a frame sequence is selected for key frame extraction according to a dynamically calculated frame extraction frequency, and the extracted key frames are combined into a set.

[0007] As a preferred solution of the video transmission method based on streaming processing described in the present invention, wherein: the step of encapsulating the video frame into a picture stream and then performing data slicing, allocating an initialization path for the slicing data and calculating the initial flow ratio refers to encapsulating the video frame into a JPG picture format through FFmpeg, integrating the picture frames to form a picture stream, setting a time window to segment the picture stream, dividing the picture frame within the time window into a slice, and each slice is marked as F k , where k represents the current shard number; Add a timestamp to each shard, set a priority identifier for each shard according to data transmission requirements, generate a CRC checksum for the complete data content of each shard, append the checksum to the end of the shard to form a complete shard structure, and obtain a shard data sequence that has been marked. Collect network status information of all current paths and normalize the data, use network performance probes to update path status in real time, and store the monitoring results as a path attribute table; The network status information is calculated by weighted fusion method to obtain the comprehensive score of each path, the paths are sorted in descending order according to the comprehensive score, and the first m paths are selected as the valid path set; For the selected path set, the Pareto distribution model is used to calculate the initial flow allocation ratio of each path in the set; According to the fragment priority identifier, the path allocation ratio is adjusted, and the path allocation table is generated by combining the path network status information and the adjusted allocation ratio.

[0008] As a preferred solution of the video transmission method based on streaming processing of the present invention, wherein: the step of optimizing path selection according to network feedback and adjusting path traffic distribution to obtain path optimization results refers to calculating a reward value R of each path according to the network status; Use Q learning algorithm to dynamically update path value, select the optimal path according to the path value and generate the optimal path set; Calculate the delay difference between path pair i and j and calculate the path attractiveness based on the delay difference ; Dynamically adjust the path flow ratio according to the glowing firefly algorithm ; Normalize the flow ratios of all paths, iterate and update the flow ratios of the paths, set the maximum number of iterations, and stop the iteration until the maximum number of iterations is reached to obtain the final flow ratio of the paths. Generate the final optimization weight G for each path based on the final flow ratio and reward value of the path. The shard data is divided into a high priority data set and a second priority data set according to the priority identifier, and the path set is sorted in descending order according to the optimization weight. Thresholds U and O are set, and U>O. If the path weight is greater than or equal to the threshold U, it is a high priority path group, which transmits the high priority data set. If the path weight is greater than or equal to the threshold O and less than the threshold U, it is a medium priority path group, which transmits the second priority data set. If the path weight is less than the threshold O, it is a low priority path group, which is used as a backup path when congestion occurs in the high priority path group and the medium priority path group. Integrate all path allocation results to generate the final optimized path allocation table.

[0009] As a preferred solution of the video transmission method based on streaming processing described in the present invention, wherein: the transmission of sliced ​​data according to the path optimization result refers to traversing each data slice, selecting the target path group according to the priority, initializing the path scheduler, transmitting the slices in the high priority data set to the target receiving end through the low-latency transmission protocol, transmitting the slices in the second priority data set to the target receiving end through the bandwidth optimization protocol, recording the transmission results of each path to generate a transmission log and storing it in a database.

[0010] As a preferred solution of the video transmission method based on streaming processing described in the present invention, wherein: after receiving the fragmented data, reorganizing the fragmented data into a video stream and optimizing the image quality refers to receiving the fragmented data transmitted by each path through the transmission protocol and using a cyclic redundancy check to detect the integrity of each fragment, recording the fragment number that failed the check and requesting retransmission from the sending end, and if the retransmission fails, using adjacent fragments for interpolation compensation; Sorting the slice data according to the slice number and timestamp, merging the slice data of all paths into a complete time sequence according to the number to obtain a complete data frame sequence, initializing the video decoder at the receiving end, decoding the data frame sequence frame by frame, generating a video stream, and correcting the frame errors that occur during the decoding process using error concealment technology; Bilateral filtering is used to remove noise and retain edge details from the decoded image frame sequence, histogram equalization is used to enhance image contrast, and inverse wavelet transform is used to enhance details. The optimized image frame sequence is re-encoded into a complete video stream.

[0011] As a preferred solution of the video transmission method based on streaming processing described in the present invention, wherein: the stored and transmitted video data refers to compressing the video stream using context-adaptive binary arithmetic coding, storing the compressed video stream in layers, and uploading the data to the cloud for backup after storage.

[0012] In a second aspect, the present invention provides a video transmission system based on streaming processing, comprising: Data collection module, used to collect video streams and perform preprocessing; The data slicing and path initialization module is used to encapsulate the preprocessed video frames into image streams and perform data slicing, allocate initialization paths for the slicing, and calculate the initial traffic ratio; Path optimization module, used to dynamically optimize path selection and adjust path traffic distribution based on network feedback; Data transmission module, used to efficiently transmit the shard data to the receiving end according to the path optimization result; The image quality optimization module is used to receive the fragmented data and reassemble it into a video stream while optimizing the image quality; The storage backup module is used to compress and store the transmitted video stream and upload it to the cloud for backup.

[0013] In a third aspect, the present invention provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: when the computer program is executed by the processor, any step of the video transmission method based on streaming processing as described in the first aspect of the present invention is implemented.

[0014] In a fourth aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein: when the computer program is executed by a processor, it implements any step of the video transmission method based on streaming processing as described in the first aspect of the present invention.

[0015] The beneficial effects of the present invention are as follows: the present invention realizes the fragmentation encapsulation of video data through multi-level feature extraction and key frame selection of video frames, combines dynamic path optimization technology, dynamically adjusts path selection and traffic distribution based on Q learning algorithm and luminous firefly algorithm, improves transmission efficiency, fully combines video image processing and dynamic network optimization technology, and provides a new solution for efficient and stable transmission of high-definition video streams. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0017] Figure 1 is a flow chart of the video transmission method based on streaming processing in Example 1; Figure 2 Schematic diagram of a video transmission system based on streaming processing in Example 1. DETAILED DESCRIPTION

[0018] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the accompanying drawings.

[0019] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0020] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The term "in one embodiment" that appears in different places in this specification does not necessarily refer to the same embodiment, nor does it refer to a separate or selective embodiment that is mutually exclusive with other embodiments.

[0021] Example 1, reference Figure 1 and Figure 2 , which is the first embodiment of the present invention, provides a video transmission method based on streaming processing, comprising the following steps: S1, collect video streams, perform preprocessing, extract video frames, encapsulate video frames into picture streams, perform data slicing, allocate initialization paths for slicing data, and calculate initial traffic ratio; Specifically, collecting the video stream and extracting the video frame after preprocessing means using a high-resolution camera device to capture a high-definition video stream in real time at a frame rate of 30fps, storing the collected video stream in a ring buffer in units of frames, marking the timestamps of the video frames and sorting them by the timestamps to form a frame sequence, and preprocessing the frame sequence; The preprocessing refers to using a local difference detection method to detect salt and pepper noise, and marking the video frame in which the salt and pepper noise is detected as type 1, and marking other noises as type 2; A median filter is applied to frames marked as type 1 to remove salt and pepper noise, and a bilateral filter is applied to frames marked as type 2 to preserve edge details; Through salt and pepper noise detection and a variety of filtering methods (such as median filtering and bilateral filtering), image noise can be effectively removed, edge details of frames can be retained, and frame quality can be improved. Through timestamp marking and sorting, the timing consistency of frame sequences can be ensured, supporting subsequent dynamic optimization strategies based on time information. For each frame of image A, pixel values ​​are extracted from the RGB channels and color moments are calculated, including extracting the color mean C1, variance C2 and skewness C3, and combining C1, C2 and C3 to form a color feature vector; Each frame of image is converted into a grayscale image, and the co-occurrence matrix of each pair of pixels is calculated based on the neighborhood relationship. The texture features of each frame of image A are calculated, including contrast B1, homogeneity B2, and correlation B3. B1, B2, and B3 are combined to form a texture feature vector. The calculation formulas of B1, B2, and B3 are as follows: , , , Where ij is the gray level index, P(i,j) is the joint probability of position (i,j) in the gray level co-occurrence matrix, and is the normalized mean of the gray levels, is the normalized standard deviation of the gray level; RGB color moment and grayscale co-occurrence matrix provide multi-dimensional frame description information, which enhances the ability of video frames to distinguish scene changes. High-quality frames achieve deep information analysis through multi-level feature extraction, while color and texture features provide a powerful supplement to deep features and improve classification accuracy. The pre-processed high-quality frames provide accurate basic data such as color, texture and timestamp for key frame extraction, ensuring accurate selection of key frames. Based on the grayscale image of texture analysis, the frequency of occurrence of each grayscale value is counted and normalized to the probability distribution p(f). Based on p(f), the Shannon entropy E of each frame image is calculated. f , to measure the information complexity of the image:

[0022] Shannon entropy as information complexity characteristic; Load the pre-trained VGG16 model, freeze the convolutional layer weights and select Conv1 to Conv5 as the multi-layer feature output layer. Each layer extracts low-level structural features and high-level semantic features respectively. Input the frame sequence into the VGG16 model and extract features layer by layer: Conv1 layer: captures the edge and contour information of the image; Conv2 layer: extract local texture features; Conv3 layer: generates shape and area features of the middle layer; Conv4 layer: captures complex image structure information; Conv5 layer: aggregates high-level semantic features; Calculate the gradient amplitude of the frame sequence and count the frequency distribution of the gradient amplitude to generate a multi-channel histogram H M ; From Conv1 to Conv5 layers, low, medium, and high-level features are extracted respectively to ensure that information of different granularities is fully utilized. By freezing the weights of the convolutional layer, the capabilities of the pre-trained model are fully utilized to capture the semantic hierarchical information of the image and improve the recognition ability of the classification model. The multi-channel histogram HGMF and color and texture features are combined to generate a fused deep feature vector F, which improves the comprehensiveness and expressiveness of the features. Combine the multi-layer convolutional features of VGG16 with the HGMF-MC features to obtain the fused deep feature vector F, fuse the deep feature vector F with the color, texture, and information complexity features to obtain the final deep feature vector, and store the feature vectors of all frames as a feature vector sequence; The extraction of deep features depends on high-quality frames. Frames that have undergone noise removal and feature extraction can better reflect their low-level and high-level semantic features. Multi-level deep features provide rich semantic information support for frame classification and key frame extraction, enhance the accurate evaluation of inter-frame differences, and promote the accuracy of key frame selection. The fusion of deep features, color features, texture features, and information complexity features makes the stored feature sequences more distinguishable and applicable. Use convolutional neural network as the classification model, input the comprehensive feature vector of each frame into the classification model, calculate the classification probability, set the classification threshold D based on the ROC curve, and if the classification probability is greater than or equal to the classification threshold D, mark the current frame as a semantic scene frame, otherwise mark the current frame as a static scene frame, and mark the frame sequence as a semantic scene frame set and a static scene frame set; By classifying semantic scene frames and static scene frames, unnecessary static data is reduced from participating in subsequent transmission and optimization, thus improving efficiency. Based on the comprehensive feature vector input, the classification performance of the convolutional neural network is better than that of the traditional classifier, and the classification accuracy is higher. By classifying scenes, the amount of data that needs to be transmitted and stored is reduced, and the efficiency of system resource utilization is improved. The classification model directly uses the generated deep feature vectors, and the classification performance depends on the comprehensiveness and accuracy of the feature expression. The classified semantic scene frame set and static scene frame set provide basic data for key frame extraction, ensuring that the key frame selection is concentrated in scenes with significant semantic changes, thus reducing the interference of meaningless data. Calculate the gradient component and gradient amplitude of each frame in the semantic scene frame set, divide the gradient amplitude of each frame into multiple channels, and perform bucket statistics on the gradient amplitude of each channel to construct the histogram HGMF; Calculate the difference value of the histogram HGMF of each pair of adjacent frames one by one, calculate the mean and standard deviation of the differences between all frames, and set the threshold s according to the sum of the mean and standard deviation; Traverse the semantic scene frame set, select frames whose inter-frame difference values ​​are greater than the threshold s as key frames, and add a timestamp to each key frame; Perform 2D-DWT on each frame in the static scene frame set to extract low-frequency features S:

[0023] Where H and W are the height and width of the image, f(x,y) is the grayscale value of the image, is the Haar function, is the wavelet scale currently used, h is the translation parameter of the wavelet function in the vertical direction, w is the translation parameter of the wavelet function in the horizontal direction, x and y are the position indexes of the pixel in the horizontal and vertical directions; Use the perceptual hashing method to generate a hash value for the low-frequency features of each frame, calculate the difference in hash values ​​of adjacent frames through the Hamming distance, calculate the mean and standard deviation of all hash value differences, and set the threshold a according to the sum of the mean and standard deviation; Traverse the hash value difference sequence, select the frame with the local extremum greater than a as the key frame, and add a timestamp to each key frame; The key frame selection strategy is based on the difference in gradient amplitude and hash value between frames to ensure that the key frame reflects the main information of semantic changes; Dynamically calculate the frame rate F based on the actual network bandwidth, number of windows and video content:

[0024] Where L is the total frame processing capacity, which is determined by the maximum processing capacity supported by the system hardware (CPU, GPU), q is the number of parallel windows, and Y is the priority factor of the key frame; The key frame sets are sorted by timestamp, and the frame sequence is selected for key frame extraction based on the dynamically calculated frame extraction frequency, and the extracted key frames are combined into sets. The dynamic frame extraction frequency method dynamically adjusts the number of key frames according to the network bandwidth, which improves the flexibility of transmission. Only key frame sets are transmitted, which significantly reduces data redundancy and optimizes bandwidth utilization. Key frame extraction depends on the classification results of semantic scene frame sets and static scene frame sets. The key frames extracted from the semantic scene frames can effectively reflect the main changes in the video content. The extracted key frame sets provide a high-value data source for subsequent feature fusion, transmission and storage.

[0025] Combining deep learning models and dynamic optimization algorithms, it not only comprehensively extracts the multi-dimensional features of the image (such as color, texture, information complexity, etc.), but also greatly reduces data redundancy through semantic scene frame classification and key frame extraction. Dynamic frame extraction frequency adjustment and path optimization strategy accurately adapt to the volatility of network bandwidth, effectively improving the reliability and efficiency of transmission. At the same time, through error hiding and image quality enhancement technology, high-fidelity presentation after video reconstruction is ensured. The present invention not only solves the problems of low transmission efficiency and insufficient image quality optimization in the prior art, but also realizes the multi-scenario adaptability of video data transmission and processing, providing an innovative and systematic solution for streaming media transmission and intelligent video analysis.

[0026] Furthermore, the video frames are encapsulated into a picture stream and then the data is segmented, an initialization path is allocated for the segmented data, and the initial flow ratio is calculated. The video frames are encapsulated into a JPG picture format through FFmpeg, and the picture frames are integrated to form a picture stream. FFmpeg is a widely used open source multimedia processing framework that supports audio and video recording, conversion, streaming and playback. FFmpeg is used to extract frames from the video stream and encapsulate them into a JPG picture format to form a standardized picture stream, which is convenient for subsequent segmentation operations. A time window is set to segment the picture stream, and the picture frames in the time window are divided into a segment. Each segment is marked as F k , where k represents the current shard number. After the data is sharded, it is numbered F k Identification can track the status of each shard, facilitating accurate retransmission when data is lost or delayed; Add a timestamp to each shard and set a priority for each shard based on data transmission requirements. Generate a CRC checksum for the complete data content of each fragment, and append the checksum to the end of the fragment to form a complete fragment structure, and obtain a fragment data sequence with a completion mark. The timestamp ensures the order of the fragment data, and the priority identifier supports dynamic allocation of transmission resources according to the importance of the content. For example, a high priority path is allocated to key frame fragments to ensure the timely transmission of key frames. By appending a CRC checksum to the end of the fragment, the receiving end can quickly verify the integrity of the data fragment. If an error is found, it requests retransmission or corrects it through redundant data, thereby improving the reliability of data transmission. Collect network status information of all current paths and normalize the data, use network performance probes to update path status in real time, and store the monitoring results as a path attribute table; The network status information includes bandwidth, delay and packet loss rate; The network status information is calculated by weighted fusion method to obtain the comprehensive score of each path, the paths are sorted in descending order according to the comprehensive score, and the first m paths are selected as the valid path set; For the selected path set, the Pareto distribution model is used to calculate the initial traffic allocation ratio of each path in the set. Pareto distribution is a probability distribution model, which is usually used to describe the imbalance of path traffic allocation in network traffic. Using the Pareto distribution model to calculate the initial traffic ratio can effectively balance the priority utilization of efficient paths and the backup allocation of redundant paths, and prevent the occurrence of transmission bottlenecks. According to the shard priority identifier, the path allocation ratio is adjusted, and the path allocation table is generated by combining the path network status information and the adjusted allocation ratio. The path allocation ratio is dynamically adjusted by combining the path network status information and the priority identifier, which can adapt to real-time network fluctuations and ensure the high-priority transmission requirements of important data.

[0027] By encapsulating video frames into image streams and transmitting them in fragments, the present invention effectively overcomes the shortcomings of traditional video transmission technology in network environments with high packet loss rates and large bandwidth fluctuations. The addition of fragment numbers and timestamps ensures the traceability and order of fragments; the combination of priority identification and dynamic path optimization algorithm significantly improves the stability and efficiency of data transmission. At the same time, the integrity detection of CRC check and the traffic allocation strategy of Pareto distribution make the transmission process more reliable and efficient. The present invention is suitable for scenarios with strong demand for multi-path transmission optimization and has broad application prospects.

[0028] S2. Optimize path selection and adjust path traffic distribution according to network feedback to obtain path optimization results, and transmit shard data according to the path optimization results; Specifically, the path optimization result obtained by optimizing the path selection and adjusting the path traffic distribution according to the network feedback refers to calculating the reward value R of each path according to the network status:

[0029] In the formula, is the delay of path a, is the packet loss rate of path a, is the bandwidth utilization of path a, and It is the weight factor of packet loss rate and bandwidth utilization, which is adjusted according to application requirements; Use Q learning algorithm to dynamically update path value, select the optimal path according to the path value and generate the optimal path set; The reward value is calculated by weighted fusion of delay, packet loss rate and bandwidth utilization, which can accurately measure path performance in multiple dimensions. Calculate the delay difference between path pair i and j and calculate the path attractiveness based on the delay difference :

[0030] In the formula, is the initial attraction, is the light intensity attenuation coefficient, which is set through historical experience to control the rate of attraction attenuation. is the delay difference between path i and path j, reflecting the difference in performance between the two paths; Dynamically adjust the path flow ratio according to the glowing firefly algorithm :

[0031] In the formula, and is the current traffic ratio of path i and path j, is the path attraction, is the random perturbation factor, is a generated normally distributed random number with mean 0, is the variance; By dynamically adjusting the traffic ratio through path attraction, load balancing can be achieved in multi-path transmission to avoid resource waste or path congestion; Normalize the flow ratios of all paths, iterate and update the flow ratios of the paths, set the maximum number of iterations, and stop the iteration until the maximum number of iterations is reached to obtain the final flow ratio of the paths. According to the final flow ratio and reward value of the paths, generate the final optimization weight G for each path: , The shard data is divided into a high-priority data set and a second-priority data set according to the priority identifier, and the path set is sorted in descending order according to the optimization weight. The thresholds U and O are set by statistical analysis of historical data, and U>O. If the path weight is greater than or equal to the threshold U, it is a high-priority path group and transmits the high-priority data set. If the path weight is greater than or equal to the threshold O and less than the threshold U, it is a medium-priority path group and transmits the second-priority data set. If the path weight is less than the threshold O, it is a low-priority path group, which is used as a backup path when congestion occurs in the high-priority path group and the medium-priority path group. Dividing high-priority path groups, medium-priority path groups, and backup path groups by path weight thresholds can achieve differentiated resource scheduling. Integrate all path allocation results to generate the final optimized path allocation table; The path allocation table includes path number, flow ratio, reward value, comprehensive weight, priority grouping and path flow.

[0032] Through the core steps of reward value calculation, path attraction modeling and glowing firefly algorithm optimization, the path selection, traffic allocation and priority scheduling problems in multi-path transmission scenarios are systematically solved. Combined with the path allocation table and dynamic priority grouping mechanism, this method can not only achieve efficient video data transmission in complex network environments, but also significantly improve the stability and robustness of the system.

[0033] Furthermore, transmitting the fragmented data according to the path optimization result means traversing each data fragment, selecting the target path group according to the priority, improving the transmission success rate of the high-priority fragments, ensuring that the key data is given priority, dynamically adapting to the changes in the network environment, effectively avoiding path congestion, improving the transmission efficiency, initializing the path scheduler, providing dynamic path scheduling capabilities, flexibly adapting to the real-time changes in the network status, reducing the retention time of the data fragments in the low-priority path group, and improving the overall system throughput. The fragments in the high-priority data set are transmitted to the target receiving end through the low-latency transmission protocol (UDP), ensuring that the data transmission of key tasks meets the real-time requirements, such as the key frames in video calls or high-priority business data transmission, and the fragments in the secondary priority data set are transmitted to the target receiving end through the bandwidth optimization protocol (link aggregation protocol LACP), enhancing the bandwidth utilization of the system, reducing the waste of resources caused by idle links, and recording the transmission results of each path to generate a transmission log and store it in the database; The transmission log includes the number of successfully transmitted fragments, packet loss rate and transmission delay.

[0034] The dynamic fragmentation transmission method based on the path optimization results effectively solves the problems of static path selection, insufficient priority scheduling and lack of transmission performance monitoring in the prior art, provides low-latency guarantee for high-priority data transmission, and fully utilizes network bandwidth resources to complete secondary priority data transmission. At the same time, the system establishes a performance feedback mechanism through transmission logs, further enhancing the intelligence and optimization capabilities of the transmission system, and providing strong technical support for streaming video transmission.

[0035] S3, after receiving the fragmented data, reorganize the fragmented data into a video stream and optimize the image quality, and store the transmitted video data; Specifically, after receiving the fragmented data, reorganizing the fragmented data into a video stream and optimizing the image quality means receiving the fragmented data transmitted by each path through the transmission protocol and using a cyclic redundancy check (CRC) to detect the integrity of each fragment, ensuring the integrity of data transmission, quickly identifying transmission errors, and improving transmission reliability, laying the foundation for video stream reorganization, recording the fragment number that failed the check and requesting retransmission from the sender. If the retransmission fails, interpolation compensation is performed using adjacent fragments. The retransmission request mechanism provides the first line of defense to ensure that the repairable errors are minimized. When the retransmission fails, the video data is restored to the greatest extent through the interpolation compensation mechanism to avoid the overall picture interruption caused by the loss of a single fragment. Sort the slice data according to the slice number and timestamp, merge the slice data of all paths into a complete time sequence according to the number to obtain a complete data frame sequence, sort the transmission slices into a complete time sequence according to the slice number and timestamp to ensure the order consistency of the frame data, initialize the video decoder at the receiving end, decode the data frame sequence frame by frame, generate the video stream, initialize the video decoder and decode it frame by frame to ensure the efficient restoration of the video stream, provide accurate frame-level error detection capability, provide input for error concealment technology, and use error concealment technology to correct frame errors that occur during the decoding process; The error correction includes spatial domain correction and temporal domain correction; The spatial domain correction refers to filling the error area with pixel information of adjacent frames, and the temporal domain correction refers to interpolation and repair through adjacent frames on the time axis. The spatial domain correction directly fills the pixel area of ​​the error frame to reduce obvious visual defects. The temporal domain correction uses the time series characteristics to restore the dynamic picture and improve the naturalness and coherence after repair. The comprehensive correction method improves the video quality and is suitable for different scenes and error types. Bilateral filtering is used to remove noise and retain edge details from the decoded image frame sequence, histogram equalization is used to enhance image contrast, and inverse wavelet transform is used to enhance details. The optimized image frame sequence is re-encoded into a complete video stream.

[0036] The CRC checksum and retransmission mechanism ensures high integrity and accuracy of data transmission. The interpolation compensation mechanism effectively copes with irrecoverable data loss and improves the availability of video streams. From image preprocessing to error correction, a variety of optimization technologies are combined to achieve image quality improvement. Through filtering and enhancement technology, the clarity and contrast of decoded videos are improved. The retransmission and correction strategies are dynamically adjusted to adapt to different network conditions to ensure the continuity and stability of video stream transmission. From data reception to reorganization, decoding and optimization, the entire process is covered to build an efficient video transmission method, providing targeted solutions to meet the needs of multiple scenarios and multiple terminals.

[0037] Furthermore, storing the transmitted video data refers to compressing the video stream using context-adaptive binary arithmetic coding, storing the compressed video stream in layers, and uploading the data to the cloud for backup after storage.

[0038] An efficient video data storage solution is built through the three core steps of CABAC compression, tiered storage and cloud backup. CABAC technology significantly improves the compression efficiency of video data and reduces transmission costs. The tiered storage strategy optimizes the utilization of storage resources and improves the access efficiency of key data. Cloud backup enhances data security and scalability, meeting the diverse needs of modern video data management.

[0039] This embodiment also provides a video transmission system based on streaming processing, including: Data collection module, used to collect video streams and perform preprocessing; The data slicing and path initialization module is used to encapsulate the preprocessed video frames into image streams and perform data slicing, allocate initialization paths for the slicing, and calculate the initial traffic ratio; Path optimization module, used to dynamically optimize path selection and adjust path traffic distribution based on network feedback; Data transmission module, used to efficiently transmit the shard data to the receiving end according to the path optimization result; The image quality optimization module is used to receive the fragmented data and reassemble it into a video stream while optimizing the image quality; The storage backup module is used to compress and store the transmitted video stream and upload it to the cloud for backup.

[0040] This embodiment also provides a computer device, which is suitable for the case of a video transmission method based on streaming processing, including: a memory and a processor; the memory is used to store computer-executable instructions, and the processor is used to execute computer-executable instructions to implement the video transmission method based on streaming processing proposed in the above embodiment.

[0041] The computer device may be a terminal, and the computer device includes a processor, a memory, a communication interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be achieved through WIFI, an operator network, NFC (near field communication) or other technologies. The display screen of the computer device may be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device may be a touch layer covered on the display screen, or a key, trackball or touchpad provided on the housing of the computer device, or an external keyboard, touchpad or mouse, etc.

[0042] This embodiment also provides a storage medium on which a computer program is stored. When the program is executed by a processor, the video transmission method based on streaming processing proposed in the above embodiment is implemented; the storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (Static Random Access Memory, referred to as SRAM), electrically erasable programmable read-only memory (Electrically Erasable Programmable Read-Only Memory, referred to as EEPROM), erasable programmable read-only memory (Erasable Programmable Read Only Memory, referred to as EPROM), programmable read-only memory (Programmable Red-Only Memory, referred to as PROM), read-only memory (Read-Only Memory, referred to as ROM), magnetic storage, flash memory, magnetic disk or optical disk.

[0043] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A video transmission method based on streaming processing, characterized in that: include, Collect video streams, perform preprocessing, and extract video frames. Encapsulate the video frames into image streams and perform data slicing. Assign initialization paths to the slicing data and calculate the initial traffic ratio. Optimize path selection and adjust path traffic distribution according to network feedback to obtain path optimization results, and transmit shard data according to the path optimization results; After receiving the fragmented data, the fragmented data is reassembled into a video stream and the image quality is optimized, and the transmitted video data is stored.

2. The video transmission method based on streaming processing as claimed in claim 1, characterized in that: The collecting of video streams and preprocessing and then extracting video frames refers to using a high-resolution camera device to capture a high-definition video stream in real time at a frame rate of 30fps, storing the collected video stream in a ring buffer in units of frames, marking the timestamps of the video frames and sorting them by the timestamps to form a frame sequence, and preprocessing the frame sequence; For each frame of image A, pixel values ​​are extracted from the RGB channels and color moments are calculated, including extracting the mean C1, variance C2 and skewness C3 of the color, and combining C1, C2 and C3 to form a color feature vector; Convert each frame of image into a grayscale image, calculate the co-occurrence matrix of each pair of pixels based on the neighborhood relationship, calculate the texture features of each frame of image A, including contrast B1, homogeneity B2 and correlation B3, and combine B1, B2 and B3 to form a texture feature vector; Based on the grayscale image of texture analysis, the frequency of occurrence of each grayscale value is counted and normalized to the probability distribution p(f). Based on p(f), the Shannon entropy E of each frame image is calculated. f , measures the information complexity of the image; Load the pre-trained VGG16 model, freeze the convolutional layer weights and select Conv1 to Conv5 as the multi-layer feature output layer. Each layer extracts low-level structural features and high-level semantic features respectively. Input the frame sequence into the VGG16 model and extract features layer by layer. Calculate the gradient amplitude of the frame sequence and count the frequency distribution of the gradient amplitude to generate a multi-channel histogram H M ; Combine the multi-layer convolutional features of VGG16 with the HGMF-MC features to obtain the fused deep feature vector F, fuse the deep feature vector F with the color, texture, and information complexity features to obtain the final deep feature vector, and store the feature vectors of all frames as a feature vector sequence; Use convolutional neural network as the classification model, input the comprehensive feature vector of each frame into the classification model, calculate the classification probability, set the classification threshold D, if the classification probability is greater than or equal to the classification threshold D, then mark the current frame as a semantic scene frame, otherwise mark the current frame as a static scene frame, and mark the frame sequence as a semantic scene frame set and a static scene frame set; Calculate the gradient component and gradient amplitude of each frame in the semantic scene frame set, divide the gradient amplitude of each frame into multiple channels, and perform bucket statistics on the gradient amplitude of each channel to construct the histogram HGMF; Calculate the difference value of the histogram HGMF of each pair of adjacent frames one by one, calculate the mean and standard deviation of the differences between all frames, and set the threshold s according to the sum of the mean and standard deviation; Traverse the semantic scene frame set, select frames whose inter-frame difference values ​​are greater than the threshold s as key frames, and add a timestamp to each key frame; Perform 2D-DWT on each frame in the static scene frame set to extract low-frequency features S; Use the perceptual hashing method to generate a hash value for the low-frequency features of each frame, calculate the difference in hash values ​​of adjacent frames through the Hamming distance, calculate the mean and standard deviation of all hash value differences, and set the threshold g according to the sum of the mean and standard deviation; Traverse the hash value difference sequence, select the frames whose local extremum is greater than g as key frames, and add a timestamp to each key frame; Dynamically calculate the frame rate F based on the actual network bandwidth, number of windows and video content; The key frame set is sorted by timestamp, a frame sequence is selected for key frame extraction according to a dynamically calculated frame extraction frequency, and the extracted key frames are combined into a set.

3. The video transmission method based on streaming processing as claimed in claim 2, characterized in that: The method of encapsulating the video frame into a picture stream and then performing data slicing, allocating an initialization path for the sliced ​​data and calculating the initial flow ratio refers to encapsulating the video frame into a JPG picture format through FFmpeg, integrating the picture frames to form a picture stream, setting a time window to segment the picture stream, dividing the picture frame in the time window into a slice, and each slice is marked as F k , where k represents the current shard number; Add a timestamp to each shard, set a priority identifier for each shard according to data transmission requirements, generate a CRC checksum for the complete data content of each shard, append the checksum to the end of the shard to form a complete shard structure, and obtain a shard data sequence that has been marked. Collect network status information of all current paths and normalize the data, use network performance probes to update path status in real time, and store the monitoring results as a path attribute table; The network status information is calculated by weighted fusion method to obtain the comprehensive score of each path, the paths are sorted in descending order according to the comprehensive score, and the first m paths are selected as the valid path set; For the selected path set, the Pareto distribution model is used to calculate the initial flow allocation ratio of each path in the set; According to the fragment priority identifier, the path allocation ratio is adjusted, and the path allocation table is generated by combining the path network status information and the adjusted allocation ratio.

4. The video transmission method based on streaming processing as claimed in claim 3, characterized in that: The optimizing path selection and adjusting the path flow distribution according to the network feedback to obtain the path optimization result refers to calculating the reward value R of each path according to the network status; Use Q learning algorithm to dynamically update path value, select the optimal path according to the path value and generate the optimal path set; Calculate the delay difference between path pair i and j and calculate the path attractiveness based on the delay difference ; Dynamically adjust the path flow ratio according to the glowing firefly algorithm ; Normalize the flow ratios of all paths, iterate and update the flow ratios of the paths, set the maximum number of iterations, and stop the iteration until the maximum number of iterations is reached to obtain the final flow ratio of the paths. Generate the final optimization weight G for each path based on the final flow ratio and reward value of the path. The shard data is divided into a high priority data set and a second priority data set according to the priority identifier, and the path set is sorted in descending order according to the optimization weight. Thresholds U and O are set, and U>O. If the path weight is greater than or equal to the threshold U, it is a high priority path group, which transmits the high priority data set. If the path weight is greater than or equal to the threshold O and less than the threshold U, it is a medium priority path group, which transmits the second priority data set. If the path weight is less than the threshold O, it is a low priority path group, which is used as a backup path when congestion occurs in the high priority path group and the medium priority path group. Integrate all path allocation results to generate the final optimized path allocation table.

5. The video transmission method based on streaming processing as claimed in claim 4, characterized in that: The transmitting of the sliced ​​data according to the path optimization result refers to traversing each data slice, selecting the target path group according to the priority, initializing the path scheduler, transmitting the slices in the high priority data set to the target receiving end through the low-latency transmission protocol, transmitting the slices in the second priority data set to the target receiving end through the bandwidth optimization protocol, recording the transmission results of each path to generate a transmission log and storing it in the database.

6. The video transmission method based on streaming processing as claimed in claim 5, characterized in that: After receiving the fragmented data, reorganizing the fragmented data into a video stream and optimizing the image quality means receiving the fragmented data transmitted by each path through the transmission protocol and using a cyclic redundancy check to detect the integrity of each fragment, recording the fragment number that failed the check and requesting retransmission from the sending end, and if the retransmission fails, using adjacent fragments for interpolation compensation; Sorting the slice data according to the slice number and timestamp, merging the slice data of all paths into a complete time sequence according to the number to obtain a complete data frame sequence, initializing the video decoder at the receiving end, decoding the data frame sequence frame by frame, generating a video stream, and correcting the frame errors that occur during the decoding process using error concealment technology; Bilateral filtering is used to remove noise and retain edge details from the decoded image frame sequence, histogram equalization is used to enhance image contrast, and inverse wavelet transform is used to enhance details. The optimized image frame sequence is re-encoded into a complete video stream.

7. The video transmission method based on streaming processing as claimed in claim 6, characterized in that: The video data for storage and transmission refers to compressing the video stream using context-adaptive binary arithmetic coding, storing the compressed video stream in layers, and uploading the data to the cloud for backup after storage.

8. A video transmission system based on streaming processing, based on the video transmission method based on streaming processing according to any one of claims 1 to 7, characterized in that: include, Data collection module, used to collect video streams and perform preprocessing; The data slicing and path initialization module is used to encapsulate the preprocessed video frames into image streams and perform data slicing, allocate initialization paths for the slicing, and calculate the initial traffic ratio; Path optimization module, used to dynamically optimize path selection and adjust path traffic distribution based on network feedback; Data transmission module, used to efficiently transmit the shard data to the receiving end according to the path optimization result; The image quality optimization module is used to receive the fragmented data and reassemble it into a video stream while optimizing the image quality; The storage backup module is used to compress and store the transmitted video stream and upload it to the cloud for backup.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the video transmission method based on streaming processing are implemented in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the video transmission method based on streaming processing described in any one of claims 1 to 7 are implemented.

Images