Cross-layer optimization method, device and server for ensuring video stream switching delay

Abstract

The present application provides a cross-layer optimization method, device and server for ensuring the delay of video stream exchange, which relates to the field of image communication technology. The method includes: determining the input sequence length based on a dynamic adjustment strategy of a time window and generating a first time series sequence to obtain a second time series sequence according to a traffic prediction model; the first time series sequence includes a feature vector corresponding to each historical time point; the second time series sequence includes video traffic prediction result data corresponding to each target time point after the current moment; extracting a third time series from the second time series according to the predicted time period and performing traffic shaping to obtain cross-layer transmission rate optimization data for ensuring the delay of video stream exchange so as to transmit the video clips in the application layer to the video stream request end via a time-sensitive network. The present application can improve the reliability and stability of video stream transmission, and can ensure the certainty of video stream exchange delay, so as to improve video stream transmission efficiency and video playback quality.

Description

Technical Field

[0001] The present application relates to the field of image communication technology, and in particular to a cross-layer optimization method, device, and server for ensuring delay in video stream exchange. Background Art

[0002] Video streaming not only consumes a large amount of bandwidth resources but also requires extremely low exchange latency (milliseconds or even lower). Therefore, Time-Sensitive Networking (TSN) technology can be used to ensure efficient video streaming. For example, video data from the application layer of a streaming server is exchanged to the application on the client device via a time-sensitive network.

[0003] However, existing video application traffic is highly bursty, which does not match the resource response speed of the media access control layer (MAC Layer). This results in poor video stream transmission quality, manifested in extended switching time, low video bit rate (low bandwidth utilization) and frequent resolution switching.

[0004] Therefore, there is an urgent need to design a method that can improve the reliability and stability of video stream transmission and ensure the determinism of video stream exchange delay to improve the efficiency of video stream transmission. Summary of the Invention

[0005] In view of this, embodiments of the present application provide a cross-layer optimization method, device, and server for ensuring delay in video stream switching, so as to eliminate or improve one or more defects in the prior art.

[0006] One aspect of the present application provides a cross-layer optimization method for ensuring video stream exchange latency, which is executed in the kernel layer of a server. The method includes:

[0007] Determine the current input sequence length based on the time window dynamic adjustment strategy; generate a first time series sequence according to the input sequence length, and input the first time series sequence into the traffic prediction model so that the traffic prediction model outputs a second time series sequence; wherein, the first time series sequence includes a feature vector corresponding to each historical time point; the feature vector is used to represent the context information corresponding to the video segment transmission process of transmitting the video segment in the application layer of the server to the video stream request end via the time-sensitive network at a corresponding historical time point; the second time series includes: video traffic prediction result data corresponding to each target time point after the current moment;

[0008] According to the predicted time period, video traffic prediction result data corresponding to multiple target time points are extracted from the second timing sequence to form a third timing sequence, and traffic shaping is performed on the third timing sequence to obtain cross-layer transmission rate optimization data for ensuring the video stream exchange delay, so as to obtain cross-layer transmission rate optimization data, at each target time point in the third timing sequence, the video clips in the application layer are respectively transmitted to the video stream request end via the time-sensitive network.

[0009] In some embodiments of the present application, determining the current input sequence length based on the time window dynamic adjustment strategy includes:

[0010] Obtaining a first bit rate of the video segment transmission process corresponding to a historical time point closest to the current moment, and a second bit rate of the video segment transmission process corresponding to another historical time point adjacent to the historical time point;

[0011] Dynamically adjust the size of the target time window according to the bit rate change amplitude between the first bit rate and the second bit rate;

[0012] The current input sequence length is determined according to the size of the target time window.

[0013] In some embodiments of the present application, dynamically adjusting the size of the target time window according to the bit rate variation amplitude between the first bit rate and the second bit rate includes:

[0014] The size of the target time window is calculated according to the code rate change amplitude between the first code rate and the second code rate, the pre-stored current time window size, the preset adjustment step coefficient, the sensitivity coefficient, and the upper and lower limits of the time window.

[0015] In some embodiments of the present application, performing traffic shaping on the third time sequence to obtain cross-layer transmission rate optimization data for ensuring video stream exchange delay includes:

[0016] Performing a fast Fourier transform on the third time series to convert the video traffic prediction result data corresponding to each target time point in the third time series from the time domain to the frequency domain, thereby obtaining frequency domain data corresponding to each target time point in the third time series;

[0017] Using a low-pass filter to filter each of the frequency domain data to obtain filtered frequency domain data corresponding to each of the target time points in the third time series;

[0018] Performing an inverse fast Fourier transform on each of the filtered frequency domain data to convert each of the filtered frequency domain data from the frequency domain back to the time domain, thereby obtaining traffic shaped data corresponding to each of the target time points in the third time series;

[0019] According to the traffic shaped data corresponding to each target time point in the third timing sequence, cross-layer transmission rate optimization data for ensuring the video stream exchange delay is obtained.

[0020] In some embodiments of the present application, the traffic prediction model includes: a time series prediction model based on the Transformer architecture.

[0021] In some embodiments of the present application, obtaining cross-layer transmission rate optimization data for ensuring video stream switching delay based on the traffic shaped data corresponding to each target time point in the third time sequence includes:

[0022] Obtaining an integral difference between the traffic shaped data corresponding to each target time point in the third time series and the video traffic prediction result data corresponding to each target time point in the third time series;

[0023] and, obtaining, according to the traffic shaped data corresponding to each target time point in the third time series, a rate ratio corresponding to each target time point in the third time series;

[0024] Based on the traffic shaped data corresponding to each target time point in the third timing sequence, the integrated difference, the rate ratio corresponding to each target time point in the third timing sequence, and the preset sampling interval, the target bandwidth rate corresponding to each target time point in the third timing sequence is determined, so that the target bandwidth rate corresponding to each target time point in the third timing sequence constitutes cross-layer transmission rate optimization data for ensuring video stream exchange delay.

[0025] A second aspect of the present application provides a cross-layer optimization device for ensuring video stream exchange delay, which is provided in the kernel layer of a server, and includes:

[0026] A traffic predictor is configured to determine the current input sequence length based on a time window dynamic adjustment strategy; generate a first time series sequence according to the input sequence length, and input the first time series sequence into a traffic prediction model so that the traffic prediction model outputs a second time series sequence; wherein the first time series sequence includes a feature vector corresponding to each historical time point; the feature vector is used to represent context information corresponding to a video segment transmission process of transmitting a video segment in the application layer of the server to a video stream request end via a time-sensitive network at a corresponding historical time point; the second time series includes: video traffic prediction result data corresponding to each target time point after the current moment;

[0027] A timing-aware traffic shaper is used to extract video traffic prediction result data corresponding to multiple target time points from the second timing sequence according to a predicted time period to form a third timing sequence, and to perform traffic shaping on the third timing sequence to obtain cross-layer transmission rate optimization data for ensuring the video stream exchange delay, so as to transmit the video clips in the application layer to the video stream request end via the time-sensitive network at each target time point in the third timing sequence based on the cross-layer transmission rate optimization data.

[0028] A third aspect of the present application provides a server, wherein a cross-layer optimization device for ensuring delay in video stream exchange is provided in a kernel layer of the server; an application layer of the server stores various video clips;

[0029] The cross-layer optimization device for ensuring time delay in video stream switching is used to execute the cross-layer optimization method for ensuring time delay in video stream switching;

[0030] The cross-layer optimization device for video stream switching delay guarantee is communicatively connected to the time-sensitive switch, and the time-sensitive switch is communicatively connected to the video stream request end, so that the cross-layer optimization device for video stream switching delay guarantee transmits the video clips in the application layer to the video stream request end via the time-sensitive network.

[0031] In some embodiments of the present application, the video stream requester includes: an application in a vehicle-mounted client device.

[0032] The fourth aspect of the present application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and runnable on the processor, wherein the processor implements the cross-layer optimization method for ensuring the delay of video stream exchange when executing the computer program.

[0033] A fifth aspect of the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the cross-layer optimization method for ensuring the delay of video stream exchange.

[0034] The sixth aspect of the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the cross-layer optimization method for ensuring the delay of video stream switching.

[0035] The cross-layer optimization method for video stream exchange delay guarantee provided by the present application is executed in the kernel layer of the server, and determines the current input sequence length based on the time window dynamic adjustment strategy; generates a first time series sequence according to the input sequence length, and inputs the first time series sequence into the traffic prediction model so that the traffic prediction model outputs a second time series sequence; wherein, the first time series sequence includes a feature vector corresponding to each historical time point; the feature vector is used to represent the context information corresponding to the video segment transmission process of transmitting the video segment in the application layer of the server to the video stream request end via the time-sensitive network at a corresponding historical time point; the second time series includes: the video traffic prediction result data corresponding to each target time point after the current moment; according to the prediction time period, extracting video traffic prediction result data corresponding to multiple target time points from the second timing sequence to form a third timing sequence, and performing traffic shaping on the third timing sequence to obtain cross-layer transmission rate optimization data for ensuring the video stream exchange delay, so as to be based on the cross-layer transmission rate optimization data, at each of the target time points in the third timing sequence, transmit the video clips in the application layer to the video stream request end via the time-sensitive network, which can resolve the conflict between traffic burstiness and resource response hysteresis, improve the reliability and stability of video stream transmission, ensure the certainty of video stream exchange delay, and thus improve video stream transmission efficiency, improve video playback quality and improve network resource utilization, so as to ensure that users have a smoother viewing experience.

[0036] Additional advantages, purposes, and features of the present application will be described in part in the following description and will become apparent to those skilled in the art upon study of the following or may be learned from practice of the present application. The purposes and other advantages of the present application may be achieved and obtained by the structures specifically pointed out in the specification and drawings.

[0037] Those skilled in the art will understand that the purposes and advantages that can be achieved by the present application are not limited to the above specific description, and the above and other purposes that can be achieved by the present application will be more clearly understood based on the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] The drawings described herein are intended to provide a further understanding of the present application, constitute a part of the present application, and do not constitute a limitation of the present application. The components in the drawings are not drawn to scale, but are only for the purpose of illustrating the principles of the present application. In order to facilitate the illustration and description of some parts of the present application, the corresponding parts in the drawings may be enlarged, that is, they may become larger than other components in the exemplary device actually manufactured according to the present application. In the drawings:

[0039] Figure 1 This is a first flow chart of a cross-layer optimization method for ensuring video stream exchange delay in one embodiment of the present application.

[0040] Figure 2 This is a second flow chart of the cross-layer optimization method for ensuring video stream exchange delay in one embodiment of the present application.

[0041] Figure 3 This is a third flow chart of a cross-layer optimization method for ensuring delay in video stream exchange in one embodiment of the present application.

[0042] Figure 4 This is a structural diagram of a cross-layer optimization device for ensuring video stream exchange delay in one embodiment of the present application.

[0043] Figure 5 This is a schematic diagram of the architecture between the server, time-sensitive switch and video stream requester in one embodiment of the present application.

[0044] Figure 6 This is a schematic diagram of the architecture between the server, time-sensitive switch, and vehicle-mounted client device in a video in an application example of this application.

[0045] Figure 7 This is an algorithm flow chart of a traffic predictor based on PatchTST in an application example of this application. DETAILED DESCRIPTION

[0046] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail in conjunction with the embodiments and drawings. Here, the illustrative embodiments of this application and their descriptions are used to explain this application, but are not intended to limit this application.

[0047] It should also be noted here that in order to avoid obscuring the present application due to unnecessary details, the accompanying drawings only show structures and / or processing steps that are closely related to the scheme according to the present application, while other details that are not closely related to the present application are omitted.

[0048] It should be emphasized that the term "include / comprises" when used herein refers to the existence of features, elements, steps or components, but does not exclude the existence or addition of one or more other features, elements, steps or components.

[0049] It should also be noted that, unless otherwise specified, the term "connection" herein may refer not only to a direct connection but also to an indirect connection involving an intermediate.

[0050] Hereinafter, embodiments of the present application will be described with reference to the accompanying drawings. In the accompanying drawings, the same reference numerals represent the same or similar components, or the same or similar steps.

[0051] Time-Sensitive Networking (TSN) is a MAC layer technology defined by the IEEE 802.1 standard. It meets the transmission requirements of different data flows through mechanisms such as high-precision clock synchronization (802.1AS), resource reservation (802.1Qat), and traffic shaping (802.1Qav / Qbv). TSN effectively coordinates the transmission of various data streams, ensuring that mission-critical data (such as autonomous driving data) and entertainment data can be efficiently and reliably transmitted on the same network, thereby improving overall system performance and user experience.

[0052] While time-sensitive networking (TSN) technology can theoretically guarantee performance for applications like autonomous driving and entertainment video, in actual deployments, existing video streaming methods and TSN solutions often result in poor video playback quality. This is primarily due to the conflict between bursty traffic patterns at the application layer and sluggish resource allocation at the MAC layer. The specific issues are as follows:

[0053] (1) The bursty transmission characteristics of video streams. Video streams have strong bursty transmission characteristics. The application layer divides the video content into multiple video chunks and transmits all video chunks as instantaneously as possible. This bursty traffic pattern generates a large number of data packets in a short period of time without considering the bandwidth limitations and transmission constraints of the underlying network. The result is the instantaneous consumption of a large amount of bandwidth resources, increased transmission delays, and an increased risk of buffer overflows.

[0054] (2) Slow resource scheduling response. In time-sensitive networks, the MAC layer manages traffic through time slot scheduling and resource reservation mechanisms (such as credit-based scheduling, CBS). However, the MAC layer responds slowly to burst traffic. Because burst traffic typically arrives simultaneously within a short period of time, the MAC layer may not be able to allocate sufficient resources to these packets in a timely manner, leading to the following problems: packets are blocked while waiting for resources; transmission continues only when resources are available, and this delay may cause traffic transmission to lag; further, it may cause video playback to freeze, affecting the user experience.

[0055] Limitations of Traditional Approaches: To address these issues, traditional methods typically reserve redundant resources for bursty application flows. For example, the 802.1Qat protocol allocates resources by calculating worst-case demand. However, this approach leads to inefficient resource utilization in low-traffic scenarios, resulting in wasted resources.

[0056] Based on this, in order to design a method that can improve the reliability and smoothness of video stream transmission, and can ensure the certainty of video stream exchange delay to improve the efficiency of video stream transmission, the embodiments of the present application respectively provide a cross-layer optimization method for video stream exchange delay guarantee, a cross-layer optimization device, system, physical device, computer-readable storage medium and computer program product for executing the cross-layer optimization method for video stream exchange delay guarantee, by adjusting the transmission timing of burst data packets so that they are scheduled when the MAC layer bandwidth resources are sufficient, thereby resolving the conflict between burst traffic and resource response.

[0057] The details are described in detail through the following examples.

[0058] Based on this, the embodiment of the present application provides a cross-layer optimization method for video stream exchange delay guarantee that can be implemented by a cross-layer optimization device for video stream exchange delay guarantee, which is executed in the kernel layer of the server, see Figure 1 The cross-layer optimization method for ensuring the delay of video stream switching specifically includes the following contents:

[0059] Step 100: Determine the current input sequence length based on the time window dynamic adjustment strategy; generate a first time series sequence according to the input sequence length, and input the first time series sequence into the traffic prediction model so that the traffic prediction model outputs a second time series sequence; wherein, the first time series sequence includes feature vectors corresponding to each historical time point; the feature vector is used to represent the context information corresponding to the video segment transmission process of transmitting the video segment in the application layer of the server to the video stream request end via the time-sensitive network at a corresponding historical time point; the second time series includes: video traffic prediction result data corresponding to each target time point after the current moment.

[0060] It should be noted that there is a strong correlation between video streams and transmission parameters (such as the send window and buffer size), so their burst patterns can be learned in a data-driven manner. Therefore, in step 100 of the present application, a traffic prediction model is used that can output a time series consisting of video traffic prediction result data for each future target time point based on a time series constructed based on contextual information at each historical time point. This model can learn the dynamic pattern of traffic forwarding from large-scale data, which helps improve the accuracy of traffic change prediction.

[0061] However, for scenarios with high latency requirements, such as in-vehicle video traffic prediction, the traffic characteristics of this scenario have large dynamic fluctuations. Therefore, if a fixed-length historical window is used as input processing, this static window design may lead to a decrease in prediction accuracy when facing drastically changing video traffic.

[0062] Based on this, step 100 of the present application first determines the current input sequence length based on the time window dynamic adjustment strategy; it can be understood that the time window dynamic adjustment strategy is a method that can achieve dynamic adjustment of the time window, and specifically can adjust the input window size according to the change of video bit rate, so as to enable it to expand the window when the traffic changes drastically to obtain richer historical information, and to reduce the window when the traffic is stable to improve computing efficiency.

[0063] At the same time, the time window dynamic adjustment strategy can also ensure that the traffic prediction model fully captures key historical information, and thus can cooperate with the traffic shaping process of step 200 below to improve the accuracy of video stream transmission control.

[0064] The input sequence length refers to the length of the first sequence for traffic prediction. The first time series includes the feature vectors corresponding to each historical time point; the historical time point refers to the time point before the current moment, and the target time point refers to the time point after the current moment. In the embodiment of the present application, the time intervals between each historical time point and each target time point are the same. The specific values can be set according to actual application requirements and are not limited in this application.

[0065] It can be understood that the feature vector corresponding to a historical time point is used to represent the context information corresponding to the video segment transmission process of transmitting the video segment in the application layer of the server to the video stream request end via the time-sensitive network at the historical time point. It can be defined as:

[0066]

[0067] in, Indicates the historical time point corresponding to the feature vector, that is, a step length; Indicates a point in history , a video quality corresponding to a video segment transmission process of transmitting the video segment in the application layer of the server to the video stream requesting end via a time-sensitive network; Indicates a point in history , transmitting the video clip in the application layer of the server to the video stream request end via the time-sensitive network to obtain the corresponding video bit rate of the video clip transmission process; Indicates a point in history , the buffer length corresponding to the video segment transmission process of transmitting the video segment in the application layer of the server to the video stream request end via the time-sensitive network; wherein, Used to reflect on historical time points , transmitting the video clip in the application layer of the server to the video stream request end via a time-sensitive network to obtain context information corresponding to the video clip transmission process; .

[0068] The first time series sequence is the historical input sequence. If the current time window length contains N time periods, the first time series sequence X1 can be expressed as:

[0069]

[0070] in, 、 to ( That is ) are the feature vectors corresponding to each historical time point.

[0071] The second time series sequence is the predicted output sequence. The number of target time points in the second time series sequence is the same as the number of historical time points in the first time series sequence. The second time series sequence X2 can be expressed as:

[0072]

[0073] in, 、 to These are the video traffic prediction result data corresponding to each target time point.

[0074] Step 200: According to the predicted time period, extract the video traffic prediction result data corresponding to multiple target time points from the second timing sequence to form a third timing sequence, and perform traffic shaping on the third timing sequence to obtain cross-layer transmission rate optimization data for ensuring the video stream exchange delay, so as to transmit the video clips in the application layer to the video stream request end via the time-sensitive network at each target time point in the third timing sequence based on the cross-layer transmission rate optimization data.

[0075] The prediction time period can be denoted as T, which is a constant. It is understandable that the number of target time points required by the prediction time period must be less than or equal to the number of target time points in the second time series.

[0076] In step 200, the third time sequence can be recorded as , including video traffic prediction result data corresponding to multiple target time points extracted from the second time series, the third time series After the traffic shaping process is performed on the third time series, the traffic shaped data corresponding to each target time point in the third time series are obtained. The sequence composed of the traffic shaped data corresponding to each target time point can be recorded as ; Then, based on the The cross-layer transmission rate optimization data used to ensure the video stream exchange delay is calculated.

[0077] The cross-layer transmission rate optimization data for ensuring the video stream exchange delay may include the rate ratio corresponding to each target time point in the third timing sequence. . Then, at each of the target time points in the third timing sequence, according to the rate ratio corresponding to each of the target time points, the video clip in the application layer of the server can be transmitted to the video stream request end via the time-sensitive network in the kernel layer of the server.

[0078] From the above description, it can be seen that the cross-layer optimization method for video stream exchange delay guarantee provided in the embodiment of the present application can resolve the conflict between traffic burstiness and resource response hysteresis, improve the reliability and stability of video stream transmission, ensure the certainty of video stream exchange delay, and thus improve video stream transmission efficiency, improve video playback quality and improve network resource utilization to ensure that users have a smoother viewing experience.

[0079] In order to further improve the application effectiveness and reliability of the time window dynamic adjustment strategy, in the cross-layer optimization method for video stream exchange delay guarantee provided in the embodiment of the present application, see Figure 2 Step 100 of the cross-layer optimization method for ensuring video stream exchange delay specifically includes the following:

[0080] Step 110: Obtain a first bit rate of the video segment transmission process corresponding to a historical time point closest to the current moment, and a second bit rate of the video segment transmission process corresponding to another historical time point adjacent to the historical time point.

[0081] Step 120: Dynamically adjust the size of the target time window according to the bit rate variation amplitude between the first bit rate and the second bit rate.

[0082] Step 130: Determine the current input sequence length according to the size of the target time window.

[0083] Step 140: Generate a first time series sequence based on the length of the input sequence, and input the first time series sequence into a traffic prediction model so that the traffic prediction model outputs a second time series sequence; wherein, the first time series sequence includes a feature vector corresponding to each historical time point; the feature vector is used to represent the context information corresponding to the video segment transmission process of transmitting the video segment in the application layer of the server to the video stream request end via the time-sensitive network at a corresponding historical time point; the second time series includes: video traffic prediction result data corresponding to each target time point after the current moment.

[0084] In order to further improve the effectiveness and reliability of dynamically adjusting the size of the target time window according to the bit rate variation between the first bit rate and the second bit rate, in a cross-layer optimization method for video stream exchange delay guarantee provided in an embodiment of the present application, see Figure 3 Step 120 in the cross-layer optimization method for ensuring video stream exchange delay specifically includes the following:

[0085] Step 121: Calculate the size of the target time window according to the bit rate variation between the first bit rate and the second bit rate, the pre-stored current time window size, the preset adjustment step coefficient, the sensitivity coefficient, and the upper and lower limits of the time window.

[0086] It is understood that the current time window refers to the time window used to obtain the previous input sequence length before the current input sequence length. Since this time window is still the latest time window when step 120 is executed, it is referred to as the current time window. The target time window refers to the time window used to obtain the current input sequence length. After the target time window size is calculated in step 120, this target time window becomes the latest time window.

[0087] The size of the target time window The calculation formula is as follows:

[0088]

[0089] in, Indicates the size of the current time window, Indicates the historical time point closest to the current moment The first bit rate of the corresponding video clip transmission process and another historical time point adjacent to the historical time point a bit rate variation range between the second bit rates in the corresponding video segment transmission process; As the adjustment step coefficient, it is used to control the amplitude of the time window change; As a sensitivity coefficient, it is used to determine the response degree of the time window to the change of the bit rate; the tanh function is used to control: when the bit rate changes greatly, the time window size is increased; when the bit rate remains basically unchanged, the time window size also remains unchanged, and the window size is adjusted smoothly. Finally, and They are used to control the lower and upper limits of the time window respectively, limiting the range of the time window to prevent the time window from being too large, which will increase the computational cost. At the same time, they can also prevent the time window from being too small, which will result in insufficient information obtained by the traffic prediction model.

[0090] In order to further improve the application effectiveness and reliability of the time window dynamic adjustment strategy, in the cross-layer optimization method for video stream exchange delay guarantee provided in the embodiment of the present application, see Figure 2 Step 200 of the cross-layer optimization method for ensuring video stream exchange delay specifically includes the following:

[0091] Step 210: Extracting video traffic prediction result data corresponding to a plurality of target time points from the second time series according to the prediction time period to form a third time series.

[0092] It is understandable that the extraction of multiple target time points from the second time series can be performed in a time series from front to back order; for example, if the prediction time period T corresponds to two target time points, and the second time series contains 5 target time points The corresponding video traffic prediction result data ; then the third timing sequence The target time point is included and The corresponding video traffic prediction result data .

[0093] Then, in order to smooth the burst traffic at the application layer, the embodiment of the present application innovatively applies the Fast Fourier Transform (FFT) method and related filtering operations in the subsequent steps 220 and 230 to shape the video stream at the application layer. The contribution of this method is reflected in two aspects: (1) Compared with heuristic shaping methods (such as Pacing), it effectively retains the original change trend of the traffic and adapts to the future dynamic changes of the video traffic; (2) Compared with traditional shaping algorithms based on machine learning, it can quickly provide shaping results and respond to the scheduling needs of the system in a timely manner. That is: the following steps 220 and 230 first apply FFT to convert the future traffic predicted by the traffic prediction model from the time domain to the frequency domain. The key to smoothing is the use of a low-pass filter - it can remove high-frequency components while retaining low-frequency components, which can capture the overall pattern of the traffic. Finally, the inverse Fast Fourier Transform (iFFT) is applied to convert the filtered low-frequency information back into usable predicted traffic time domain data.

[0094] Step 220: Perform a fast Fourier transform on the third time series to convert the video traffic prediction result data corresponding to each target time point in the third time series from the time domain to the frequency domain, and obtain the frequency domain data corresponding to each target time point in the third time series.

[0095] Step 230: Using a low-pass filter, filter each of the frequency domain data to obtain filtered frequency domain data corresponding to each of the target time points in the third time series.

[0096] Step 240: Perform an inverse fast Fourier transform on each of the filtered frequency domain data to convert each of the filtered frequency domain data from the frequency domain back to the time domain to obtain the traffic shaped data corresponding to each of the target time points in the third time series.

[0097] Step 250: Obtain cross-layer transmission rate optimization data for ensuring video stream exchange delay based on the traffic shaped data corresponding to each target time point in the third timing sequence.

[0098] Step 260: Based on the cross-layer transmission rate optimization data, at each target time point in the third timing sequence, the video clip in the application layer is transmitted to the video stream request end via the time-sensitive network.

[0099] However, in the above-mentioned filtering process, the fast Fourier transform process requires sampling, and the traffic has the characteristics of bursts and dynamic changes, making it difficult to accurately capture its dynamic change trends; therefore, in order to provide a more accurate third timing sequence to the fast Fourier transform process, in a cross-layer optimization method for video stream exchange delay guarantee provided in an embodiment of the present application, the traffic prediction model includes: a timing prediction model based on the Transformer architecture, namely, the PatchTST model.

[0100] The PatchTST model is used to capture contextual associations in the time dimension. Although the PatchTST model has demonstrated advantages in the above-mentioned traffic prediction process, when it is directly applied to the in-vehicle video traffic prediction scenario, since the traffic characteristics of this scenario have large dynamic fluctuations, and the PatchTST model uses a fixed-length historical window as input processing, this static window design may lead to a decrease in prediction accuracy when faced with drastic changes in video traffic. In addition, when the PatchTST model is combined with a timing-aware shaper, the fixed window length may not be able to fully capture key historical information, affecting the accuracy of the control strategy. Therefore, in step 100 of the present application, in order to solve this problem, the present application improves the input window mechanism of the PatchTST model so that it can adjust the input window size according to changes in the video bit rate. The purpose is to enable it to expand the window to obtain richer historical information when the traffic changes drastically, and to reduce the window to improve computational efficiency when the traffic is stable.

[0101] The Transformer is essentially an encoder-decoder architecture, consisting of an encoder component and a decoder component. The encoder component consists of multiple layers of encoders, each containing two sublayers: a self-attention layer and a feedforward neural network layer. The decoder component also consists of multiple layers of decoders, each containing two sublayers: a self-attention layer and a feedforward neural network layer.

[0102] Initially, the first time series sequence X1 is split into multiple chunks and mapped into a high-dimensional space using an embedding mechanism. These embeddings serve as the input to the Transformer. In the Transformer, the self-attention mechanism plays a key role by capturing the global dependencies between different time segments. This is implemented using a multi-head attention mechanism, which can be expressed as:

[0103] (1)

[0104] in, is a trainable parameter, Represents the concatenation of multiple attention heads; represents the multi-head attention mechanism for the input x; Represents different attention heads.

[0105] Each individual attention head The calculation is:

[0106] (2)

[0107] here, The function represents the self-attention mechanism, which allows the model to capture the dynamic correlation between time series and further process the temporal features by combining multiple layers of feed-forward networks. Indicates the video quality corresponding to the video segment transmission process of transmitting the video segment in the application layer of the server to the video stream requesting end via the time-sensitive network; Indicates the video bit rate corresponding to the video segment transmission process of transmitting the video segment in the application layer of the server to the video stream request end via the time-sensitive network; The length of the buffer corresponding to the video segment transmission process of transmitting the video segment in the application layer of the server to the video stream request end via the time-sensitive network; represents the trainable weight matrix used to process video quality features in the i-th attention head; represents the trainable weight matrix used to process video bitrate features in the i-th attention head; Represents the trainable weight matrix used to process the buffer length feature in the i-th attention head.

[0108] Furthermore, in the filtering process of step 200, controlling data packet transmission using the obtained rate may result in data packet loss. This is because the low-pass filter removes high-frequency components, which substantially reduces the resources available for data transmission, thereby reducing the total amount of transmitted data.

[0109] Based on this, in order to solve the above problems, in order to ensure the normal transmission of video streams and maintain the overall flow rate, the present invention introduces adjustment measures based on Fourier transform to maintain the integrity of the flow. In a cross-layer optimization method for video stream exchange delay guarantee provided in an embodiment of the present application, see Figure 3 Step 250 of the cross-layer optimization method for ensuring video stream exchange delay specifically includes the following:

[0110] Step 251: Obtain the integral difference between the traffic shaped data corresponding to each target time point in the third time series and the video traffic prediction result data corresponding to each target time point in the third time series.

[0111] Among them, the difference in points The calculation formula is as follows:

[0112]

[0113] in, is the third time series; A sequence consisting of traffic shaped data corresponding to each target time point; It is a symbolic expression in differential and integral operations, used to describe the infinitesimal change or integral dimension of the variable t.

[0114] And, step 252: according to the traffic shaped data corresponding to each target time point in the third time series, obtain the rate ratio corresponding to each target time point in the third time series.

[0115] Rate Ratio The calculation formula is as follows:

[0116]

[0117] Step 253: Determine the target bandwidth rate corresponding to each target time point in the third timing sequence based on the traffic shaped data corresponding to each target time point in the third timing sequence, the integrated difference, the rate ratio corresponding to each target time point in the third timing sequence, and the preset sampling interval, so that the target bandwidth rate corresponding to each target time point in the third timing sequence constitutes cross-layer transmission rate optimization data for ensuring video stream exchange delay.

[0118] The target bandwidth rate corresponding to each target time point in the third time series constitutes the compensated time domain rate distribution. The calculation formula is as follows:

[0119]

[0120] in, Indicates the sampling interval.

[0121] From the software level, the present application also provides a cross-layer optimization device for executing all or part of the cross-layer optimization method for video stream exchange delay guarantee, which is set in the kernel layer of the server, see Figure 4 The cross-layer optimization device for ensuring video stream exchange delay specifically includes the following contents:

[0122] The traffic predictor 10 is used to determine the current input sequence length based on the time window dynamic adjustment strategy; generate a first time series sequence according to the input sequence length, and input the first time series sequence into the traffic prediction model so that the traffic prediction model outputs a second time series sequence; wherein, the first time series sequence includes feature vectors corresponding to each historical time point; the feature vector is used to represent the context information corresponding to the video segment transmission process of transmitting the video segment in the application layer of the server to the video stream request end via the time-sensitive network at a corresponding historical time point; the second time series includes: video traffic prediction result data corresponding to each target time point after the current moment.

[0123] The timing-aware traffic shaper 20 is used to extract the video traffic prediction result data corresponding to multiple target time points from the second timing sequence according to the prediction time period to form a third timing sequence, and perform traffic shaping on the third timing sequence to obtain cross-layer transmission rate optimization data for ensuring the video stream exchange delay, so as to transmit the video clips in the application layer to the video stream request end via the time-sensitive network at each target time point in the third timing sequence based on the cross-layer transmission rate optimization data.

[0124] The embodiment of the cross-layer optimization device for video stream exchange delay guarantee provided in this application can be specifically used to execute the processing flow of the embodiment of the cross-layer optimization method for video stream exchange delay guarantee in the above-mentioned embodiment. Its functions will not be described in detail here, and reference can be made to the detailed description of the embodiment of the cross-layer optimization method for video stream exchange delay guarantee in the above-mentioned embodiment.

[0125] The cross-layer optimization device for ensuring video stream exchange delay can perform cross-layer optimization for ensuring video stream exchange delay in a server kernel layer. The server can be a streaming media server, and the server is used to communicate with a client device via a time-sensitive network.

[0126] The aforementioned client device may include a communication module (i.e., a communication unit) capable of establishing a communication connection with a remote server to facilitate data transmission with the server. The server may include a server at the task scheduling center or, in other implementation scenarios, a server on an intermediate platform, such as a server on a third-party server platform that is communicatively linked to the task scheduling center server. The server may comprise a single computer device, a server cluster consisting of multiple servers, or a distributed server configuration.

[0127] The server and the client device may communicate using any suitable network protocol, including network protocols that have not yet been developed as of the filing date of this application. Examples of such network protocols include TCP / IP, UDP / IP, HTTP, and HTTPS. Furthermore, examples of such network protocols include RPC (Remote Procedure Call Protocol) and REST (Representational State Transfer) protocols, which are used on top of the aforementioned protocols.

[0128] From the above description, it can be seen that the cross-layer optimization device for video stream exchange delay guarantee provided in the embodiment of the present application can resolve the conflict between traffic burstiness and resource response hysteresis, improve the reliability and stability of video stream transmission, ensure the certainty of video stream exchange delay, and thus improve video stream transmission efficiency, improve video playback quality and improve network resource utilization to ensure that users have a smoother viewing experience.

[0129] The present application also provides a server. Figure 5 The server may be a streaming media server, wherein a cross-layer optimization device for ensuring the delay of video stream exchange is provided in the kernel layer of the server; and each video clip is stored in the application layer of the server;

[0130] The cross-layer optimization device for video stream exchange delay guarantee is used to execute the cross-layer optimization method for video stream exchange delay guarantee described in the embodiment of the present application;

[0131] The cross-layer optimization device for video stream switching delay guarantee is communicatively connected to the time-sensitive switch, and the time-sensitive switch is communicatively connected to the video stream request end, so that the cross-layer optimization device for video stream switching delay guarantee transmits the video clips in the application layer to the video stream request end via the time-sensitive network.

[0132] Wherein, the video stream request end includes: an application in a vehicle-mounted client device.

[0133] The server may include a processor, a memory, a receiver, and a transmitter, wherein the processor and the memory may be connected via a bus or other means, with bus connection being an example. The receiver may be connected to the processor and the memory via a wired or wireless means. The processor may be a central processing unit (CPU). The processor may also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or a combination of the above chips.

[0134] The memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs, non-transitory computer-executable programs, and modules, such as the program instructions / modules corresponding to the cross-layer optimization method for ensuring latency in video stream switching according to the embodiments of the present application. The processor executes the non-transitory software programs, instructions, and modules stored in the memory to perform various processor functions and data processing, thereby implementing the cross-layer optimization method for ensuring latency in video stream switching according to the aforementioned method embodiments.

[0135] The memory may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created by the processor, etc. In addition, the memory may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some embodiments, the memory may optionally include a memory remotely located relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0136] The one or more modules are stored in the memory, and when executed by the processor, perform the cross-layer optimization method for ensuring delay in video stream switching in the embodiment.

[0137] In some embodiments of the present application, the user equipment may include a processor, a memory and a transceiver unit, and the transceiver unit may include a receiver and a transmitter. The processor, memory, receiver and transmitter may be connected through a bus system. The memory is used to store computer instructions, and the processor is used to execute the computer instructions stored in the memory to control the transceiver unit to send and receive signals.

[0138] As an implementation method, the functions of the receiver and transmitter in this application can be considered to be implemented through a transceiver circuit or a dedicated transceiver chip, and the processor can be considered to be implemented through a dedicated processing chip, a processing circuit or a general-purpose chip.

[0139] As another implementation method, it is possible to use a general-purpose computer to implement the server provided in the embodiments of the present application. That is, the program code for implementing the functions of the processor, receiver, and transmitter is stored in a memory, and the general-purpose processor implements the functions of the processor, receiver, and transmitter by executing the code in the memory.

[0140] The present application also provides a computer-readable storage medium having a computer program stored thereon. When executed by a processor, the computer program implements the steps of the aforementioned cross-layer optimization method for ensuring latency in video stream switching. The computer-readable storage medium can be a tangible storage medium, such as a random access memory (RAM), internal memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, register, floppy disk, hard disk, removable storage disk, CD-ROM, or any other form of storage medium known in the art.

[0141] An embodiment of the present application also provides a computer program product, including a computer program, which, when executed by a processor, implements the steps of the aforementioned cross-layer optimization method for ensuring delay in video stream exchange.

[0142] To further illustrate the above embodiment, the present application also provides a specific application example of a cross-layer optimization method for video stream exchange delay guarantee, taking an in-vehicle application scenario as an example. First, the in-vehicle application scenario is described:

[0143] The prevalence of entertainment systems in smart cars has greatly enriched passengers' audiovisual experiences, but it also places higher demands on the transmission performance of in-vehicle networks. With the automotive industry's accelerated transformation toward intelligent technology, core technologies such as autonomous driving and advanced driver assistance systems (ADAS) are experiencing rapid development. In particular, Tesla's vision-based fully autonomous driving solution has garnered widespread attention in the industry. At the same time, passenger demand for immersive entertainment experiences is growing, prompting automakers to widely incorporate audiovisual equipment into in-vehicle systems. For example, modern smart cockpits often feature multiple video screens for front and rear passengers, along with multi-channel audio players, significantly enhancing the passenger entertainment experience.

[0144] However, the large number of concurrent video streams places enormous pressure on in-vehicle networks. These streams not only consume significant bandwidth but also impose extremely high transmission latency requirements (typically within 10 milliseconds). Furthermore, in-vehicle networks must address the co-transmission of audio and video streams with autonomous driving-related streams (such as video analytics and radar point cloud data), which require even more deterministic transmission. Therefore, ensuring the smooth transmission of multiple data streams within the same in-vehicle network while avoiding interference between them presents a technical challenge. To address this issue, time-sensitive networking (TSN) technology has been introduced into in-vehicle networks.

[0145] See also Figure 6 , the video stream request end can be an application in a client device, and the client device can be an in-vehicle client device. The in-vehicle client device first sends a video request to the server, prompting the server to select a target video segment. The server then returns a video segment of appropriate resolution based on the actual network link conditions. These video segments are transmitted through a time-sensitive network. In the above workflow, the method proposed in this application runs on the middleware (i.e., the kernel layer) of the streaming media server. Its goal is to coordinate the aggressive application layer video stream transmission and the sluggish MAC layer resource allocation in the in-vehicle time-sensitive network. It smoothes the transmission window before returning the video segment to the in-vehicle client device and adapts to the resource allocation of the underlying time-sensitive network. Figure 6 In the sequence, "GCL" represents a gating list, which is used to configure the gating state of each port in a time slot in a time-sensitive network (TSN). This controls the forwarding window of a specific priority queue and ensures deterministic communication. "T0:oCooCoo" represents the gating state of different priority queues in time slot T0. Each letter represents the gating state of a queue (C for Close, o for Open). Therefore, this sequence indicates that in time slot T0, the gates of priority queues 0, 2, 3, 5, and 6 are open, allowing data to be sent, while the gates of priority queues 1 and 4 are closed, preventing data from being sent. "T1:CCooCoo" indicates that in time slot T1, the gates of priority queues 2, 3, 5, and 6 are open, allowing data to be sent, while the gates of priority queues 0, 1, and 4 are closed, preventing data from being sent.

[0146] In the above process, the method proposed in the application example of this application needs to solve two problems.

[0147] (1) Accurately understand the application layer traffic pattern:

[0148] Video streams have a strong correlation with transmission parameters (such as the send window and buffer size), so their burst patterns can be learned through a data-driven approach. This application example designs a traffic predictor based on PatchTST. Based on the Transformer model, it learns dynamic patterns of traffic forwarding from large-scale data and utilizes a self-attention mechanism to effectively capture long-term and short-term dependencies in historical traffic forwarding, thereby improving the accuracy of traffic change prediction.

[0149] (2) Driving application layer traffic regulation to adapt to MAC layer resource scheduling:

[0150] Traditional approaches use pacing, but this sacrifices the transmission efficiency of delay-sensitive packets. Other learning-based traffic shaping algorithms, due to inference delays (typically in the order of seconds), cannot match the millisecond-level resource scheduling granularity of the MAC layer. In contrast, this application example combines Fast Fourier Transform (FFT) and related filtering operations to shape application-layer traffic. These methods are commonly used in digital signal processing and have not yet been applied to traffic shaping.

[0151] This application example provides a cross-layer optimization method for video stream exchange latency assurance in an in-vehicle application scenario. By analyzing historical traffic behavior at the application layer, identifying its transmission patterns, and adjusting the timing of burst data packets based on these pattern characteristics, it ensures scheduling when MAC layer bandwidth resources are sufficient. Specifically, it includes the following:

[0152] (1) Traffic predictor based on PatchTST model

[0153] Feasibility of Burst Traffic Prediction: Network-layer metrics such as throughput and latency variation indicate that packet transmission patterns are highly dynamic and difficult to accurately predict. However, from the perspective of application-layer sending behavior, traffic characteristics exhibit a certain degree of identifiability and can be used for traffic prediction. In DASH-based video streaming, application-layer parameters such as the sending window size, traffic trigger time, and buffer size are closely correlated, suggesting that data-driven approaches can be used to learn the burst patterns of traffic.

[0154] However, the traditional GRU model used for traffic prediction primarily models short-term time series, and its prediction performance degrades significantly over the longer time windows required by the time-aware traffic shaper module. PatchTST, a Transformer-based neural network model, has garnered widespread attention in recent years. It efficiently learns hidden dynamic patterns from large-scale datasets and constructs an accurate time series prediction framework through nonlinear mapping of multi-layer neural networks. Furthermore, it leverages a self-attention mechanism to effectively capture long-term and short-term dependencies, enabling it to better learn traffic variations during video transmission.

[0155] See also Figure 7 , the input of each step of the traffic predictor based on PatchTST is a feature vector ,The traffic predictor based on PatchTST uses a sliding window mechanism for data ,preprocessing to generate a first time series sequence, and captures the ,contextual association on the time dimension of the first time series sequence through the ,PatchTST model to obtain the second time series sequence.

[0156] Specifically, the layers of the PatchTST model are: instance normalization layer, Transformer encoder, and smoothing and linear head layer. Among them, the instance normalization layer first normalizes the input data to ensure that the mean of each feature is zero and the standard deviation is one, thereby eliminating the influence of different scales and improving the stability of model training. Specifically, the input is a feature vector containing time series data, such as the resolution of a video stream. , bit rate and buffer length , these features will have different values at each time step. The instance normalization layer calculates the mean and standard deviation of the features at each time step and normalizes the data to zero mean and unit variance. The output is the normalized feature vector. Next, the Transformer encoder processes these normalized data through the self-attention mechanism to capture the global and local dependencies between each time step in the time series. Specifically, the input of the Transformer is the normalized time series data (for example ), mapped to a high-dimensional space through an embedding layer, and then passed to the self-attention module. The self-attention mechanism calculates the relationship weights (query-key-value relationships) between each pair of elements in the input sequence and performs a weighted sum to obtain a new representation for each time step. The output is a feature matrix that contains both global and local information. The multi-head attention mechanism in the Transformer can compute multiple relationships in parallel, capturing information from different subspaces, while positional encoding ensures that the temporal information in the sequence is fully preserved. Finally, the smoothing and linearization head layers further process the Transformer encoder output. First, a low-pass filter is used to smooth the signal, filtering out high-frequency fluctuations and preserving the long-term trend of the data. This prevents short-term traffic bursts from interfering with bandwidth allocation. The smoothed data is then linearly transformed to output the final traffic prediction results, which are the bandwidth requirements for each time step. These predictions are then used by the time-aware traffic shaper to ensure smooth and low-latency transmission of the video stream, thereby improving video quality.

[0157] The core of the PatchTST model is based on a Transformer encoder, which consists of a multi-head attention layer, an instantiation layer, a feedback evaluation layer, and an instantiation layer connected in sequence. The multi-head attention layer takes as input the feature matrix of the previous layer (the instantiation layer or the output of the previous Transformer encoder). This data represents time series features, such as the bitrate, resolution, and buffer length of a video stream. Its shape is (N, D), where N is the sequence length and D is the feature dimension at each time step. The input data is first mapped into a query, key, and value matrix. The representation for each time step is then calculated by calculating the similarity between the query (Q) and the key (K) and using this similarity to perform a weighted sum of the values (V). The output shape is the same as the input, but the representation for each time step contains more information, derived from weighted information from other time steps in the entire input sequence. Multiple self-attention heads are computed in parallel to capture information from different subspaces. The outputs of each head are then concatenated to form a complete representation. Next, the instantiation layer normalizes the output of the multi-head attention mechanism layer as its input. The normalization formula is:

[0158]

[0159] in represents the mean of the feature, is the standard deviation of the feature. This process ensures that the features at each time step have zero mean and unit standard deviation across dimensions. This eliminates scale differences between features, improving data consistency and training stability. The feedback evaluation layer then uses residual connections to combine the normalized data from the instantiation layer and the input data from the previous layer. The input shape is (N, D). The output data of the residual connection is still in the shape of (N, D) and already takes into account the feedback information from the previous layer. This approach helps alleviate the vanishing gradient problem in deep networks and promotes smooth information transfer. Finally, the second instantiation layer, similar to the first, further normalizes the data to ensure a stable and consistent distribution of each feature, further enhancing stability and convergence during training. Through these four levels of processing, the Transformer encoder effectively captures the dynamic changes in video stream data, learns global and local dependencies, and generates high-quality traffic prediction results, thereby improving the predictive ability and stability of the entire PatchTST model.

[0160] Finally, the multi-head attention mechanism usually calculates the similarity between each query and key in parallel through multiple "heads", and then merges the results of these heads. This operation can help the model learn information from different subspaces, and Figure 6 Here nx represents the number of “heads” for such multiple parallel computations.

[0161] While the PatchTST model has demonstrated advantages in the aforementioned traffic prediction process, its direct application to in-vehicle video traffic prediction suffers from the highly dynamic nature of traffic. PatchTST uses a fixed-length historical window as input, and this static window design can lead to decreased prediction accuracy when faced with volatile video traffic. Furthermore, when PatchTST is combined with a time-aware shaper, the fixed window length may not fully capture critical historical information, impacting the accuracy of the control strategy.

[0162] In the application example of this application, in order to solve this problem, the application example of this application improves the input window mechanism of PatchTST so that it can adjust the input window size according to the change of video bit rate. The purpose is to enable it to expand the window when the traffic changes drastically to obtain richer historical information, and to reduce the window when the traffic is stable to improve computing efficiency.

[0163] Through this technical means, the application example of this application overcomes the prediction error problem caused by fixed window length, allowing PatchTST to adapt more flexibly to the dynamic changes of the video stream, thereby improving prediction accuracy.

[0164] (2) Time-Aware Traffic Shaper

[0165] In order to smooth the burst traffic at the application layer, this application example innovatively applies the FFT method and related filtering operations to shape the video stream at the application layer. The contribution of the time-aware traffic shaper is reflected in two aspects: (1) Compared with heuristic shaping methods (such as Pacing), it effectively preserves the original change trend of the traffic and adapts to the future dynamic changes of video traffic; (2) Compared with traditional machine learning-based shaping algorithms, it can quickly provide shaping results and respond to the system scheduling needs in a timely manner.

[0166] The time-series-aware traffic shaper first applies an FFT to convert the future traffic predicted by PatchTST from the time domain to the frequency domain. The key to smoothing this process is the use of a low-pass filter, which removes high-frequency components while retaining the low-frequency components that capture the overall traffic pattern. Finally, an inverse fast Fourier transform (iFFT) is applied to convert the filtered low-frequency information back into usable time-domain data for the predicted traffic.

[0167] However, the above filtering process has the following problems: (1) FFT requires sampling, and traffic has the characteristics of bursts and dynamic changes, making it difficult to accurately capture its dynamic change trends; (2) Using the obtained rate to control data packet transmission may result in data packet loss. This is because the low-pass filter removes high-frequency components, which essentially reduces the resources available for data transmission, thereby reducing the total amount of transmitted data. In the application example of this application, in order to solve the first problem, the traffic predictor based on PatchTST in Example 1 is used to dynamically capture the data stream transmission pattern, thereby providing traffic samples for the FFT method. To solve the second problem, that is, in order to ensure the normal transmission of the video stream and maintain the overall traffic size, the application example of this application introduces adjustment measures based on Fourier transform to maintain the integrity of the traffic. The specific operations are as follows.

[0168] To resolve this mismatch, the time domain waveforms before and after filtering are integrated, the difference between them is calculated, and the interpolated values are proportionally adjusted across the entire time domain. Then, the rate ratio is calculated for each target time point in the prediction time period T.

[0169] The above method can ensure that the data flow is better shaped, and at the same time ensure that the overall flow size can be maintained after smoothing, and no packet loss will occur.

[0170] In summary, to address the contradiction between bursty video traffic and slow MAC layer resource allocation in in-vehicle time-sensitive networks, this application example proposes a cross-layer optimization method for deterministic video stream exchange delay guarantee to ensure smooth video stream transmission and improve playback quality. Specifically:

[0171] 1) PatchTST-based traffic predictor: The PatchTST traffic prediction model learns the dynamic changes of video streams, accurately predicts future traffic demand, and provides accurate traffic data for the time-aware shaper.

[0172] 2) Timing-Aware Shaper: Innovatively combines FFT and low-pass filtering to smooth application layer video burst traffic and adjust the traffic transmission rate in real time based on MAC layer resource scheduling requirements to ensure smooth transmission.

[0173] Through the above mechanism, this method effectively solves the problem of mismatch between burst video traffic and resource response in vehicle-mounted time-sensitive networks, improves video playback quality, and ensures users have a smoother viewing experience.

[0174] It should be understood by those skilled in the art that the various exemplary components, systems, and methods described in conjunction with the embodiments disclosed herein can be implemented in hardware, software, or a combination of both. Whether it is implemented in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application. When implemented in hardware, it can be, for example, an electronic circuit, an application-specific integrated circuit (ASIC), appropriate firmware, a plug-in, a function card, etc. When implemented in software, the elements of this application are programs or code segments used to perform the required tasks. The program or code segment can be stored in a machine-readable medium or transmitted on a transmission medium or communication link via a data signal carried in a carrier.

[0175] It should be understood that the present application is not limited to the specific configurations and processes described above and illustrated in the figures. For the sake of brevity, a detailed description of known methods is omitted here. In the above embodiments, several specific steps are described and illustrated as examples. However, the method process of the present application is not limited to the specific steps described and illustrated. Those skilled in the art can make various changes, modifications, and additions, or change the order of the steps after understanding the spirit of the present application.

[0176] In this application, features described and / or illustrated for one embodiment may be used in the same or similar manner in one or more other embodiments, and / or combined with or replace features of other embodiments.

[0177] The above description is merely a preferred embodiment of the present application and is not intended to limit the present application. Those skilled in the art will appreciate that various modifications and variations of the present embodiment are possible. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present application shall be included within the scope of protection of the present application.

Claims

1. A cross-layer optimization method for ensuring video stream exchange delay, characterized in that: Executed in the kernel layer of the server, the method includes: Determine the current input sequence length based on the time window dynamic adjustment strategy; generate a first time series sequence according to the input sequence length, and input the first time series sequence into the traffic prediction model so that the traffic prediction model outputs a second time series sequence; wherein, the first time series sequence includes a feature vector corresponding to each historical time point; the feature vector is used to represent the context information corresponding to the video segment transmission process of transmitting the video segment in the application layer of the server to the video stream request end via the time-sensitive network at a corresponding historical time point; the second time series includes: video traffic prediction result data corresponding to each target time point after the current moment; According to the predicted time period, video traffic prediction result data corresponding to multiple target time points are extracted from the second timing sequence to form a third timing sequence, and traffic shaping is performed on the third timing sequence to obtain cross-layer transmission rate optimization data for ensuring the video stream exchange delay, so as to obtain cross-layer transmission rate optimization data, at each target time point in the third timing sequence, the video clips in the application layer are respectively transmitted to the video stream request end via the time-sensitive network.

2. The cross-layer optimization method for ensuring video stream exchange delay according to claim 1, characterized in that: The determining of the current input sequence length based on the time window dynamic adjustment strategy includes: Obtaining a first bit rate of the video segment transmission process corresponding to a historical time point closest to the current moment, and a second bit rate of the video segment transmission process corresponding to another historical time point adjacent to the historical time point; Dynamically adjust the size of the target time window according to the bit rate change amplitude between the first bit rate and the second bit rate; The current input sequence length is determined according to the size of the target time window.

3. The cross-layer optimization method for ensuring video stream exchange delay according to claim 2, characterized in that: The dynamically adjusting the size of the target time window according to the bit rate variation amplitude between the first bit rate and the second bit rate includes: The size of the target time window is calculated according to the code rate change amplitude between the first code rate and the second code rate, the pre-stored current time window size, the preset adjustment step coefficient, the sensitivity coefficient, and the upper and lower limits of the time window.

4. The cross-layer optimization method for video stream exchange delay guarantee according to claim 1, characterized in that: The performing traffic shaping on the third time sequence to obtain cross-layer transmission rate optimization data for ensuring video stream exchange delay includes: Performing a fast Fourier transform on the third time series to convert the video traffic prediction result data corresponding to each target time point in the third time series from the time domain to the frequency domain, thereby obtaining frequency domain data corresponding to each target time point in the third time series; Using a low-pass filter to filter each of the frequency domain data to obtain filtered frequency domain data corresponding to each of the target time points in the third time series; Performing an inverse fast Fourier transform on each of the filtered frequency domain data to convert each of the filtered frequency domain data from the frequency domain back to the time domain, thereby obtaining traffic shaped data corresponding to each of the target time points in the third time series; According to the traffic shaped data corresponding to each target time point in the third timing sequence, cross-layer transmission rate optimization data for ensuring the video stream exchange delay is obtained.

5. The cross-layer optimization method for video stream exchange delay guarantee according to claim 4 is characterized in that: The traffic prediction model includes: a time series prediction model based on the Transformer architecture.

6. The cross-layer optimization method for video stream exchange delay guarantee according to claim 4, characterized in that: The obtaining, according to the traffic shaped data corresponding to each target time point in the third time sequence, cross-layer transmission rate optimization data for ensuring the video stream switching delay includes: Obtaining an integral difference between the traffic shaped data corresponding to each target time point in the third time series and the video traffic prediction result data corresponding to each target time point in the third time series; and, obtaining, according to the traffic shaped data corresponding to each target time point in the third time series, a rate ratio corresponding to each target time point in the third time series; Based on the traffic shaped data corresponding to each target time point in the third timing sequence, the integrated difference, the rate ratio corresponding to each target time point in the third timing sequence, and the preset sampling interval, the target bandwidth rate corresponding to each target time point in the third timing sequence is determined, so that the target bandwidth rate corresponding to each target time point in the third timing sequence constitutes cross-layer transmission rate optimization data for ensuring video stream exchange delay.

7. A cross-layer optimization device for video stream switching delay guarantee, characterized in that: Set in the kernel layer of the server, the device includes: A traffic predictor is configured to determine the current input sequence length based on a time window dynamic adjustment strategy; generate a first time series sequence according to the input sequence length, and input the first time series sequence into a traffic prediction model so that the traffic prediction model outputs a second time series sequence; wherein the first time series sequence includes a feature vector corresponding to each historical time point; the feature vector is used to represent context information corresponding to a video segment transmission process of transmitting a video segment in the application layer of the server to a video stream request end via a time-sensitive network at a corresponding historical time point; the second time series includes: video traffic prediction result data corresponding to each target time point after the current moment; A timing-aware traffic shaper is used to extract video traffic prediction result data corresponding to multiple target time points from the second timing sequence according to a predicted time period to form a third timing sequence, and to perform traffic shaping on the third timing sequence to obtain cross-layer transmission rate optimization data for ensuring the video stream exchange delay, so as to transmit the video clips in the application layer to the video stream request end via the time-sensitive network at each target time point in the third timing sequence based on the cross-layer transmission rate optimization data.

8. A server, characterized in that: The core layer of the server is provided with a cross-layer optimization device for ensuring the delay of video stream exchange; the application layer of the server stores various video clips; The cross-layer optimization device for video stream exchange delay guarantee is used to perform the cross-layer optimization method for video stream exchange delay guarantee according to any one of claims 1 to 6; The cross-layer optimization device for video stream switching delay guarantee is communicatively connected to the time-sensitive switch, and the time-sensitive switch is communicatively connected to the video stream request end, so that the cross-layer optimization device for video stream switching delay guarantee transmits the video clips in the application layer to the video stream request end via the time-sensitive network.

9. The server according to claim 8, wherein: The video stream requesting end includes: an application program in a vehicle-mounted client device.

10. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the cross-layer optimization method for ensuring the delay of video stream switching is implemented as claimed in any one of claims 1 to 6.

Images