Vehicle networking video stream transmission continuity optimization method based on V2V
Patent Information
- Application Number
- CN202510316215.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-03-18
- Publication Date
- 2025-06-17
AI Technical Summary
In the Internet of Vehicles communication, the network topology changes rapidly when the vehicle moves, resulting in frequent frame loss in V2V transmission video stream information, affecting the driver's accurate judgment of road conditions.
The method based on Kalman filtering and RTSS smoothing algorithm is used to optimize the continuity of video streaming. The specific steps include: using the Kalman filtering algorithm to predict the state of the next moment in the prediction stage; updating the state estimates in the update stage with observation data; when there is a poor communication state, it is determined whether the frame needs to be filled through the adaptive threshold, and generating an image with a resolution close to the next frame through the frame segmentation and image interpolation algorithm.
It effectively improves the continuity and clarity of video streaming, reduces frame loss, and improves the driver's accuracy in judging road conditions.
Smart Images
Figure CN120166237A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of vehicle networking communication, and particularly relates to an optimization method for the continuity of vehicle networking video stream transmission based on V2V. Background Art
[0002] With the development of modern automotive industry and intelligent transportation systems, the new energy vehicle industry has become one of the important pillar industries promoting social and economic development. In recent years, due to the explosive growth of vehicle ownership, the resulting traffic accidents and traffic jams have become more and more frequent. Vehicle networking is a technology that emerged to solve traffic safety problems and avoid traffic accidents, and has become a key development direction of intelligent transportation systems. Vehicle networking quickly and reliably spreads road emergency messages to each target vehicle node through broadcasting, enabling vehicles to obtain out-of-line-of-sight warning information, giving drivers sufficient time to respond to traffic accidents, and avoiding potential collisions and congestion accidents, with great social and safety benefits.
[0003] However, in actual scenario applications, due to the very fast change of network topology during vehicle movement, the instability of the vehicle networking communication transmission link often causes frame loss in the V2V transmitted video stream information, resulting in the accuracy of drivers' judgment of road conditions. Summary of the Invention
[0004] To achieve the above object, the technical solution adopted by the present invention is an optimization method for the continuity of vehicle networking video stream transmission based on V2V, characterized in that the method comprises the following steps:
[0005] During the video stream transmission between two vehicles, the vehicle sending end packets the data packet information and sends it to the receiving end, and the receiving end will also re-statistics the sending end information and calculate the data as the input of the algorithm at the same time. The receiving end packet information includes: image data length, resolution column pixels, resolution row pixels, sending packet sequence number, timestamp; the information re-packeted by the receiving end includes: number of sent packets, data packet delay, number of lost packets, packet loss speed, packet loss acceleration.
[0006] The data re-packeted by the receiving end will be used as the input of the Kalman prediction algorithm. The implementation process of Kalman filtering is mainly divided into two stages: the prediction stage and the update stage. In the prediction stage, it is mainly to predict the state and error covariance at the next moment according to the dynamic model of the system. This stage does not use observation data and only depends on the system model and the state estimation at the previous moment. The main purpose of the update stage is to use the observation data at the current moment to update the state estimation and error covariance. This stage combines the prediction result and the observation data to obtain a more accurate state estimation;
[0007] To further improve the accuracy of state estimation, especially when dealing with video stream data with continuity and temporal correlation, an RTSS smoother can be connected after a standard Kalman filter prediction is completed. This method not only improves the accuracy of state prediction but also provides a more reliable basis for subsequent data processing and analysis;
[0008] When it is predicted that the communication state is poor, an adaptive threshold is designed to determine whether a frame needs to be filled in the next moment;
[0009] In the frame filling algorithm part, a buffer is first set up to save historical images, and the previous frame image is cropped by a frame segmentation factor to obtain video data closest to the next frame;
[0010] Since the clarity of the cropped image will be affected to a certain extent, in order to improve the real-time continuous clarity of the video, we will perform an image interpolation algorithm on the previous frame image processed by frame segmentation to generate an image with a resolution close to that of the next frame.
[0011] Furthermore, the data repackaged by the receiver will be used as the input of the Kalman prediction algorithm. The algorithm process includes:
[0012] The state equation and observation equation in the prediction stage from t to t + 1 are
[0013] The calculation formulas for the state error and observation error of the system are
[0014] Furthermore, the predicted value of the error covariance of the algorithm is
[0015] Next, the calculation formula for the gain matrix needs to be obtained as
[0016] After the prediction stage is completed, the update stage is entered. The state update equation in the update stage is
[0017] Finally, the error covariance of the system is updated, and the formula for the updated error covariance of the system is
[0018] The state vector is defined as
[0019] Among them, q(t) is the number of packet losses, q'(t) is the packet loss rate, and q”(t) is the packet loss acceleration. The first-order variation included in the state vector is the packet loss rate, and the second-order variation is the packet loss acceleration. Compared with the state vector that only considers the first-order variation, it can more accurately describe the predicted packet loss rate. The state transition matrix is
[0020] Furthermore, in the RTSS smoothing algorithm part of a V2V-based vehicle-to-everything (V2X) video stream transmission continuity optimization method, the algorithm process includes:
[0021] Calculate the smoothing gain:
[0022] Calculate the smoothing estimate:
[0023] Calculate the smoothing mean square error:
[0024] Among them, is the smoothing estimate of the state, is the smoothing gain matrix, P i (s) is the covariance matrix of the smoothing estimate. The smoothing estimate at any time i can be regarded as a combination of the filtering estimate value at time i and the smoothing estimate at time i + 1. Only after the forward filtering is completed, the RTSS algorithm starts the smoothing process to improve the filtering result.
[0025] Set an adaptive threshold, and perform overload detection and discrimination on the transmission channel according to the delay gradient value and the adaptive threshold. The adaptive threshold formula is: Adapt_value = Adapt_value + (kalman_value - Adapt_value) * K u
[0026] Among them, Adapt_value is the adaptive threshold, kalman_value is the output of the Kalman prediction algorithm, and K u is the threshold parameter. When kalman_value > Adapt_value, it is determined that a frame needs to be filled in the next moment.
[0027] According to the frame filling algorithm part of a V2V-based vehicle-to-everything (V2X) video stream transmission continuity optimization method according to claim 3, characterized in that a buffer is first set to save historical images, and the previous frame image is cropped by a frame segmentation factor to obtain video data closest to the next frame. The frame segmentation calculation formula is:
[0028] The clarity of the cropped image will be affected to a certain extent. To achieve real-time continuous clarity improvement of the video, we need to perform an image interpolation algorithm on the image processed by frame segmentation in the previous frame to generate an image with a resolution close to that of the next frame. When interpolation is required at point P, given A 11 =(x1,y1), A 12 =(x1,y2)=(x1,y1 + 1), A 21 =(x2,y1)=(x1 + 1,y1), A 22 =(x2,y2)=(x1 + 1,y1 + 1), after determining the 4 coordinate points around point E, first select an X-axis of the coordinate axis for interpolation through bilinear interpolation to obtain B1 between A 11 and A 21 f(B1)=f(x1 + n,y1)=n[f(x1 + 1,y1)-f(x1,y1)]+f(x1,y1)
[0029] Similarly, interpolate to obtain B2 between A 12 and A 22 f(B2)=f(x1 + n,y1 + 1)=n[f(x1 + 1,y1 + 1)-f(x1,y1 + 1)]+f(x1,y1 + 1)
[0030] Further, in the Y direction, interpolate through B1 and B2 to obtain the gray value result of point P. f(P)=f(x1 + n,y1 + m)=m[f(x1 + n,y1 + 1)-f(x1 + n,y1)]+f(x1 + n,y1)
[0031] Simplify the above formula, and we can obtain the final frame interpolation result f(x1 + n,y1 + m)=mn[f(x1 + 1,y1 + 1)-f(x1,y1 + 1)+f(x1 + 1,y1)-f(x1,y1)]+ f(x1,y1)+n[f(x1 + 1,y1)-f(x1,y1)]+m[f(x1,y1 + 1)-f(x1,y1)]
[0032] Furthermore, in the final frame interpolation part of the V2V-based vehicle-to-vehicle network video stream transmission continuity optimization method, after generating the next frame of image through the frame filling algorithm, it is also necessary to judge whether the next frame of image is received. If received, display the original image; if not received, display the generated image of this frame on the driver's observation screen. Brief Description of the Drawings
[0033] Figure 1 It is a flowchart of a prediction algorithm
[0034] Figure 2 It is a legend of a cutting algorithm
[0035] Figure 3 It is an example diagram of a super-resolution algorithm Detailed Implementation Manner
[0036] The present invention will be further described below:
[0037] During the video stream transmission of two vehicles, the vehicle sending end packets up the data packet information and sends it to the receiving end. The receiving end will also re-statistics the sending end information and calculate the data as the input of the algorithm at the same time. The receiving end packet information includes: image data length, resolution column pixels, resolution row pixels, sending packet sequence number, time stamp; the information re-packeted by the receiving end includes: number of sent packets, data packet delay, number of lost packets, packet loss speed, packet loss acceleration.
[0038] The data re-packeted by the receiving end will be used as the input of the Kalman prediction algorithm. The implementation process of Kalman filtering is mainly divided into two stages: the prediction stage and the update stage. In the prediction stage, the state and error covariance at the next moment are mainly predicted according to the dynamic model of the system. Observation data is not used in this stage, only relying on the system model and the state estimation of the previous moment. The main purpose of the update stage is to use the observation data at the current moment to update the state estimation and error covariance. This stage combines the prediction result and the observation data to obtain a more accurate state estimation;
[0039] In order to further improve the accuracy of state estimation, especially when facing video stream data with continuity and time correlation, an RTSS smoother can be connected after a standard Kalman filtering prediction is completed. This method not only improves the accuracy of state prediction, but also provides a more reliable basis for subsequent data processing and analysis;
[0040] In the case where a poor communication state is predicted, an adaptive threshold is designed to determine whether a frame needs to be filled in the next moment;
[0041] For the frame filling algorithm part, first set a buffer to save historical images, and crop the previous frame image through a frame segmentation factor to obtain video data closest to the next frame;
[0042] The cropped image will be affected to some extent in terms of image clarity. To achieve real-time continuous clarity improvement of the video, we need to perform an image interpolation algorithm on the image obtained by frame segmentation of the previous frame to generate an image with a resolution close to that of the next frame.
[0043] Further, the data repackaged at the receiving end will be used as the input of the Kalman prediction algorithm. The algorithm process includes:
[0044] Further, the RTSS smoothing algorithm part in an optimization method for the continuity of V2V-based vehicle-to-vehicle video stream transmission
[0045] Set an adaptive threshold, and perform overload detection and discrimination on the transmission channel according to the delay gradient value and the adaptive threshold. The formula for the adaptive threshold is: Adapt_value=Adapt_value+(kalman_value-Adapt_value)*K u
[0046] where Adapt_value is the adaptive threshold, kalman_value is the output of the Kalman prediction algorithm, and K u is the threshold parameter. When kalman_value>Adapt_value, it is determined that a supplementary frame is needed at the next moment.
[0047] According to the supplementary frame algorithm part of an optimization method for the continuity of V2V-based vehicle-to-vehicle video stream transmission as described in claim 3, it is characterized in that first, a buffer is set to save historical images, and the previous frame image is cropped through a frame segmentation factor to obtain video data closest to the next frame.
[0048] The cropped image will be affected to some extent in terms of image clarity. To achieve real-time continuous clarity improvement of the video, we need to perform an image interpolation algorithm on the image obtained by frame segmentation of the previous frame to generate an image with a resolution close to that of the next frame. After determining the 4 coordinate points around point P, first, bilinear interpolation is used. First, an interpolation is performed along one coordinate axis, the X-axis, to obtain the final supplementary frame interpolation result.
[0048] Further, in the final frame interpolation part of the optimization method for the continuity of V2V-based vehicle-to-vehicle video stream transmission, when the next frame image is generated through the supplementary frame algorithm, it is also necessary to determine whether the next frame image has been received. If it has been received, the original image is displayed; if it has not been received, the generated image of this frame is displayed on the driver's viewing screen.
Claims
1. A method for optimizing the continuity of video stream transmission in a V2V-based vehicle network, characterized in that: include: Step 1: During the video stream transmission between two vehicles, the vehicle transmitter assembles the data packet information and sends it to the receiver. The receiver will also re-count the transmitter information and calculate the data as the algorithm input. The receiver's packaged information includes: image data length, resolution column pixels, resolution row pixels, packet sequence number, and timestamp; the receiver's re-packaged information includes: number of packets sent, data packet delay, number of packet losses, packet loss speed, and packet loss acceleration. Step 2: The repackaged data at the receiving end will be used as the input of the Kalman prediction algorithm. The implementation process of Kalman filtering is mainly divided into two stages: the prediction stage and the update stage. In the prediction stage, the state and error covariance of the next moment are predicted based on the dynamic model of the system. This stage does not use observation data, but only relies on the system model and the state estimate of the previous moment. The main purpose of the update stage is to use the observation data at the current moment to update the state estimate and error covariance. This stage combines the prediction results and observation data to obtain a more accurate state estimate; Step 3: In order to further improve the accuracy of state estimation, especially when facing video stream data with continuity and time correlation, an RTSS smoother can be connected after completing a standard Kalman filter prediction. This method not only improves the accuracy of state prediction, but also provides a more reliable foundation for subsequent data processing and analysis; Step 4: When a poor communication state is predicted, an adaptive threshold method is designed to determine whether a frame supplement is needed at the next moment; Step 5: In the frame supplement algorithm, firstly, a buffer is set to save the historical images, and the previous frame image is cropped by the frame segmentation factor to obtain the video data closest to the next frame; Step 6: The image clarity will be affected to a certain extent after cropping. In order to achieve real-time continuous clarity improvement of the video, we need to perform image interpolation algorithm on the image that has been processed by frame segmentation to generate an image with a resolution close to that of the next frame.
2. According to claim 1, a vehicle queue clustering cruising method based on V2V beyond-horizon perception image transparent transmission optimization is characterized in that: The repackaged data at the receiving end will be used as the input of the Kalman prediction algorithm, which includes: The state equation and observation equation in the prediction stage from t to t+1 are: The system state error and observation error calculation formula is: Furthermore, the error covariance prediction value of the algorithm is Next, we need to calculate the gain matrix as follows: After completing the prediction phase, we enter the update phase. The state update equation in the update phase is: Finally, the error covariance of the system is updated. The error covariance of the updated system is: Define the state vector as Among them, q(t) is the number of packet losses, q'(t) is the packet loss rate, and q"(t) is the packet loss acceleration. The first-order variation contained in the state vector is the packet loss rate, and the second-order variation is the packet loss acceleration. Compared with the state vector that only considers one section of variation, it can more accurately describe the predicted packet loss rate. The state transfer matrix is 3. According to the RTSS smoothing algorithm part of the vehicle queue clustering cruising method based on V2V beyond-horizon perception image transparent transmission optimization according to claim 1, the algorithm process includes: Calculate the smoothing gain: Compute the smoothed estimate: Compute the smoothed mean square error: in, is a smoothed estimate of the state, is the smoothing gain matrix, P i (s) is the covariance matrix of the smoothed estimate. The smoothed estimate at any time i can be regarded as a combination of the filtered estimate at time i and the smoothed estimate at time i+1. Only after the forward filtering is completed, the RTSS algorithm begins smoothing to improve the filtering result. S41: Setting an adaptive threshold, and performing overload detection and discrimination on the transmission channel according to the delay gradient value and the adaptive threshold. The adaptive threshold formula is: Adapt_value=Adapt_value+(kalman_value-Adapt_value)*K u Where Adapt_value is the adaptive threshold, kalman_value is the output of the Kalman prediction algorithm, K u is the threshold parameter. When kalman_value>Adapt_value, it is determined that a frame needs to be supplemented at the next moment.
4. The frame supplement algorithm part of the V2V-based vehicle network video stream transmission continuity optimization method according to claim 3 is characterized in that: First, set the buffer area to save the historical images, and crop the previous frame image by the frame segmentation factor to obtain the video data closest to the next frame. The frame segmentation calculation formula is: The image clarity will be affected to a certain extent after cropping. In order to achieve real-time continuous video clarity improvement, we need to perform image interpolation algorithm on the image of the previous frame after frame segmentation to generate an image with a resolution close to the next frame. When interpolation is required at point P, it is known that A 11 =(x1,y1),A 12 =(x1,y2)=(x1,y1+1), A 21 =(x2,y1)=(x1+1,y1), A 22 =(x2,y2)=(x1+1,y1+1) The coordinates of the four points around point E are determined. First, a coordinate axis X is selected for interpolation through bilinear interpolation to obtain A. 11 and A 21 B1 between f(B1)=f(x1+n,y1)=n[f(x1+1,y1)-f(x1,y1)]+f(x1,y1) Similarly in A 12 and A 22 Interpolate between them to get B2, f(B2)=f(x1+n,y1+1)=n[f(x1+1,y1+1)-f(x1,y1+1)]+f(x1,y1+1) Further in the Y direction, the grayscale value of point P is obtained by interpolating B1 and B2. f(P)=f(x1+n,y1+m)=m[f(x1+n,y1+1)-f(x1+n,y1)]+f(x1+n,y1) Simplifying the above formula, we can get the final interpolation result of the interpolation frame f(x1+n,y1+m)=mn[f(x1+1,y1+1)-f(x1,y1+1)+f(x1+1,y1)-f(x1,y1)]+ f(x1,y1)+n[f(x1+1,y1)-f(x1,y1)]+m[f(x1,y1+1)-f(x1,y1)].
5. According to the final frame insertion part of the V2V-based vehicle network video stream transmission continuity optimization method described in claim 1, after the next frame image is generated by the frame insertion algorithm, it is also necessary to judge whether the next frame image is received. If it is received, the original image is displayed; if it is not received, the generated image is displayed on the driver's observation screen.