Vehicle scene prediction method

Abstract

The invention relates to the technical field of scene prediction, in particular to a vehicle scene prediction method, and provides the following scheme: a vehicle-mounted end collects vehicle end data and vehicle control data, a cloud end receives and generates a predicted scene video stream, and a cabin end performs color adjustment and image quality evaluation after receiving the predicted scene video stream. And adjusting a display queue according to an evaluation result and generating vehicle control data. In the color adjustment process, motion compensation, color distortion processing and secondary local compensation are performed based on the vehicle control data so as to improve the color consistency and space-time coherence of the prediction frame. Image quality evaluation is based on multi-color space feature extraction and a nonlinear regression network, quality scores are output in real time, and remote control decisions are guided. And the cloud end generates a continuous and high-quality prediction scene video stream by combining light stream prediction with self-motion compensation and shielding repair, so that low-delay and high-stability remote vehicle control is realized.

Description

Technical Field

[0001] The present invention relates to the field of scene prediction technology, and in particular to a vehicle scene prediction method. Background Art

[0002] In existing remote vehicle control technologies, video data is usually collected in real time from the vehicle side, processed in the cloud, and then transmitted to the cabin side for remote driving control. However, due to inference errors in scene prediction algorithms, especially in complex driving conditions such as acceleration, braking, and steering, problems such as local color distortion, brightness drift, and occlusion holes are prone to occur in the predicted video stream, resulting in a decrease in the accuracy of the vehicle control data generated by the cabin side based on the video stream, affecting the safety and reliability of remote control. Existing technologies mostly use traditional image quality assessment methods, which mainly focus on structural clarity or resolution indicators. They are difficult to sensitively capture the unique color distortion characteristics in the predicted scene, and lack the ability to perform adaptive color compensation and motion trend compensation based on vehicle motion characteristics. It is difficult to meet the strict requirements of the unmanned driving cabin side for continuous, high-quality visual input. Therefore, there is an urgent need for a technical solution that can perform fine color correction on the predicted video stream, reliable image quality assessment, and optimize the prediction generation based on vehicle motion information to improve the overall stability and safety of the remote control system.

[0003] For example, Chinese patent application publication number CN118094456A discloses a prediction model for factors influencing autonomous driving network performance in urban scenarios. This LSTM-based model includes feature fusion, establishing factors influencing autonomous driving network performance, building a neural network structure, constructing a loss function, training the neural network, and conducting simulation experiments to verify the model. The model includes the following steps: Step 1: Feature fusion, which serves as the input to the neural network; Step 2: Establishing factors influencing autonomous driving network performance, which serves as the output of the neural network; Step 3: Building the neural network structure; Step 4: Constructing the loss function; Step 5: Training the neural network; and Step 6: Verifying the model through simulation experiments.

[0004] The above existing technologies all have the problems raised by this background technology: local color distortion, brightness drift and occlusion holes are prone to occur in the predicted video stream, resulting in a decrease in the accuracy of vehicle control data generated by the cabin based on the video stream. In order to solve the above problems, this application designs a vehicle scene prediction method. Summary of the Invention

[0005] The technical problem to be solved by the present invention is to address the deficiencies of the existing technology and provide a vehicle scene prediction method. The vehicle-mounted end collects vehicle-side data and vehicle control data, the cloud receives and generates a predicted scene video stream, and the cabin end performs color adjustment and image quality evaluation after receiving it. According to the evaluation results, the display queue is adjusted and the vehicle control data is generated. During the color adjustment process, motion compensation, color distortion processing and secondary local compensation are performed based on the vehicle control data to improve the color consistency and spatiotemporal coherence of the predicted frame. Image quality assessment is based on multi-color space feature extraction and nonlinear regression network, and real-time output quality scores are used to guide remote control decisions. The cloud uses optical flow prediction combined with self-motion compensation and occlusion repair to generate a continuous, high-quality predicted scene video stream, realizing low-latency and high-stability remote vehicle control.

[0006] To achieve the above object, the present invention provides the following technical solutions:

[0007] A vehicle scene prediction method is applied to the cabin side of a remote video transmission platform. The remote video transmission platform also includes a vehicle-mounted terminal and a cloud. The vehicle-mounted terminal is used to transmit vehicle-side data, and the cloud is used to receive vehicle-side data from the vehicle-mounted terminal and generate a predicted scene video stream. The vehicle scene prediction includes:

[0008] The predicted scene video stream is received, and color adjustment is performed on the predicted scene video stream to generate an enhanced video stream, wherein the color adjustment includes:

[0009] Identify target and background areas through semantic segmentation, generate processing frames based on vehicle control data, and perform color distortion processing;

[0010] Processing the enhanced video stream using a pre-trained image quality assessment model to output a real-time quality score, wherein the image quality assessment model is constructed based on multi-color space feature extraction and a nonlinear regression network;

[0011] Comparing the real-time quality score with a preset quality threshold, and generating a compensation control signal based on the comparison result to adjust the display queue and generate a continuous display video stream;

[0012] Vehicle control data is generated according to the display video stream.

[0013] Performing color adjustment on the predicted scene video stream to generate an enhanced video stream includes:

[0014] Performing semantic segmentation on the predicted scene video stream to identify the target area and background area;

[0015] Acquiring vehicle control data of a current frame and a historical frame, adjusting the target area and the background area according to the vehicle control data of the current frame and the historical frame, and generating a first processing frame and a second processing frame;

[0016] Extracting a color space of the first processed frame, and performing color distortion processing on the color space according to vehicle control data of a current frame to generate a corrected frame;

[0017] The corrected frame is fused with the second processed frame according to the timestamp to generate an enhanced video stream.

[0018] Adjusting the target area and the background area according to the vehicle control data of the current frame and the historical frame includes:

[0019] Predicting a pixel displacement vector of the current frame in the historical frame based on vehicle posture parameters in the historical frame vehicle control data to obtain a first displacement matrix, wherein the vehicle posture parameters include a steering wheel angle, a longitudinal speed, and a brake opening;

[0020] Performing motion estimation on the first displacement matrix according to the current frame vehicle control data to obtain a second displacement matrix;

[0021] Fusing the first displacement matrix and the second displacement matrix to obtain a motion trend matrix;

[0022] Perform pixel-level motion compensation on the target area according to the motion trend matrix to generate a first processing frame;

[0023] Global affine motion compensation is performed on the background area according to the motion trend matrix to generate a second processed frame.

[0024] The performing color distortion processing on the color space according to the vehicle control data of the current frame includes:

[0025] Determine the current frame according to the vehicle control data to obtain a motion phase, wherein the motion phase includes an acceleration phase, a steering phase, and a braking phase;

[0026] Extracting pixels in a historical target area having a color entropy greater than a preset color threshold from a rolling buffer corresponding to the motion phase to generate a phase reference palette, wherein the rolling buffer is set based on a predicted scene video stream corresponding to a historical frame;

[0027] Comparing the color space with the phase reference palette channel by channel for deviation to generate a color shift metric;

[0028] The color space of the first processed frame is graded compensated according to the color shift metric to generate a corrected frame, wherein the graded compensation includes offset cancellation of all frame channels and chroma remapping of pixel blocks.

[0029] The fusing the corrected frame with the second processed frame according to the timestamp includes:

[0030] Performing spatiotemporal synchronization on the correction frame and the second processing frame according to the timestamp;

[0031] For each target pixel in the correction frame, a neighborhood set is constructed at the corresponding position in the second processing frame, wherein the search radius of the neighborhood set is adjusted according to the acceleration in the vehicle control data of the current frame;

[0032] In the neighborhood set, a preset number of neighborhood pixels are selected and their color offset vectors are extracted by jointly sorting them according to color similarity and spatial distance;

[0033] A local color compensation coefficient is calculated according to the color shift vector to generate a quadratic compensation matrix, and the target pixel is compensated according to the quadratic compensation matrix until each target pixel is processed.

[0034] The processing of the enhanced video stream by using a pre-trained image quality assessment model includes:

[0035] Sequentially acquiring frames to be evaluated from the enhanced video stream at a preset sampling frequency;

[0036] Perform multiple color space conversions in parallel on each frame to be evaluated to obtain RGB, YCbCr, Lab and HSV representations of the frame to be evaluated;

[0037] Extract brightness, chroma component mean, chroma component variance, color contrast, edge gradient amplitude and local texture energy in each color space, and splice them to form a frame-level feature vector;

[0038] Inputting the frame-level feature vector into a nonlinear regression network trained offline to obtain a corresponding frame-level quality score;

[0039] Temporal exponential smoothing is performed on the frame-level quality scores to generate real-time quality scores.

[0040] Generating a compensation control signal according to the comparison result includes:

[0041] If the real-time quality score is less than the preset quality threshold, generating a first compensation control signal, wherein the first compensation control signal marks the current frame as an invalid frame and deletes it from the display queue, and performs motion compensation interpolation between adjacent valid frames to generate a replacement frame;

[0042] If the real-time quality score is greater than or equal to the preset quality threshold, generating a second compensation control signal, wherein the second compensation control signal inserts the current frame into the display queue as a valid frame;

[0043] The first compensation control signal or the second compensation control signal is written into a display buffer control register at the cabin end to update a display queue.

[0044] A vehicle scene prediction method is applied to the cloud side of a remote video transmission platform. The remote video transmission platform also includes an on-board terminal and a cabin terminal. The on-board terminal is used to transmit vehicle-side data. The cloud side is used to receive vehicle-side data from the on-board terminal and generate a predicted scene video stream. The cabin side is used to receive the predicted scene video stream from the cloud side and perform quality assessment. Vehicle control data is generated based on the quality assessment results. The vehicle scene prediction includes:

[0045] Receiving the vehicle-side data and the vehicle control data of the current frame, preprocessing two consecutive frames of video data in the vehicle-side data to generate a first preprocessed frame and a second preprocessed frame;

[0046] The pre-processed vehicle-side data and vehicle control data are input into a preset scene prediction model. Based on the temporal characteristics of adjacent video frames and the vehicle motion state represented by the vehicle control information, the pixel-level motion vector is calculated using the optical flow prediction algorithm to generate a predicted scene video stream of multiple frames.

[0047] The predicted scene video stream and the original video stream are synchronously marked with timestamps, encoded and encrypted, and then sent to the cabin end.

[0048] The method calculates pixel-level motion vectors based on the temporal characteristics of adjacent video frames and the vehicle motion state represented by the vehicle control information through an optical flow prediction algorithm to generate a plurality of frames of predicted scene video streams, including:

[0049] Constructing a multi-scale pyramid in the first pre-processed frame and the second pre-processed frame, and traversing the multi-scale pyramid using an optical flow algorithm to generate an initial pixel-level motion vector field;

[0050] Calculating a vehicle self-motion transformation matrix based on the vehicle control data of the current frame, and compensating the initial pixel-level motion vector field based on the vehicle self-motion transformation matrix to generate a motion vector field;

[0051] Taking brightness constancy constraint and gradient constancy constraint as the goal, the motion vector field is iteratively optimized in combination with total variation regularization to generate a refined vector field;

[0052] performing inverse warping and resampling on the second pre-processed frame according to the refined vector field to generate a first predicted frame;

[0053] Taking the first predicted frame as the current frame, optical flow estimation, self-motion compensation, vector optimization and resampling are performed cyclically until the target number of predicted frames are generated to form a predicted frame sequence. The predicted frame sequence is then subjected to forward and backward consistency detection frame by frame, the occluded areas are marked, and edge-preserving interpolation and block matching are used to repair and fill the occluded holes, and the predicted scene video stream is output.

[0054] A vehicle scene prediction method is applied to a vehicle-mounted terminal of a remote video transmission platform, wherein the vehicle-mounted terminal includes multiple cameras. The remote video transmission platform also includes a cloud terminal and a cabin terminal. The cloud terminal is used to receive vehicle-side data from the vehicle-mounted terminal and generate a predicted scene video stream. The cabin terminal is used to receive the predicted scene video stream from the cloud terminal and perform quality assessment, and generate vehicle control data based on the quality assessment results. The vehicle scene prediction method includes:

[0055] Send the vehicle-side data collected by the camera and the vehicle control data of the current frame to the cloud;

[0056] Receive vehicle control data from the cabin.

[0057] Compared with the prior art, the present invention has the following beneficial effects:

[0058] This application introduces an image quality assessment method based on vehicle-controlled data, including motion trend compensation, local color distortion processing, and multi-color space feature extraction. This method effectively addresses image quality degradation caused by color shift, occlusion holes, and motion errors in existing remote prediction video streams. Acceleration-based adaptive neighborhood search and local secondary color compensation techniques improve the color consistency and spatial naturalness of the predicted frames. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] Other features, objects and advantages of the present invention will become more apparent upon reading the detailed description of non-limiting embodiments made with reference to the following drawings:

[0060] Figure 1 This is a schematic diagram of an exemplary application scenario of an embodiment of the present invention;

[0061] Figure 2 A schematic flow chart of a vehicle scene prediction method according to an embodiment of the present invention;

[0062] Figure 3 A schematic diagram of a process flow of a predictive video enhancement method according to an embodiment of the present invention;

[0063] Figure 4 This is a schematic diagram of an image quality scoring process according to an embodiment of the present invention;

[0064] Figure 5 A schematic flow chart of another vehicle scene prediction method according to an embodiment of the present invention;

[0065] Figure 6 The figure is a flow chart of another vehicle scene prediction method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0066] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments.

[0067] References to "embodiments" herein mean that a particular feature, structure, or characteristic described in connection with the embodiments may be included in at least one embodiment of the present application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor does it refer to independent or alternative embodiments that are mutually exclusive of other embodiments. It will be understood, both explicitly and implicitly, by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0068] An embodiment of the present application provides a vehicle scene prediction method that can achieve remote control of unmanned driving based on 5G communication. Specifically, unmanned vehicles need to transmit high-definition images and videos in real time so that the remote control center or other vehicles can analyze and make decisions.

[0069] See also Figure 1 , which is a schematic diagram of an exemplary application scenario provided in an embodiment of the present application.

[0070] like Figure 1 As shown in the figure, the application scenario consists of the vehicle side, cloud side, and cabin side. The three realize data exchange through the transmission network, among which:

[0071] Figure 1 The vehicle-side system includes image sensors and dynamic sensors, which are used to collect real-time raw video streams of the vehicle driving scene (i.e., vehicle-side data) and vehicle motion parameters (i.e., vehicle control data, including longitudinal speed, steering angle, brake opening, etc.). This vehicle-side data and vehicle control data are uploaded to the cloud via a transmission network and serve as the basic input for the cloud to generate a video stream for the predicted scene.

[0072] In one example, the vehicle side can also collect vehicle data through other sensors. The sensors mentioned here include but are not limited to multi-camera arrays, steering wheel angle sensors, accelerometers, and brake opening sensors.

[0073] Figure 1 The cloud-based workflow is shown. First, data preprocessing is performed, including format conversion, resolution adjustment, and format conversion of vehicle control information. Then, the data is analyzed through the scene prediction model deployed in the cloud to predict subsequent scene changes, generate a predicted scene video stream, and upload the predicted scene video stream to the cabin end through the transmission network.

[0074] Figure 1The workflow of the cabin side is shown. First, the video data sent by the cloud is received and decoded. In order to address the problem of insensitivity to color distortion in the predicted video, which makes it impossible to accurately screen high-quality predicted frames and affects the security of remote control, the predicted video is color adjusted. Then, it is evaluated through the image quality assessment model, the display queue is adjusted, and a continuous display video stream is generated. Finally, vehicle control data is generated based on the display video stream and uploaded to the vehicle side through the transmission network.

[0075] In one example, Figure 1 The transmission network shown can be set up through a 5G network to reduce transmission latency and achieve real-time control of unmanned vehicles.

[0076] In one example, Figure 1 The cabin end shown also includes a manual assisted image assessment module, which is used to introduce a manual intervention mechanism based on the automatic image quality assessment. When the real-time quality score output by the image quality assessment model is lower than the second preset manual review threshold, or when there are abnormalities such as color anomalies, image blur, motion vector anomalies, etc. in the continuous multi-frame predicted scene video stream, the manual review process is automatically triggered, prompting the cabin end operator to perform manual confirmation. The operator quickly reviews the current frame based on the visual interface of the predicted scene video stream and manually confirms whether to accept, discard or replace the current frame, thereby further improving the accuracy of image screening in abnormal situations and the security of remote control on the basis of automatic processing. The review results can be synchronously recorded in the back-end log system for subsequent data training and system optimization.

[0077] Next, the vehicle scene prediction method provided in the embodiment of the present application is introduced with reference to the accompanying drawings.

[0078] See also Figure 2 , which is a flow chart of a vehicle scene prediction method provided in an embodiment of the present application. Figure 2 The method shown can be applied to the cabin side of a remote video transmission platform, which also includes a vehicle-mounted side and a cloud side. The vehicle-mounted side is used to transmit vehicle-side data, and the cloud side is used to receive vehicle-side data from the vehicle-mounted side and generate a predicted scene video stream. Figure 2 The method shown includes the following steps S1-S4, and the specific steps are as follows:

[0079] S1: Receive the predicted scene video stream, perform color adjustment on the predicted scene video stream, and generate an enhanced video stream;

[0080] In this embodiment, after receiving a scene video stream generated through cloud-based prediction, the cabin first uses semantic segmentation to identify the target and background regions within the video frame. Based on vehicle control data from the current and historical frames, motion trend compensation and color distortion correction are performed on each region to generate a continuous, enhanced video stream with enhanced color quality. By inferring a color distortion adjustment strategy based on motion state, this effectively addresses the issue of traditional predicted videos being insensitive to color distortion in the target region under changing scenes, improving the color fidelity of the predicted frames and the reliability of subsequent decisions.

[0081] S2: Processes the enhanced video stream using a pre-trained image quality assessment model and outputs a real-time quality score;

[0082] In this embodiment, a pre-trained image quality assessment model is used to perform real-time quality scoring for enhanced video streams. This model extracts features from three-dimensional multi-color spaces (RGB, YCbCr, Lab, and HSV) and combines them with a nonlinear regression network to achieve frame-level quality prediction. By introducing complex feature dimensions such as luminance, chrominance, and edge gradients in multiple color spaces, the ability to predict video distortion is significantly improved, particularly the accuracy of detecting color shift and localized distortion, thereby providing a high-confidence reference for subsequent compensation strategies.

[0083] S3: comparing the real-time quality score with a preset quality threshold, generating a compensation control signal based on the comparison result, and generating a continuous display video stream;

[0084] In this embodiment, the real-time quality score is compared against a set quality threshold. If the current frame's score falls below the threshold, it is marked as invalid and removed from the display queue. A replacement frame is generated using motion-compensated interpolation based on adjacent valid frames. If the score is higher than or equal to the threshold, the current frame is directly added to the display queue. This dynamic update of the display queue effectively ensures the stability of the final output video stream in terms of color continuity and motion coherence, avoiding visual discontinuity in remote control caused by single-frame distortion and improving the security of remote control decision-making.

[0085] In one example, generating a compensation control signal according to the comparison result includes:

[0086] If the real-time quality score is less than the preset quality threshold, generating a first compensation control signal, wherein the first compensation control signal marks the current frame as an invalid frame and deletes it from the display queue, and performs motion compensation interpolation between adjacent valid frames to generate a replacement frame;

[0087] If the real-time quality score is greater than or equal to the preset quality threshold, generating a second compensation control signal, wherein the second compensation control signal inserts the current frame into the display queue as a valid frame;

[0088] The first compensation control signal or the second compensation control signal is written into a display buffer control register at the cabin end to update a display queue.

[0089] S4: Generate vehicle control data based on the displayed video stream;

[0090] In this embodiment, key target objects and environmental state changes are extracted from a continuously output, high-quality display video stream. This data is then combined with the time series relationships within the display queue to generate vehicle control data for remote control. This generated vehicle control data includes commands such as steering, acceleration, and deceleration, and is transmitted back to the vehicle via a transmission network in real time for execution. By ensuring high color consistency and motion coherence within the video frames used to generate vehicle control commands, safety risks caused by image misinterpretation during remote control can be further reduced, improving the accuracy and robustness of overall driving decisions.

[0091] Although several solutions have been proposed in the prior art for optimizing the quality of remotely predicted video streams, such as image enhancement processing based on convolutional neural networks or applying image quality correction algorithms directly at the generation end, in practical applications, these methods generally fail to fully consider the problem of local color distortion caused by rapid scene changes and vehicle self-motion in the predicted scene video stream in a dynamic driving environment. Especially in the application scenario of the unmanned cockpit end, if the predicted image has color deviation or local distortion, it is very easy to interfere with the judgment criteria of the image quality assessment model, resulting in the misjudgment of low-quality frames as high-quality frames, or vice versa, thereby seriously affecting the remote control system's screening and use of safety decision-making frames. Existing single color correction methods generally focus on global adjustments and lack precise control over motion trends, color drift, and local area distortion. They are unable to dynamically and finely achieve color distortion compensation and realism restoration in changing scenes.

[0092] This application establishes an adaptive color adjustment process by introducing motion phase determination, rolling cache to extract reference samples with high color entropy, dynamically generating reference palettes, and pixel-level hierarchical compensation mechanisms. Specifically, by combining vehicle posture parameters such as steering wheel angle, longitudinal speed, and brake opening, the motion phase of the current predicted frame is first inferred, thereby screening out the historical reference color data with the best spatiotemporal consistency, avoiding directly using the distortion characteristics of the current frame as a benchmark, and reducing the system's sensitivity to abnormal frames. Pixels in the historical target area with high color entropy are then dynamically extracted through a rolling cache method to construct a spatiotemporally stable reference palette, further ensuring that the color compensation strategy has adaptability and noise resistance. Finally, in the compensation stage, the color distortion in the predicted image is accurately corrected through a hierarchical strategy of global bias cancellation and local pixel block chromaticity remapping, so that the subsequent image quality assessment network based on multi-color space feature extraction can obtain more realistic and reliable input features, significantly improving the accuracy of high-quality frame screening. Compared to existing solutions that rely solely on single spatial enhancement or static color adjustment, this embodiment not only considers the dynamic correlation between the predicted frame and the vehicle's motion state, but also integrates spatiotemporal stability and pixel-level flexible correction in the local compensation process. This allows for a deep coupling of color distortion suppression with the image quality assessment process, improving the overall system's adaptability to complex driving scenarios. For example, under conditions of high-speed scene switching, continuous steering, or strong lighting changes, this embodiment can effectively stabilize the color characteristics of the predicted image, increasing the cabin's success rate in screening continuous, high-quality predicted frames, significantly enhancing the remote control system's decision-making accuracy and driving safety.

[0093] See also Figure 3 , which is a flow chart of a predictive video enhancement method provided in an embodiment of the present application. Figure 3 The method shown can be applied to step S1 of the aforementioned method, and the specific steps are as follows:

[0094] S1.1: Perform semantic segmentation on the predicted scene video stream to identify the target area and background area;

[0095] Specifically, after the predicted scene video stream undergoes scene generation, different regions in the image have significant differences in sensitivity to color distortion due to vehicle motion, camera angle changes, and varying ambient lighting conditions. This is especially true for target areas such as the road ahead and obstacles, which play a key role in subsequent remote control decisions. Therefore, semantic segmentation of the predicted scene video stream is necessary to distinguish key target areas from background areas in the image. This provides a precise regional division basis for subsequent color adjustment and local processing, ensuring that the color compensation strategy is targeted and avoiding ineffective processing or over-compensation.

[0096] In this embodiment, a real-time semantic segmentation model based on a deep convolutional neural network is used to perform frame-by-frame processing on each received frame of the predicted scene video stream. The semantic segmentation model adopts an encoder-decoder structure. The encoder part is used to extract multi-scale semantic features, and the decoder part combines the jump connection mechanism to perform high-precision spatial feature restoration, thereby outputting a pixel-level category segmentation map. According to the segmentation map, the image is divided into target areas (such as the road ahead, pedestrians, vehicles, traffic signs) and background areas (such as the sky, buildings, vegetation, etc.). Each pixel is classified according to the predicted category to form a target mask and a background mask, which serve as the basis for subsequent adjustment processing.

[0097] S1.2: Obtain vehicle control data of the current frame and historical frames, and adjust the target area and background area according to the vehicle control data of the current frame and historical frames to generate a first processed frame and a second processed frame;

[0098] Specifically, during actual vehicle operation, acceleration, braking, steering, and other movements can cause rapid changes in the position of objects in the image. Directly processing color based on the current frame can easily lead to inaccurate color matching due to spatial misalignment. Therefore, it is necessary to incorporate historical vehicle control data to compensate for motion trends in the target and background areas, correct for displacement effects caused by vehicle motion, and generate aligned frames to ensure consistency and accuracy in subsequent color feature extraction.

[0099] In this embodiment, the expected pixel displacement of the current frame in the historical frame is first calculated based on the vehicle posture data of the historical frame to construct a preliminary first displacement matrix. Then, based on the actual vehicle control data of the current frame, the first displacement matrix is ​​motion-estimated and corrected to generate a second displacement matrix that reflects the actual motion trend. The first and second displacement matrices are fused to obtain a final motion trend matrix. Based on this motion trend matrix, pixel-level motion compensation is performed on the target area to correct for target displacement deviations caused by motion, generating the first processed frame. Simultaneously, a global affine transformation is applied to the background area for overall motion compensation to generate the second processed frame to ensure overall scene consistency.

[0100] Preferably, during motion compensation, to avoid the problem of increased compensation errors due to intense driving maneuvers such as strong acceleration and sharp turns, this embodiment introduces a motion trend correction mechanism based on acceleration thresholds. Specifically, before performing motion trend compensation, the longitudinal and lateral acceleration indicators in the current frame's vehicle control data are first extracted and compared against preset dynamic thresholds. If either acceleration indicator exceeds the set threshold, the system determines that the vehicle is in a state of intense motion, such as sudden acceleration, deceleration, or sharp turns, triggering the motion trend correction logic.

[0101] In one example, the correction process specifically includes the following steps: first, the primary direction of impact is determined based on the type of acceleration exceeding the threshold (longitudinal or lateral); when the longitudinal acceleration exceeds the limit, the displacement vector component in the fore-aft direction is prioritized for correction; when the lateral acceleration exceeds the limit, the displacement vector component in the left-right direction is prioritized for correction. Subsequently, based on the absolute value of the acceleration, the vector amplitude in the motion trend matrix in the corresponding direction is dynamically adjusted according to a preset scaling factor. The scaling factor is adaptively set based on the deviation amplitude of the acceleration. The greater the deviation, the greater the adjustment amplitude, thereby effectively suppressing displacement estimation anomalies caused by violent vehicle body movement.

[0102] In one example, to ensure consistent motion compensation, this embodiment employs a gradual vector adjustment strategy during the correction process. This strategy involves smoothly stretching or compressing the vectors in the motion trend matrix over several consecutive frames, rather than abruptly changing them. This prevents the introduction of new spatial distortions due to excessive compensation changes. Furthermore, a capping mechanism is implemented to limit the vector adjustment amplitude to a maximum allowable threshold. This ensures that even in extreme driving conditions, the first and second processed frames maintain reasonable spatial alignment after compensation.

[0103] In an example, the specific steps for adjusting the target area and the background area are as follows:

[0104] S1.2.1: Predicting a pixel displacement vector of the current frame in the historical frame based on vehicle posture parameters in the historical frame vehicle control data to obtain a first displacement matrix, wherein the vehicle posture parameters include steering wheel angle, longitudinal speed, and brake opening;

[0105] In this embodiment, vehicle control data for the corresponding time point in the historical frame is first extracted, including steering wheel angle, longitudinal velocity, and brake position. The steering wheel angle is used to estimate vehicle yaw, longitudinal velocity is used to estimate forward travel distance, and brake position reflects the longitudinal velocity trend. These vehicle posture parameters are then fed into a deduction model, which primarily relies on the vehicle's driving dynamics model. The model then infers the translation and rotation trends of the current frame relative to the historical frame. This is then mapped onto the image plane to obtain an estimated displacement vector for each pixel between the historical frame and the current frame, thereby forming a first displacement matrix.

[0106] Furthermore, during displacement estimation, this embodiment adjusts the estimation frequency based on driving mode to account for differences in vehicle behavior under different road conditions and driving modes. When historical frame vehicle control data indicates a sudden acceleration or sharp turn, the system automatically switches to a high-dynamic response model, employing higher-order nonlinear motion estimation to improve displacement prediction accuracy and avoid the cumulative prediction errors associated with using a single linear model.

[0107] S1.2.2: Perform motion estimation on the first displacement matrix based on the current frame vehicle control data to obtain a second displacement matrix;

[0108] Specifically, because historical vehicle control data only reflects past vehicle states, and vehicle motion is constantly changing, relying solely on historical data can introduce lags and lead to inaccurate displacement estimates. To correct this inaccuracy, the initial first displacement matrix, generated in real time, must be estimated and updated in conjunction with the current frame's vehicle control data to improve displacement prediction accuracy.

[0109] In this embodiment, real-time vehicle control data is extracted at the time point corresponding to the current frame, including steering wheel angle, longitudinal speed, and brake position. By comparing current vehicle control data with historical data, changes in vehicle motion trends, such as yaw rate changes and acceleration / deceleration trends, are inferred. Based on these changes, each pixel displacement vector in the first displacement matrix is ​​adjusted, refining its offset direction and magnitude. This generates a second displacement matrix that better reflects the current vehicle's actual motion state and reflects the impact of real-time vehicle motion on image displacement.

[0110] S1.2.3: Fusing the first displacement matrix and the second displacement matrix to obtain a motion trend matrix;

[0111] Specifically, the first displacement matrix extrapolated from the historical frame and the second displacement matrix corrected based on current vehicle control data represent the predicted trend and the real-time corrected trend, respectively. Using them independently can easily lead to under-prediction or over-correction. To balance historical continuity with real-time responsiveness, the two need to be fused, integrating predictions and real-time changes to form a unified motion trend matrix that reflects the vehicle's actual motion trends.

[0112] In this embodiment, fusion weights are set to dynamically adjust the proportions of the first and second displacement matrices based on the vehicle's motion state. For example, during stable driving, the weight of the historical prediction component is increased to ensure consistent image displacement compensation; during intense motion, the weight of the real-time correction component is increased to enhance the real-time nature of compensation. During the fusion process, the corresponding displacement vectors of the two matrices are synthesized using a pixel-level weighted average, ensuring both local and global physical coherence of the motion trend matrix.

[0113] Preferably, in the fusion stage, in order to prevent the introduction of motion artifacts after fusion, this embodiment also includes a local consistency detection mechanism, that is, verifying the consistency of the displacement direction before and after fusion in each local area. When the local inconsistency exceeds the threshold, it automatically triggers the regional weight redistribution to enhance local coherence and improve the overall spatial coordination of the motion trend matrix.

[0114] S1.2.4: Perform pixel-level motion compensation on the target area according to the motion trend matrix to generate a first processed frame;

[0115] Specifically, the target region typically includes important elements such as the road ahead, vehicles, and pedestrians. The accuracy of its position directly impacts subsequent color adjustments and remote control command generation. To ensure the spatial consistency of the target region in the predicted video stream, pixel-level motion compensation is performed on the target region based on the motion trend matrix, accurately mapping the target region from the historical frame to the current frame.

[0116] In this embodiment, pixel-level reverse mapping is performed for each pixel within the target region mask based on the displacement vector in the motion trend matrix. This involves migrating the color value of the corresponding pixel in the historical frame to the target pixel position in the current frame according to the compensation vector, generating the aligned first processed frame. This process employs bilinear interpolation to mitigate image tearing and aliasing caused by discrete displacement, enhancing the smoothness and naturalness of the compensated image.

[0117] S1.2.5: Performing global affine motion compensation on the background area according to the motion trend matrix to generate a second processed frame;

[0118] Specifically, the background region is less sensitive to displacement accuracy than the target region and typically exhibits large, continuous features. To ensure reasonable compensation while reducing computational complexity, a holistic affine motion compensation method can be used for the background region. This avoids the high overhead of pixel-by-pixel calculations while improving background consistency and stability.

[0119] In this embodiment, a global affine transformation matrix is ​​estimated based on the overall displacement distribution characteristics of background pixels in the motion trend matrix. This affine transformation matrix is ​​calculated using a least-squares fit method based on the relationship between the initial and expected positions of background pixels. It effectively approximates the large-scale translation, rotation, and scale changes of the background region. Subsequently, the background region is globally mapped based on the affine transformation matrix, and resampled to obtain the second processed frame.

[0120] S1.3: Extracting the color space of the first processed frame, and performing color distortion processing on the color space according to the vehicle control data of the current frame to generate a corrected frame;

[0121] Specifically, the motion phase (acceleration, steering, braking) of the current frame is first comprehensively judged based on the steering wheel angle, longitudinal speed and brake opening to determine the dominant trend of color drift. Subsequently, pixel samples of the historical target area with color entropy higher than the set color threshold are extracted from the rolling buffer corresponding to the motion phase to construct a reference palette with strong spatiotemporal correlation. A multi-channel deviation comparison is performed on the color space of the extracted first processing frame, and the offset between the current color component and the reference palette is calculated channel by channel to generate a color offset metric. Based on the color offset metric, a hierarchical compensation strategy is implemented: first, the offset of the full-frame channel is offset to correct the global color drift, and then local chromaticity remapping is performed in units of pixel blocks to restore the details and realism of the local color, thereby outputting a precisely corrected correction frame.

[0122] In an example, the specific steps of performing color distortion processing on a color space are as follows:

[0123] S1.3.1: Determine the current frame based on the vehicle control data to obtain a motion phase, where the motion phase includes an acceleration phase, a steering phase, and a braking phase;

[0124] Specifically, the appearance of color distortion in the predicted scene video stream is closely related to the vehicle's current motion state. The distribution patterns of image color shift and brightness changes vary significantly under different motion states. Therefore, to improve the specificity of subsequent color distortion processing, it is necessary to first determine the motion phase corresponding to the current frame and develop an adaptive color processing strategy based on this phase difference, thereby more precisely controlling the distortion compensation process.

[0125] In this embodiment, the vehicle control data accompanying the current frame, including the longitudinal velocity change rate, steering wheel angle change rate, and brake opening change rate, are used to determine the characteristic quantities of the acceleration, steering, and braking states, respectively. The primary motion characteristics of the current frame are determined by setting acceleration thresholds, steering angle change thresholds, and brake opening thresholds. When the longitudinal velocity change rate exceeds the acceleration threshold and the steering wheel angle change rate is less than the steering angle change threshold, the phase is determined to be an acceleration phase; when the steering wheel angle change rate exceeds the steering angle change threshold, the phase is determined to be a steering phase; and when the brake opening change rate exceeds the brake opening threshold, the phase is determined to be a braking phase.

[0126] As a preference, in order to avoid the problem of unstable motion phase judgment under continuous and rapid changing conditions, in this embodiment, a short-time smoothing mechanism is adopted to determine the motion phase by adopting the principle of dominance of dominant motion features within three consecutive frames, that is, the motion feature occupying the largest number of frames is used as the current motion phase, which improves the consistency of motion judgment and prevents the color processing process from causing processing oscillations due to frequent phase switching.

[0127] S1.3.2: Extracting pixels in the historical target region having a color entropy greater than a preset color threshold from a rolling buffer corresponding to the motion phase to generate a phase reference palette, wherein the rolling buffer is set based on a predicted scene video stream corresponding to the historical frame;

[0128] Specifically, there is a large uncertainty in simply relying on the current frame for color compensation, especially in unmanned driving scenarios. Due to prediction errors, local occlusion caused by vehicle motion, and environmental changes, the color characteristics of the current frame may be significantly offset, affecting the accuracy and stability of subsequent color correction. Therefore, in order to obtain a more reliable color reference, it is necessary to look back in time, use historical data to extract stable color features, and establish a spatiotemporal continuous color benchmark. Since the cabin only receives the predicted scene video stream transmitted from the cloud, and in order to minimize the transmission delay, the cloud will not send the original video data from the vehicle side, this embodiment preferably uses the historical predicted scene video stream as the data source. The historical predicted scene video stream is generated based on the actual motion state of the vehicle and environmental changes. It is highly correlated with the current frame in terms of temporal continuity and motion consistency, and can effectively support the spatiotemporal consistency requirements in the process of color distortion correction, thereby reasonably replacing the original frame as the color reference source.

[0129] In this embodiment, independent rolling cache modules are set up for different vehicle motion phases (acceleration, steering, braking). Each cache module is responsible for storing the pixel information of the target area in several prediction frames received within a limited time window under the corresponding motion phase. In order to screen representative and stable pixel samples, this embodiment calculates the color entropy of the target area image block in the rolling cache, and determines the color expression ability of the pixel block by evaluating the color richness and texture complexity of the local area. A color threshold is set, and pixel blocks with entropy values ​​higher than the color threshold are marked as valid samples. The main color components are extracted to construct a phase reference palette. Pixel blocks with large color entropy are often located in areas with rich color changes and obvious lighting details. Their color characteristics maintain good consistency during motion, and therefore can be used as a reliable basis for compensation.

[0130] Furthermore, to prevent pixel samples in the cache from losing their representativeness due to long-term accumulation, this embodiment introduces a dynamic time window management mechanism in the rolling cache, retaining only valid pixels with high color entropy within the most recently set time length (e.g., 1 or 2 seconds). When the time window is exceeded, expired samples are automatically eliminated, ensuring that the color palette can reflect the true color distribution characteristics of the current motion environment in real time. At the same time, combined with motion phase classification, color reference confusion caused by vehicle motion state switching is avoided, further improving the targeted and robustness of color distortion processing.

[0131] S1.3.3: Compare the color space to the phase reference palette channel by channel for deviation and generate a color shift metric;

[0132] Specifically, in order to quantify the degree of color distortion in the target area of ​​the current frame, it is necessary to compare the color characteristics of the current frame with the historical reference colors channel by channel. Through deviation measurement calculation, the changing trend of the spatial distribution of different color components (such as R, G, B, or L, a, b in Lab space) can be accurately described to provide a numerical basis for subsequent compensation.

[0133] In this embodiment, the first processed frame is mapped to a unified color space (preferably Lab space to improve the decoupling of brightness and color channels). Then, the L, a, and b channels are statistically analyzed for mean, variance, and extreme value differences compared to the corresponding channels in the phase reference palette. Based on the statistical results, an offset metric is calculated for each channel, characterized by the magnitude of the difference between the current frame channel mean and the reference channel mean, as well as the rate of change of variance.

[0134] S1.3.4: Performing hierarchical compensation on the color space of the first processed frame according to the color shift metric to generate a corrected frame, wherein the hierarchical compensation includes offset cancellation of all frame channels and chroma remapping of pixel blocks;

[0135] Specifically, color distortion usually manifests itself as the superposition of overall drift and local chromatic distortion. Therefore, a hierarchical compensation strategy is needed. First, the global channel bias correction is performed to unify the overall color benchmark of the frame, and then the local area is carefully corrected to achieve refined correction and avoid new local distortion problems caused by simple global adjustments.

[0136] In this embodiment, a unified offset is first applied to each color channel of the first processed frame based on the color offset measurement results. This means that the overall luminance or chromaticity of each channel is shifted toward the reference direction according to the offset direction to offset the overall color drift. After completing the global offset, the frame is divided into several small pixel blocks (preferably 16×16 or 32×32 in size). Local color statistics are again performed on the pixels within each block and compared with local samples of the reference palette. If the local offset exceeds a fine-grained threshold, the pixel block is chromatically remapped. The remapping operation is completed using a local color space transformation matrix to restore the color distribution within the region.

[0137] S1.4: Fusion the corrected frame with the second processed frame according to the timestamp to generate an enhanced video stream;

[0138] It should be noted that the corrected frame and the second processed frame here both correspond to single-frame images, and the enhanced video stream is generated by aggregating and sorting the single-frame images by time.

[0139] Specifically, the corrected frame and the second processed frame represent the target area and background area after color correction, respectively. Due to the asynchronous processing of the two in motion compensation and processing strategies, direct superposition may lead to visual incoherence problems such as edge dislocation and color mutation. Therefore, strict spatiotemporal synchronization processing is required according to the timestamp, and seamless splicing is achieved through the neighborhood fusion strategy to ensure the integrity and consistency of the enhanced video stream.

[0140] In one example, the correction frame and the second processing frame are first aligned based on the timestamp to ensure that the spatial references of the two are consistent at the same moment. Subsequently, for each target pixel in the correction frame, a neighborhood set is established at the corresponding position of the second processing frame. The neighborhood radius is dynamically adjusted according to the acceleration in the vehicle control data to adapt to local differences under different motion intensities. The neighborhood set is jointly sorted according to color similarity and spatial distance, and several optimal neighborhood pixels are selected to extract the color offset vector to generate a local compensation coefficient matrix. The local compensation coefficient matrix is ​​applied to the target pixels of the correction frame to achieve secondary local color compensation and boundary transition processing, and finally the fusion of the correction frame and the second processing frame is completed, and a continuous and natural enhanced video stream is output. The specific steps for the fusion of the correction frame and the second processing frame are as follows:

[0141] S1.4.1: Performing spatiotemporal synchronization on the correction frame and the second processed frame according to the timestamp;

[0142] Specifically, the corrected frame and the second processed frame originate from different processing paths, each independently compensated for the target and background regions. Due to differences in their processing timing, direct fusion can easily lead to spatial misalignment or timeline asynchrony. To ensure precise correspondence between pixel positions and timelines during fusion, spatiotemporal synchronization based on timestamps is performed before fusion.

[0143] In this embodiment, the generated timestamps of the correction frame and the second processed frame are first extracted. The timestamps of the two frames are compared, and the pair of frames closest in time is selected as synchronization candidate frames based on a set maximum allowable time deviation threshold. If there is a small time offset, this embodiment preferably uses a method based on bidirectional optical flow compensation to perform subtle pixel-level temporal interpolation correction on the earlier frame to align the two frames as closely as possible on the time axis. This ensures that after spatial synchronization, the fused region will not experience jumps due to time misalignment.

[0144] S1.4.2: For each target pixel in the correction frame, construct a neighborhood set at the corresponding position in the second processing frame, wherein the search radius of the neighborhood set is adjusted according to the acceleration in the vehicle control data of the current frame;

[0145] Specifically, as the vehicle's acceleration changes dramatically during motion, the pixel displacement amplitude of objects in the scene also changes accordingly, resulting in varying degrees of offset in the spatial correspondence between the target area and the background area. Using a fixed-radius neighborhood search may not cover the truly relevant background pixels, affecting the accuracy of subsequent color compensation. Therefore, it is necessary to dynamically adjust the neighborhood search radius based on the vehicle's acceleration in the current frame to ensure that the neighborhood set can both accommodate large displacements caused by violent motion and control the search range during stable driving, thereby improving processing accuracy and efficiency.

[0146] In this embodiment, for each pixel marked as a target area in the correction frame, its initial corresponding position in the second processing frame is first determined. The longitudinal and lateral accelerations in the vehicle control data of the current frame are compared with the set low, medium, and high acceleration thresholds, respectively, and the search radius is set accordingly for different speed levels. The greater the acceleration, the larger the search radius to capture the offset effects that may be caused by intense movement. Conversely, a smaller radius is used to improve local search accuracy. The pixels within the neighborhood set include all valid background pixels within the search radius and their color attribute data.

[0147] Preferably, in this embodiment, the motion intensity reference value is compared with the set multi-level acceleration threshold, and a specific neighborhood search radius is set according to different acceleration intervals. The specific settings are as follows:

[0148] When the exercise intensity is less than 1m / s 2 (low-speed steady driving stage), the search radius is set to 3 pixels;

[0149] When the exercise intensity is greater than or equal to 1m / s 2 And less than 3m / s 2 In between (medium speed cruising or slight turning phase), the search radius is set to 5 pixels;

[0150] When the exercise intensity is greater than or equal to 3m / s 2 And less than or equal to 6m / s 2 During the period of rapid acceleration or moderate turning, the search radius is set to 7 pixels.

[0151] When the exercise intensity is greater than 6m / s 2 During rapid acceleration, deceleration, or sharp turns, the search radius is set to 10 pixels.

[0152] The exercise intensity is calculated based on the longitudinal acceleration and the lateral acceleration.

[0153] In this example, a neighborhood radius protection mechanism is set. That is, regardless of the acceleration change, the search radius is limited to an upper limit of 10 pixels and a lower limit of 3 pixels. This ensures that the neighborhood set remains within a reasonable range, covering the displacement changes caused by motion without introducing too many irrelevant pixels, thereby ensuring the local accuracy of color compensation.

[0154] In addition, when dynamically switching the search radius, in order to avoid neighborhood discontinuity caused by radius jumps, a radius smoothing update method can be adopted. That is, when the acceleration change between adjacent frames exceeds the preset threshold, the search radius is gradually adjusted at a maximum growth or reduction rate of 2 pixels per frame to smoothly transition the neighborhood size and ensure the temporal stability of the neighborhood set structure and the search consistency.

[0155] S1.4.3: In the neighborhood set, perform a joint sorting based on color similarity and spatial distance, select a preset number of neighborhood pixels, and extract their color shift vectors;

[0156] Specifically, if there are multiple candidate pixels within a neighborhood set, directly using all of them for color compensation can easily introduce noise, especially in edge or dynamic areas, where interference from low-quality pixels can amplify color errors. Therefore, it's necessary to filter and sort the pixels within the neighborhood based on their color characteristics and spatial relationships, prioritizing pixels with similar color characteristics and proximity. This ensures a more accurate color offset vector for subsequent calculations, improving compensation effectiveness.

[0157] In this embodiment, for each target pixel, the Euclidean distance between it and all pixels in the neighborhood set in color space (preferably Lab color space) is calculated as a color similarity measure, and the spatial distance is calculated as a position similarity measure. The color similarity and spatial distance are weighted and fused using a set weighting coefficient to obtain a comprehensive ranking score. According to the comprehensive ranking score, the pixels are sorted in ascending order, and a preset number of the best neighborhood pixels at the top are selected as valid reference pixels.

[0158] To ensure the stability of color compensation, a color anomaly rejection mechanism can be used to screen out extremely abnormal pixel values. This means that neighboring pixels whose color distance exceeds twice the standard deviation of the local mean are removed to prevent extreme outliers from affecting the local color estimation results. At the same time, a color offset vector is calculated for the final selected neighboring pixels. This is the difference vector between the current color value of the target pixel and the color mean of the neighboring pixels, which serves as the basic data for subsequent local compensation.

[0159] S1.4.4: Calculating a local color compensation coefficient based on the color shift vector to generate a quadratic compensation matrix, and compensating the target pixel according to the quadratic compensation matrix until each target pixel is processed;

[0160] In this embodiment, a local color compensation coefficient is calculated for each target pixel based on the color offset vector extracted in the previous stage. This coefficient is determined by the color channel offset and a weighted averaging strategy. Specifically, based on the difference between the color mean of the neighboring pixels and the current pixel color value, the color values ​​of each color channel (such as the L, a, and b channels in Lab space) are adjusted by a certain scaling factor to generate a local quadratic compensation matrix. The compensation matrix is ​​applied to the current target pixel to achieve fine-grained correction of the color value. The specific scaling factor can be determined by those skilled in the art through extensive experimentation.

[0161] See also Figure 4 , which is a schematic diagram of an image quality scoring process provided by an embodiment of the present application. Specifically, in the remote predictive control scenario, due to the spatiotemporal inference, motion compensation and rendering errors when the cloud generates the predicted scene video stream, the most common type of image distortion is not traditional problems such as structural blur and resolution degradation, but visual anomalies mainly characterized by color distortion. This color distortion manifests itself as local hue drift, brightness discontinuity, saturation abnormality, color gamut compression or color gamut expansion, etc. Since the generation mechanism of the predicted scene is different from the direct acquisition of the real picture by the camera, the color rendering is subject to model inference errors and compensation errors, and its distortion law is local, asymmetric and unpredictable. In particular, when the vehicle accelerates, turns or complex lighting changes, the color shift of the predicted image is more obvious. Traditional quality assessment methods based on texture clarity, structural similarity (such as SSIM) or resolution measurement cannot sensitively capture such subtle and local color deviations, and are therefore insufficient to support the requirements of high-reliability and high-precision prediction frame screening. In this embodiment, based on an in-depth analysis of the distortion mechanism of the predicted scene video stream, an image quality assessment strategy dominated by color characteristics is proposed. Specifically, by extracting composite features such as the mean, variance, contrast, edge gradient, and local texture energy of luminance and chrominance components in multiple color spaces, we can fully perceive color changes and distortion patterns, construct frame-level feature vectors, and use this as a basis for quality regression modeling. By focusing on local anomalies and global offsets in the color dimension, we can not only accurately capture subtle distortions in the predicted image, but also improve the ability to identify abnormal predicted frames under complex scene changes, ensuring that high-quality predicted frames that truly meet driving safety requirements are screened out. This method uses color as its core feature and adapts to the typical distortion patterns brought about by the prediction generation mechanism. It is highly targeted and has reliable evaluation results, significantly outperforming traditional evaluation methods that use resolution or structural stability as the main indicators. Figure 4 The method shown can be applied to step S2 of the aforementioned method, and the specific steps are as follows:

[0162] S2.1: Sequentially obtain frames to be evaluated from the enhanced video stream at a preset sampling frequency;

[0163] Specifically, because enhanced video streams are continuously generated, transmission and processing are subject to factors such as bandwidth fluctuations, latency, and prediction errors. Performing quality assessment on every frame would impose a significant computational load and make real-time performance difficult to guarantee. Therefore, it is necessary to selectively sample a subset of frames for evaluation at a reasonable sampling frequency. This ensures consistent overall image quality while balancing processing efficiency and real-time system response speed, thus meeting the low-latency and high-reliability requirements of vehicle remote control.

[0164] In this embodiment, the sampling frequency is set to a multiple of a fixed frame rate, for example, sampling one frame every five. This frequency can be adjusted dynamically based on the transmission link bandwidth and the processing power of the pod. Sequentially acquired frames to be evaluated are arranged by timestamp to ensure that the samples cover the entire video stream, without concentrated sampling or omissions. This results in a temporally representative set of frames to be evaluated, ensuring that subsequent quality assessment effectively reflects the overall video quality trends.

[0165] S2.2: Perform multiple color space conversions on each frame to be evaluated in parallel to obtain RGB, YCbCr, Lab, and HSV representations of the frame to be evaluated;

[0166] Specifically, in vehicle prediction scenarios, a single color space (such as RGB alone) is unable to fully capture the color distortion characteristics in complex scenes. Because predicted images often exhibit saturation anomalies, brightness drift, or color shift in localized areas, and different distortion types exhibit different feature sensitivities in different color spaces, it is necessary to extract multiple color space representations in parallel to enhance the model's ability to perceive multiple types of distortion, thereby improving the comprehensiveness and accuracy of image quality assessment.

[0167] In this embodiment, for each frame to be evaluated, the original RGB representation is first retained, followed by simultaneous conversions to the YCbCr color space to separate luminance and chrominance components, to the Lab color space for perceptual uniformity mapping, and to the HSV color space to enhance responsiveness to changes in hue and saturation. Each color space conversion utilizes standard high-precision conversion formulas to ensure consistency and physical interpretability of color components across different spaces. Parallel conversions utilize asynchronous multi-threaded processing to maximize utilization of the multi-core computing resources onboard the cabin, ensuring no additional latency.

[0168] S2.3: Extract brightness, chroma component mean, chroma component variance, color contrast, edge gradient amplitude, and local texture energy in each color space, and concatenate them to form a frame-level feature vector;

[0169] Specifically, image quality is affected by multiple factors. Single luminance or chrominance metrics cannot fully describe the distortion that may occur in the predicted image. This is especially true in complex driving scenarios, where blurred local details, color shift, and edge information loss are highly likely to occur. Therefore, it is necessary to comprehensively extract luminance, chrominance, and texture features in multiple color spaces and construct high-dimensional frame-level feature vectors to fully capture both global and local image quality variations.

[0170] In this embodiment, the following features are extracted for each of the RGB, YCbCr, Lab, and HSV color spaces: the full-frame mean and variance of the luminance and chrominance channels (reflecting overall brightness and color stability), color contrast (measuring the breadth of the color gamut), edge gradient amplitude (measuring image clarity and detail preservation), and local texture energy (reflecting local structural complexity). Feature extraction uses a combination of local window scanning and global statistics to ensure that both global trends and local anomaly detection are taken into account.

[0171] S2.4: Inputting the frame-level feature vector into an offline-trained nonlinear regression network to obtain a corresponding frame-level quality score;

[0172] Specifically, the quality of the video streams predicted in vehicle remote control scenarios varies in a complex and nonlinear manner. Relying solely on simple linear models makes it difficult to accurately model the mapping relationship between multidimensional features and actual visual quality. Therefore, a nonlinear regression network is required to learn the complex mapping relationship between features and human-perceived quality through large-scale offline training, achieving high-precision frame-level quality prediction.

[0173] In this embodiment, a lightweight multi-layer perceptron (MLP) structure is used as the regression network. The input receives the concatenated frame-level feature vectors. The network is internally configured with two or three fully connected hidden layers, using ReLU as the activation function. The output is mapped to a continuous quality score (e.g., in the range of 0-1). During the network training phase, a manually annotated video quality score database is used as the supervision signal, and optimization is performed using a mean squared error loss to ensure that the trained network can accurately regress the input features.

[0174] Furthermore, to improve the stability and robustness of the model in practical applications, this embodiment introduces data enhancement strategies during the training process, such as color perturbation, local occlusion simulation, and motion blur processing on training samples, to enhance the model's generalization ability for various distortion types, thereby ensuring that reliable quality scores can be output in different cabin scenarios.

[0175] S2.5: Perform temporal exponential smoothing on the frame-level quality score to generate a real-time quality score;

[0176] Specifically, single-frame quality scores are significantly affected by local noise and occasional distortion. Directly using the original score can lead to excessive quality fluctuations in the continuously predicted video stream, affecting the stability of subsequent display stream control. Therefore, it is necessary to perform temporal smoothing on the frame-level quality score to eliminate short-term noise interference and extract the main trend of quality changes, thereby forming a stable and reliable real-time quality score.

[0177] In this embodiment, the exponentially weighted moving average (EWMA) method is used to smooth the continuous frame-level quality scores. The current frame quality score is weighted and superimposed by the smoothed result of the previous frame and the current original score according to a fixed weight. The weight factor is preferably set between 0.7 and 0.9 and is dynamically adjusted according to the needs of the actual scenario, so that the quality score can respond to rapid changes (such as sudden distortion) while effectively suppressing random fluctuations.

[0178] The embodiment of the present application also provides another vehicle scene prediction method. Next, the vehicle scene prediction method is introduced with reference to the accompanying drawings.

[0179] See also Figure 5 , this figure is a flow chart of another vehicle scene prediction method provided in an embodiment of the present application. Figure 5 The method shown can be applied to the cloud of a remote video transmission platform, which also includes a vehicle-mounted terminal and a cabin-mounted terminal. The vehicle-mounted terminal is used to transmit vehicle-side data. The description of the cabin-mounted terminal can refer to the description section above and will not be repeated here. Figure 5 The method shown includes the following A1-A3:

[0180] A1: Receive vehicle-side data and vehicle control data of the current frame, pre-process two consecutive frames of video data in the vehicle-side data to generate a first pre-processed frame and a second pre-processed frame;

[0181] In this embodiment, the vehicle-mounted end uses an image sensor to collect real-time video streams of the vehicle's front environment, and simultaneously collects vehicle posture information, such as longitudinal speed, steering wheel angle, brake opening, and other vehicle control data. After the cloud receives two consecutive frames of vehicle-side video data, it first performs unified resolution standardization on the video data to ensure the consistency of the frame data format collected by different sensors or at different times, and performs color standard space mapping (such as unification to the sRGB color gamut) to avoid subsequent prediction errors caused by color distribution differences. For vehicle control data, numerical normalization and timestamp alignment are performed to ensure a one-to-one correspondence with the video frame. After the above processing, the two standardized frames of video are used as the first pre-processed frame and the second pre-processed frame to provide stable and comparable basic input data for subsequent scene prediction based on optical flow inference.

[0182] A2: Input the pre-processed vehicle-side data and vehicle control data into the preset scene prediction model to generate a video stream of multiple frames of predicted scenes;

[0183] In this embodiment, the standardized first pre-processed frame and the second pre-processed frame, together with the corresponding vehicle control data, are input into the scene prediction model deployed in the cloud. The scene prediction model is based on optical flow inference, and combines the vehicle motion trend (acceleration, steering, braking status) represented by the vehicle control data to perform pixel-level motion vector estimation to construct a motion correlation field between consecutive frames. According to the inferred motion vector field, the model performs reverse resampling and pixel interpolation on the input frame, and generates a predicted scene video stream of several subsequent frames in sequence. By integrating the vehicle control data, the inference deviation of the traditional optical flow algorithm in the case of rapid motion or complex occlusion can be effectively compensated, and the prediction accuracy and temporal consistency can be significantly improved. The predicted video stream generated based on this method can predict the vehicle motion trajectory and the changes in the scene ahead in advance in the cloud, provide continuous and stable visual input for the cabin-side remote control, greatly reduce the real-time inference burden of the cabin-side, and improve the overall system response speed and reliability.

[0184] A3: Synchronize the predicted scene video stream and the original video stream with timestamps, encode and encrypt them, and then send them to the cabin.

[0185] In this embodiment, in order to ensure that the cabin end can accurately and synchronously receive and play the predicted scene video stream generated by the cloud, the cloud adds a standardized timestamp label to each frame according to a strict timing relationship after generating the predicted frame sequence. The timestamp is synchronously corrected based on the original acquisition time of the vehicle end and the cloud processing delay estimate to ensure that the predicted frame playback timing is highly consistent with the actual motion state of the vehicle. After completing the timestamp marking, the entire predicted scene video stream is efficiently encoded (such as H.265 / HEVC encoding format) to reduce the transmission bandwidth overhead, and an end-to-end encryption mechanism is adopted to ensure the integrity and security of the data during transmission. Subsequently, the encoded and encrypted predicted scene video stream is sent to the cabin end via 5G or other high-speed wireless transmission links. Through synchronous marking and encrypted transmission, not only the timing accuracy and continuity of the cabin end screen playback are guaranteed, but also the security of data transmission is improved, preventing potential safety hazards caused by data loss or tampering during remote control.

[0186] The specific steps of A2 are as follows:

[0187] A2.1: Construct a multi-scale pyramid based on the first pre-processed frame and the second pre-processed frame, and traverse the multi-scale pyramid using an optical flow algorithm to generate an initial pixel-level motion vector field.

[0188] Specifically, when a vehicle is in intense motion or the scene depth varies significantly, single-scale optical flow estimation can easily fall into a local optimum, leading to inaccurate pixel-level motion vectors and affecting the overall spatiotemporal consistency of subsequent predicted frames. To address this, a multi-scale pyramid structure is introduced to perform motion estimation in a hierarchical and progressive manner, moving from a large-scale, roughly capturing overall motion trends to a small-scale, more detailed capture of local changes, thus balancing global consistency with local accuracy.

[0189] In this embodiment, a Gaussian pyramid is constructed for each of the first and second preprocessed frames, and the original resolution image is downsampled step by step to form multiple scale levels. Optical flow estimation begins at the top layer (minimum resolution) and uses a sparse matching method to preliminarily estimate the overall large-scale motion trend. Then, layer by layer, relying on the optical flow results of the previous layer as initial values, dense matching technology is used to refine the motion vector at a higher resolution. Within each scale level, optical flow estimation uses a combination of brightness constancy assumptions and local smoothness constraints to reduce noise interference. Finally, after completing all scale level iterations at the lowest level (original resolution), an initial full-frame pixel-level motion vector field is formed.

[0190] Furthermore, to prevent error accumulation during multi-scale transmission, this embodiment introduces a local anomaly detection mechanism between each scale to perform local rematching of high residual areas in the optical flow field, ensuring that abnormal areas can be corrected in a timely manner, thereby improving the stability and accuracy of the overall optical flow estimation.

[0191] A2.2: Calculating a vehicle ego-motion transformation matrix based on the vehicle control data of the current frame, and compensating the initial pixel-level motion vector field based on the vehicle ego-motion transformation matrix to generate a motion vector field;

[0192] Specifically, the optical flow vector field calculated solely through pixel matching between images cannot fully and accurately reflect image changes caused by the vehicle's actual motion, and is particularly prone to mismatching when dynamic objects or road surface changes are present. Therefore, it is necessary to incorporate vehicle egomotion information (i.e., vehicle control data) and calculate the egomotion transformation matrix based on the vehicle's actual motion state within the time interval. This is used to compensate for the initially estimated optical flow field and eliminate or correct systematic deviations caused by pure pixel matching.

[0193] In this embodiment, the vehicle's translation and rotation within the time interval are calculated based on the current frame's longitudinal velocity, steering wheel angle, brake position, and other data, combined with the vehicle's physical model. This is followed by a projection transformation matrix from 3D space to the image plane. This transformation matrix is ​​used to inversely compensate the initial pixel-level motion vector field, removing components caused by vehicle self-motion and retaining only the actual motion components within the scene, generating a compensated motion vector field.

[0194] Furthermore, in order to improve the compensation accuracy, this embodiment introduces a motion blur modeling mechanism, considers the dynamic impact of vehicle acceleration changes on the projection when calculating the self-motion transformation matrix, and dynamically adjusts the compensation vector amplitude and direction to ensure that the self-motion compensation process still has high accuracy in sudden acceleration and sharp turning scenarios.

[0195] A2.3: With the brightness and gradient constrains as the goal, the motion vector field is iteratively optimized in combination with total variation regularization to generate a refined vector field;

[0196] Specifically, even after self-motion compensation, the initial motion vector field may still contain noise, holes, or local errors due to occlusion, illumination changes, and local motion complexity. Direct use of this field can affect the accuracy and consistency of subsequent predicted frames. Therefore, the motion vector field needs to be further optimized, with brightness and gradient constancy as optimization goals. Regularization mechanisms are also introduced to suppress noise and abnormal jumps, generating a smooth and physically plausible refined vector field.

[0197] In this embodiment, a brightness constrain is imposed, requiring that the brightness change of the mapped pixels under the influence of the vector field be minimized. A gradient constrain is also imposed, requiring that the direction and magnitude of the pixel gradients remain as consistent as possible after mapping. Total variation regularization is employed as a smoothing term, penalizing regions of drastic changes in the vector field during optimization to encourage smooth and continuous motion distribution. Overall optimization is achieved using an iterative solver, where each iteration simultaneously updates the vector components and minimizes the loss objective until convergence is achieved, ultimately outputting a refined motion vector field.

[0198] A2.4: Perform inverse warping and resampling on the second pre-processed frame according to the refined vector field to generate a first predicted frame;

[0199] Specifically, in order to infer the scene image at a future moment in time, it is necessary to reversely simulate the pixel distribution of subsequent frames based on the current moment image and its motion trend. Therefore, it is necessary to perform a reverse distortion resampling operation on the second preprocessed frame to generate the first predicted frame as the starting point of the prediction sequence.

[0200] In this embodiment, the future expected position of each pixel in the second preprocessed frame is calculated based on the refined vector field, and the corresponding sampling position is determined using a reverse mapping method. To avoid artifacts caused by simple interpolation, this embodiment preferably uses bilinear interpolation combined with an edge-preserving interpolation strategy to perform a weighted fusion of pixels surrounding the target sampling position, ensuring that the resampling result is continuous and natural in spatial detail. The resulting first predicted frame maintains structural consistency while reasonably reflecting motion trends, serving as the basis for subsequent iterative predictions.

[0201] A2.5: Using the first predicted frame as the current frame, repeatedly perform optical flow estimation, self-motion compensation, vector optimization, and resampling until the target number of predicted frames is generated, forming a predicted frame sequence. This predicted frame sequence is then subjected to forward and backward consistency checking frame by frame. Occlusions are marked and filled using edge-preserving interpolation and block matching inpainting. The predicted scene video stream is then output.

[0202] Specifically, to generate a continuous video stream of predicted future scenes, a single prediction step is insufficient. The first predicted frame must be used as the new current frame, and the optical flow estimation, self-motion compensation, vector optimization, and reverse warping sampling processes must be repeated to form a coherent multi-frame prediction, fully representing the scene changes at several moments in the future. Furthermore, because occluded areas and accumulated errors are easily amplified during the multi-step prediction process, occluded areas must be continuously detected and repaired during the generation process to ensure the integrity and naturalness of the predicted scene.

[0203] In this embodiment, an iterative loop mechanism is adopted, and each time a new prediction frame is generated as input, and steps A2.1 to A2.4 are executed cyclically until a set target number of prediction frames are generated. After each step is generated, forward consistency and backward consistency detection are performed, that is, the consistency of the pixel mapping between two consecutive frames is verified to identify potential occlusion or matching failure areas. For the detected occluded areas, the edge-preserving interpolation method is first applied to flexibly fill the texture details based on the surrounding visible pixel information; when the edge interpolation cannot be effectively repaired, the block matching repair strategy is further introduced to retrieve local blocks from temporally adjacent frames for optimal matching and filling, thereby restoring the reasonable content of the occluded area. Finally, all the prediction frames are integrated to form a complete predicted scene video stream, which has temporal continuity, spatial consistency and color stability, and meets the requirements of cabin-side remote control for continuous and high-quality visual input.

[0204] The embodiment of the present application also provides another vehicle scene prediction method. Next, the vehicle scene prediction method is introduced with reference to the accompanying drawings.

[0205] See also Figure 6 , this figure is a flow chart of another vehicle scene prediction method provided in an embodiment of the present application. Figure 6 The method shown can be applied to the vehicle-mounted end of a remote video transmission platform, which includes multiple cameras. The remote video transmission platform also includes a cloud and a cabin end. The description of the cabin end and the cloud end can refer to the description section above and will not be repeated here. Figure 6 The method shown includes the following Q1-Q2:

[0206] Q1: Send the vehicle-side data collected by the camera and the vehicle control data of the current frame to the cloud;

[0207] Q2: Receive vehicle control data from the cabin.

[0208] Although the embodiments of the present invention have been shown and described above, it will be understood that the above embodiments are illustrative and are not to be construed as limitations on the present invention. A person skilled in the art may change, modify, replace and modify the above embodiments within the scope of the present invention.

Claims

1. A vehicle scene prediction method, applied to the cabin side of a remote video transmission platform, wherein the remote video transmission platform further comprises a vehicle-mounted terminal and a cloud side, wherein the vehicle-mounted terminal is used to transmit vehicle-side data, and the cloud side is used to receive vehicle-side data from the vehicle-mounted terminal and generate a predicted scene video stream, characterized in that: The vehicle scene prediction includes: The predicted scene video stream is received, and color adjustment is performed on the predicted scene video stream to generate an enhanced video stream, wherein the color adjustment includes: Identify target and background areas through semantic segmentation, generate processing frames based on vehicle control data, and perform color distortion processing; Processing the enhanced video stream using a pre-trained image quality assessment model to output a real-time quality score, wherein the image quality assessment model is constructed based on multi-color space feature extraction and a nonlinear regression network; Comparing the real-time quality score with a preset quality threshold, and generating a compensation control signal based on the comparison result to adjust the display queue and generate a continuous display video stream; Vehicle control data is generated according to the display video stream.

2. A vehicle scene prediction method according to claim 1, characterized in that: Performing color adjustment on the predicted scene video stream to generate an enhanced video stream includes: Performing semantic segmentation on the predicted scene video stream to identify the target area and background area; Acquiring vehicle control data of a current frame and a historical frame, adjusting the target area and the background area according to the vehicle control data of the current frame and the historical frame, and generating a first processing frame and a second processing frame; Extracting a color space of the first processed frame, and performing color distortion processing on the color space according to vehicle control data of a current frame to generate a corrected frame; The corrected frame is fused with the second processed frame according to the timestamp to generate an enhanced video stream.

3. A vehicle scene prediction method according to claim 2, characterized in that: Adjusting the target area and the background area according to the vehicle control data of the current frame and the historical frame includes: Predicting a pixel displacement vector of the current frame in the historical frame based on vehicle posture parameters in the historical frame vehicle control data to obtain a first displacement matrix, wherein the vehicle posture parameters include a steering wheel angle, a longitudinal speed, and a brake opening; Performing motion estimation on the first displacement matrix according to the current frame vehicle control data to obtain a second displacement matrix; Fusing the first displacement matrix and the second displacement matrix to obtain a motion trend matrix; Perform pixel-level motion compensation on the target area according to the motion trend matrix to generate a first processing frame; Global affine motion compensation is performed on the background area according to the motion trend matrix to generate a second processed frame.

4. The vehicle scene prediction method according to claim 2, characterized in that: The performing color distortion processing on the color space according to the vehicle control data of the current frame includes: Determine the current frame according to the vehicle control data to obtain a motion phase, wherein the motion phase includes an acceleration phase, a steering phase, and a braking phase; Extracting pixels in a historical target area having a color entropy greater than a preset color threshold from a rolling buffer corresponding to the motion phase to generate a phase reference palette, wherein the rolling buffer is set based on a predicted scene video stream corresponding to a historical frame; Comparing the color space with the phase reference palette channel by channel for deviation to generate a color shift metric; The color space of the first processed frame is graded compensated according to the color shift metric to generate a corrected frame, wherein the graded compensation includes offset cancellation of all frame channels and chroma remapping of pixel blocks.

5. The vehicle scene prediction method according to claim 2, characterized in that: The fusing the corrected frame with the second processed frame according to the timestamp includes: Performing spatiotemporal synchronization on the correction frame and the second processing frame according to the timestamp; For each target pixel in the correction frame, a neighborhood set is constructed at the corresponding position in the second processing frame, wherein the search radius of the neighborhood set is adjusted according to the acceleration in the vehicle control data of the current frame; In the neighborhood set, a preset number of neighborhood pixels are selected and their color offset vectors are extracted by jointly sorting them according to color similarity and spatial distance; A local color compensation coefficient is calculated according to the color shift vector to generate a quadratic compensation matrix, and the target pixel is compensated according to the quadratic compensation matrix until each target pixel is processed.

6. The vehicle scene prediction method according to claim 1, characterized in that: The processing of the enhanced video stream by using a pre-trained image quality assessment model includes: Sequentially acquiring frames to be evaluated from the enhanced video stream at a preset sampling frequency; Perform multiple color space conversions in parallel on each frame to be evaluated to obtain RGB, YCbCr, Lab and HSV representations of the frame to be evaluated; Extract brightness, chroma component mean, chroma component variance, color contrast, edge gradient amplitude and local texture energy in each color space, and splice them to form a frame-level feature vector; Inputting the frame-level feature vector into a nonlinear regression network trained offline to obtain a corresponding frame-level quality score; Temporal exponential smoothing is performed on the frame-level quality scores to generate real-time quality scores.

7. The vehicle scene prediction method according to claim 1, characterized in that: Generating a compensation control signal according to the comparison result includes: If the real-time quality score is less than the preset quality threshold, generating a first compensation control signal, wherein the first compensation control signal marks the current frame as an invalid frame and deletes it from the display queue, and performs motion compensation interpolation between adjacent valid frames to generate a replacement frame; If the real-time quality score is greater than or equal to the preset quality threshold, generating a second compensation control signal, wherein the second compensation control signal inserts the current frame into the display queue as a valid frame; The first compensation control signal or the second compensation control signal is written into a display buffer control register at the cabin end to update a display queue.

8. A vehicle scene prediction method, applied to the cloud side of a remote video transmission platform, wherein the remote video transmission platform further comprises an on-board terminal and a cabin terminal, wherein the on-board terminal is used to transmit vehicle-side data, the cloud side is used to receive vehicle-side data from the on-board terminal and generate a predicted scene video stream, and the cabin terminal is used to receive the predicted scene video stream from the cloud side and perform quality assessment, and generate vehicle control data based on the quality assessment results, characterized in that: The vehicle scene prediction includes: Receiving the vehicle-side data and the vehicle control data of the current frame, preprocessing two consecutive frames of video data in the vehicle-side data to generate a first preprocessed frame and a second preprocessed frame; The pre-processed vehicle-side data and vehicle control data are input into a preset scene prediction model. Based on the temporal characteristics of adjacent video frames and the vehicle motion state represented by the vehicle control information, the pixel-level motion vector is calculated using the optical flow prediction algorithm to generate a predicted scene video stream of multiple frames. The predicted scene video stream and the original video stream are synchronously marked with timestamps, encoded and encrypted, and then sent to the cabin end.

9. The vehicle scene prediction method according to claim 8, characterized in that: The method calculates pixel-level motion vectors based on the temporal characteristics of adjacent video frames and the vehicle motion state represented by the vehicle control information through an optical flow prediction algorithm to generate a plurality of frames of predicted scene video streams, including: Constructing a multi-scale pyramid in the first pre-processed frame and the second pre-processed frame, and traversing the multi-scale pyramid using an optical flow algorithm to generate an initial pixel-level motion vector field; Calculating a vehicle self-motion transformation matrix based on the vehicle control data of the current frame, and compensating the initial pixel-level motion vector field based on the vehicle self-motion transformation matrix to generate a motion vector field; Taking brightness constancy constraint and gradient constancy constraint as the goal, the motion vector field is iteratively optimized in combination with total variation regularization to generate a refined vector field; performing inverse warping and resampling on the second pre-processed frame according to the refined vector field to generate a first predicted frame; Taking the first predicted frame as the current frame, optical flow estimation, self-motion compensation, vector optimization and resampling are performed cyclically until the target number of predicted frames are generated to form a predicted frame sequence. The predicted frame sequence is then subjected to forward and backward consistency detection frame by frame, the occluded areas are marked, and edge-preserving interpolation and block matching are used to repair and fill the occluded holes, and the predicted scene video stream is output.

10. A vehicle scene prediction method according to claim 1, applied to a vehicle-mounted terminal of a remote video transmission platform, wherein the vehicle-mounted terminal includes multiple cameras, and the remote video transmission platform further includes a cloud terminal and a cabin terminal, wherein the cloud terminal is used to receive vehicle-side data from the vehicle-mounted terminal and generate a predicted scene video stream, and the cabin terminal is used to receive the predicted scene video stream from the cloud terminal and perform quality assessment, and generate vehicle control data based on the quality assessment results, characterized in that: The vehicle scene prediction method comprises: Send the vehicle-side data collected by the camera and the vehicle control data of the current frame to the cloud; Receive vehicle control data from the cabin.

Images