Remote control driving method and platform based on double-link redundancy and adaptive coding

Abstract

The invention relates to the technical field of remote control driving, and discloses a remote control driving method and platform based on double-link redundancy and adaptive coding. The method comprises the following steps: obtaining a double-link heterogeneous time sequence frame to construct a reference data pair, and obtaining a geometric descriptor through time sequence prediction, pixel-level subtraction and density clustering; a probe instruction is generated based on the geometric descriptor, physical evidence data are obtained through a vehicle end adaptive coding and transmission mechanism, and a final verification result is obtained through quantitative comparison of the physical evidence data and a physical signature template library; and constructing a frame rendering mechanism, and adjusting the dynamic distortion amplitude to generate a final display picture. According to the method, the problems of space-time consistency collapse and ghosting trailing of prediction compensation are solved, the emergency braking false triggering rate is reduced, perception breakage caused by time delay is repaired, and the safety and reliability of remote control driving are improved.

Description

Technical Field

[0001] This invention relates to remote control driving technology, and more specifically, to a remote control driving method and platform based on dual-link redundancy and adaptive coding. Background Technology

[0002] With the widespread adoption of 5G communication networks, the deepening of vehicle-road cooperative technologies, and the increasing demand for unmanned operation scenarios, end-to-end video latency compensation has become a key bottleneck restricting the industry's development. Traditional solutions relying on AI frame prediction technology have revealed significant shortcomings in practical applications. Existing prediction compensation technologies extrapolate future scenes based on historical frame sequences. While performing adequately in static or uniform speed scenarios, they fail when faced with nonlinear movements such as sudden vehicle braking or pedestrians crossing the road, resulting in visual distortion and interfering with driver judgment. Prediction is essentially a probabilistic extrapolation, while the displayed image must present a deterministic state, causing a collapse of spatiotemporal causality. Furthermore, it cannot distinguish between normal errors and real sudden events, and the presentation of verification information is crude, interfering with the driver's visual focus. How to accurately identify causal breaks, explore the physical nature at low cost, and repair perceptual breaks in a way that conforms to cognitive laws remains a key technical challenge for the industry.

[0003] In the prior art, Chinese patent CN113022653B discloses a remote train driving system and method based on low-latency transmission technology. The system includes a remote driving control module, a remote driving command receiver on the train, and a low-latency transmission network. The remote driving command receiver connects to the TOD (Transmission of Demand) of the onboard control (CC) to obtain local operation interface information and sends it to the control module. The control module transmits the information to the remote driving console for display and simultaneously receives control commands, converts them into control quantities, and issues them. It achieves remote control through the low-latency transmission network, improving the efficiency of fault handling for driverless trains. Chinese patent application CN118094456A discloses a prediction model for factors affecting the performance of driverless networks in urban scenarios. This model is based on LSTM and includes steps such as feature fusion, establishment of factors affecting the performance of driverless networks, construction of a neural network structure, construction of a loss function, neural network training, and simulation experiments for verification. The model optimizes the accuracy of network performance prediction through feature fusion and the neural network model.

[0004] However, while the two existing technologies mentioned above have some value in low-latency transmission and network performance prediction, they fail to address the core pain points of prediction compensation for spatiotemporal consistency collapse, causal break identification, and perception break repair in current remote-controlled driving scenarios. Specifically, patent CN113022653B focuses on latency optimization for remote train control, without addressing AI frame prediction technology. It cannot handle visual distortions caused by nonlinear movements such as sudden braking or pedestrian crossings, and lacks a mechanism to resolve the contradiction between probabilistic prediction and deterministic display. Patent CN118094456A focuses on predicting factors affecting network performance, but does not design a repair scheme for visual distortion problems such as "ghosting" in video prediction frames. It cannot distinguish between prediction errors and real sudden events, and does not consider the interference of verification information on the driver's visual focus. Neither of these technologies establishes a perception repair mechanism that conforms to cognitive laws. They cannot avoid judgment errors caused by prediction distortion, nor can they balance latency compensation and perception accuracy in dynamic scenarios, failing to meet the safety requirements of remote-controlled driving in unmanned operation scenarios. Summary of the Invention

[0005] This invention is applicable to remote-controlled driving scenarios, especially adaptable to dynamic and complex driving environments, and can meet the needs of unmanned operation for latency compensation and perception accuracy. It constructs a baseline data pair through a dual-link redundancy architecture, transforming inherent latency into a unified analysis benchmark. A temporal deviation field is generated by combining time-series prediction and pixel-level subtraction, and geometric descriptors are obtained through density clustering to accurately locate causal break events. Based on the geometric descriptors, probe commands are generated, and combined with vehicle-side adaptive encoding and transmission mechanisms, physical evidence data is efficiently acquired. This data is then quantitatively compared with a physical signature template library to clarify the nature of the causal break event. A frame rendering mechanism applies dynamic distortion effects to the background of the seen frames, adjusting the distortion amplitude based on the similarity in the verification results to generate the final display image. This invention fundamentally resolves the core contradiction between predictive probabilistics and display determinism, avoiding visual distortions such as "ghosting" in remote-controlled driving methods based on dual-link redundancy and adaptive encoding, repairing perception breaks caused by latency, and simultaneously achieving efficient identification and physical nature determination of causal breaks, thus improving the safety and reliability of remote-controlled driving.

[0006] To achieve the above objectives, the present invention provides the following technical solution:

[0007] A remote-controlled driving method based on dual-link redundancy and adaptive coding includes:

[0008] Obtain heterogeneous time-series frames to construct a reference data pair, perform time-series prediction on the reference data pair to obtain the predicted real-time frame representing the theoretical current picture, perform pixel-level subtraction operation on the predicted real-time frame to obtain the time-series deviation field that quantifies the causal break, and perform density clustering analysis on the time-series deviation field to obtain a structured geometric descriptor.

[0009] Logical judgment is performed on the geometric descriptor to obtain probe instructions. An adaptive encoding and transmission mechanism is formulated for the probe instructions to obtain physical evidence data to verify the hypothesis. A physical signature template library for storing typical physical events is constructed. The physical evidence data is quantitatively compared with the physical signature template library to obtain the final verification result confirming the nature of the causal break event.

[0010] Based on the final verification results, a frame rendering mechanism with dynamic distortion effect is created for the driver, and the final distortion intensity used to adjust the frame rendering mechanism is calculated. The frame rendering mechanism is then adjusted according to the final distortion intensity to generate the final display screen used to remind the driver.

[0011] Furthermore, the method for constructing the benchmark data pair includes:

[0012] A synchronization clock is started in the video stream processing software at the cockpit. The dual-link communication architecture of the video stream processing software has two independent communication links, namely the main view link and the probe link.

[0013] Because the configuration goals of the main view link and the probe link are fundamentally different, the total data transmission latency of the two links is different. The frame decoded from the main view link is marked as the seen frame, and the total latency of the main view link is marked as the total latency of the seen frame. The frame obtained from the probe link is marked as the real-time frame, and the total latency of the probe link is marked as the total latency of the real-time frame.

[0014] The deviation between the observed total frame delay and the real-time total frame delay is expressed as the timing deviation;

[0015] The observed frame, the total latency of the observed frame, the real-time frame, the total latency of the real-time frame, and the timing deviation are integrated to obtain a reference data pair.

[0016] Furthermore, the method for constructing the temporal deviation field includes:

[0017] In the video stream processing software at the cockpit, the real-time frames are decoded to obtain a real-time frame image matrix. The real-time frame image matrix is ​​a two-dimensional digital matrix. Each element in the real-time frame image matrix represents a pixel, and each element contains three component values: red, green, and blue.

[0018] The predicted real-time frame is a predicted image output by the video stream processing software. It is a two-dimensional digital matrix with the same size as the real-time frame image matrix, and is represented as the predicted real-time frame image matrix.

[0019] By performing pixel-by-pixel subtraction on the real-time frame image matrix and the predicted real-time frame image using three component values, the real-time frame image matrix and the predicted real-time frame image are converted into single-channel grayscale image matrices, namely the real-time frame grayscale matrix and the predicted real-time frame grayscale matrix. The grayscale image matrix is ​​a two-dimensional matrix, and each pixel represents the grayscale value of the grayscale image matrix at the corresponding spatial position.

[0020] The temporal deviation field is also a single-channel grayscale image matrix. The grayscale value corresponding to each pixel in the temporal deviation field is obtained by the real-time frame grayscale matrix and the predicted real-time frame grayscale matrix.

[0021] Furthermore, the timing deviation field includes:

[0022] The gray value corresponding to the time-series deviation field is obtained by subtracting the gray value of the pixel at the corresponding position in the predicted real-time frame gray matrix from the gray value of the corresponding pixel position in the real-time frame gray matrix and then taking the absolute value.

[0023] If the gray value of a pixel is greater than or equal to 0 and less than or equal to a preset low gray value threshold, the pixel is determined to be in a stable range. If the gray value of a pixel is greater than the low gray value threshold and less than or equal to a preset significant gray value threshold, the pixel is determined to be in a range of interest. If the gray value of a pixel is greater than the significant gray value threshold, the pixel is determined to be in a singularity range.

[0024] Furthermore, the method for obtaining the geometric descriptor includes:

[0025] Only pixels that are determined to be in the singularity interval in the temporal deviation field are selected, and pixels with gray values ​​greater than the significant gray value threshold are integrated into a sample point set;

[0026] Density clustering analysis is performed on the sample point set. The density clustering analysis includes neighborhood and core points. The neighborhood is a circle with the coordinates of each pixel in the temporal deviation field as the center and a preset neighborhood radius as the radius.

[0027] In each neighborhood, if the number of pixels is greater than the preset minimum number of neighbors of the core point, the pixel corresponding to the center of the neighborhood is determined as the core point.

[0028] Traverse all pixels in the sample point set, aggregate the pixels corresponding to the core point and all pixels in the neighborhood of the core point into the same pixel cluster, complete the clustering operation, and finally obtain one or more pixel clusters. Each pixel cluster is defined as a causal singularity region.

[0029] The identified causal singularity regions are geometrically parameterized to generate a structured geometric descriptor, which includes centroid coordinates, minimum bounding rectangle, and shape ratio.

[0030] Furthermore, the probe commands include:

[0031] In the video stream processing software at the cockpit, the geometric descriptor of the causal singularity region is read, and logical judgment is made based on the morphological ratio in the geometric descriptor to generate a structured probe instruction.

[0032] If the read morphology ratio is greater than the preset braking morphology threshold, it indicates that the causal singularity region is presented as a horizontal thin strip due to the sudden deceleration or emergency braking of the moving object. The generated probe instruction is: the problem type identifier is braking hypothesis verification request, and the target region parameter of the probe instruction is set to the parameter in the minimum bounding rectangle in the geometric descriptor. The target region parameter refers to a set of values ​​that can completely surround the causal singularity region.

[0033] If the shape ratio is within the preset square shape range and the centroid coordinates are located in front of the moving object, the problem type identifier of the generated probe command is a turn hypothesis verification request, and the target area parameter of the probe command is set to the parameter in the minimum bounding rectangle in the geometric descriptor.

[0034] Furthermore, the physical evidence data includes:

[0035] When the problem type identifier is a braking hypothesis verification request, in the latest frame image captured in real time by the vehicle camera, a one-dimensional optical flow gradient sampling band is defined according to the target area parameters. The optical flow calculation unit at the vehicle end will calculate the rate of change of all points in the optical flow gradient sampling band along the sampling band direction, i.e., the gradient. The gradient values ​​representing the rate of change of each point are arranged in spatial order on the sampling band to form an ordered one-dimensional gradient floating-point sequence.

[0036] When the problem type identifier is a steering hypothesis verification request, the vehicle-side defines a local motion curl sampling grid based on the target area parameter in the probe instruction, i.e., the minimum bounding rectangle. The optical flow calculation unit on the vehicle-side constructs an optical flow vector field through the motion curl sampling grid and uses the optical flow vector field to calculate the curl within the optical flow vector field grid.

[0037] The gradient floating-point sequence and curl are integrated to obtain physical evidence data.

[0038] Furthermore, the method for obtaining the final verification result includes:

[0039] In the video stream processing software at the cockpit, a physical signature template library is built and dynamically maintained internally. The physical signature template library is a database that stores idealized physical evidence data models corresponding to various typical physical events. Each model in the database is called a physical signature template.

[0040] After the video stream processing software receives the physical evidence data, it iterates through all the templates in the physical signature template library according to the sent question type identifier, matches the question type identifier with the question type identifier of the template, extracts all physical signature templates that match the question type identifier, calculates the similarity between the real-time physical evidence data and each extracted physical signature template, and adopts the physical event represented by the physical signature template with the highest similarity as the verified physical event.

[0041] The adopted and verified physical events are used as event types. Event types and similarity are integrated to obtain a structured final verification result.

[0042] Furthermore, the method for obtaining the final displayed image includes:

[0043] After the rendering engine in the cockpit obtains the final verification result and the minimum bounding rectangle associated with the final verification result, it applies a dynamic distortion effect to the background part of the seen frame according to the event type in the final verification result. The dynamic distortion effect is applied with a preset initial distortion amplitude.

[0044] After obtaining the similarity from the final verification result, the rendering engine in the cockpit calculates the final distortion intensity by combining the similarity with the initial distortion amplitude. After obtaining the final distortion intensity, the rendering engine performs the final compositing operation. The rendering engine merges the background part with the dynamic distortion effect adjusted by the final distortion amplitude with the foreground object in the seen frame that has not been modified, generating the final display screen used to remind the driver.

[0045] A remote-controlled driving platform based on dual-link redundancy and adaptive coding is used to implement the aforementioned remote-controlled driving method based on dual-link redundancy and adaptive coding. The platform includes:

[0046] Singularity identification module: used to acquire heterogeneous time series frames to construct reference data pairs, perform time series prediction on the reference data pairs to obtain the predicted real-time frames that represent the theoretical current picture, perform pixel-level subtraction operation on the predicted real-time frames to obtain the time series deviation field that quantifies the causal break, and perform density clustering analysis on the time series deviation field to obtain a structured geometric descriptor.

[0047] The probe verification module is used to perform logical judgments on the geometric descriptor, obtain probe instructions, formulate vehicle-side adaptive encoding and transmission mechanisms for the probe instructions, obtain physical evidence data to verify the hypothesis, construct a physical signature template library to store typical physical events, and quantitatively compare the physical evidence data with the physical signature template library to obtain the final verification result confirming the nature of the causal break event.

[0048] Image rendering module: Based on the final verification results, it creates a frame rendering mechanism with dynamic distortion effect for the driver, calculates the final distortion intensity for adjusting the frame rendering mechanism, adjusts the frame rendering mechanism according to the final distortion intensity, and generates the final display screen to remind the driver.

[0049] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0050] This invention constructs benchmark data pairs to transform inherent time delays into a unified analytical benchmark. It combines time series prediction and pixel-level subtraction to generate a time series deviation field, and obtains geometric descriptors through density clustering. This enables precise localization of causal break events, addressing the pain points of traditional solutions in distinguishing prediction errors from real sudden events and locating anomalies. Based on the geometric descriptors, probe commands are generated, and combined with vehicle-side adaptive encoding and transmission mechanisms, physical evidence data is extracted on demand. The nature of the event is clarified through quantitative comparison with a physical signature template library, achieving efficient identification of the physical essence of causal break events. This solves the problems of low analysis efficiency and high bandwidth consumption in traditional solutions. The problem is that by applying dynamic distortion effects to the background of the seen frames through a frame rendering mechanism, and adjusting the distortion amplitude according to the similarity in the verification results, the final display image is generated. This fundamentally solves the core contradiction between the predictive probability and the display determinism, avoids visual distortions such as "ghosting" in remote-controlled driving methods based on dual-link redundancy and adaptive coding, repairs the perception gap caused by time delay, and does not interfere with the driver's central vision. This solves the pain points of traditional solutions, such as crude information presentation and easy judgment errors, and comprehensively improves the safety and reliability of remote-controlled driving. Attached Figure Description

[0051] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0052] Figure 1 A flowchart illustrating the remote control driving method based on dual-link redundancy and adaptive coding provided in an embodiment of the present invention;

[0053] Figure 2 A flowchart illustrating the sequence of the timing deviation field provided in an embodiment of the present invention;

[0054] Figure 3 A logic diagram of probe commands provided in an embodiment of the present invention;

[0055] Figure 4This is a functional block diagram of a remote control driving platform based on dual-link redundancy and adaptive coding, provided in an embodiment of the present invention. Detailed Implementation

[0056] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0057] Example 1

[0058] Please see Figure 1 As shown, this embodiment provides a remote-controlled driving method based on dual-link redundancy and adaptive coding, including:

[0059] Step S10: Obtain heterogeneous time-series frames to construct a reference data pair, perform time-series prediction on the reference data pair to obtain a predicted real-time frame representing the theoretical current picture, perform pixel-level subtraction operation on the predicted real-time frame to obtain a time-series deviation field that quantifies causal breakage, and perform density clustering analysis on the time-series deviation field to obtain a structured geometric descriptor.

[0060] Further, step S10 includes:

[0061] Step S11: Obtain the heterogeneous time-series frame construction reference data pair for the dual-link communication architecture.

[0062] In remote-controlled driving practices, the driver's decisions rely entirely on the video feed transmitted from the vehicle. The entire process, from image capture by the vehicle's camera to display on the cockpit screen, inevitably involves end-to-end latency. This latency means the driver always sees a delayed version of the scene. When the remote vehicle is traveling at high speed or the surrounding environment changes rapidly, even a small time difference can cause the driver to act based on past footage, while the command applies to the vehicle's present state. This misalignment between action and perception can easily lead to misjudgments and accidents. Therefore, without adding additional hardware sensors, a baseline data pair is constructed for subsequent causal analysis. This baseline data pair includes the seen frame representing the past and its total latency, the real-time frame representing the present and its total latency, and the temporal deviation between the seen and real-time frames determined by the latency difference. The purpose is to provide a unified, time-dimensional reference system for causal analysis, ensuring the objectivity and repeatability of the analysis process. The visible frames are transmitted to the cockpit via a high-reliability, high-latency main view link and then decoded to obtain video image frames. These represent the past scene that the driver actually sees on the screen, delayed by network transmission. The real-time frames are transmitted to the cockpit via a low-latency probe link, approximating the vehicle's current environmental state as closely as possible in time and representing the current real-time scene. Timing deviation is a time difference that acts as a time scale in the reference data pair.

[0063] The specific construction process of the baseline data pair is as follows: A synchronization clock is started in the video stream processing software at the cockpit. When the video stream processing software simultaneously acquires image data from the same main camera through two independent communication links in the dual-link communication architecture, namely the main field-of-view link and the probe link, the total data transmission latency of the two links is different due to the fundamental difference in the configuration targets of the main field-of-view link and the probe link. The frame decoded from the main field-of-view link is marked as the seen frame, and the total latency of the main field-of-view link is marked as the total latency of the seen frame T1. The frame acquired from the probe link is marked as the real-time frame, and the total latency of the probe link is marked as the total latency of the real-time frame T2. The deviation between the total latency of the seen frame and the total latency of the real-time frame is expressed as the timing deviation T, i.e. The total latency of the seen frame and the total latency of the real-time frame are integrated into a heterogeneous time-series frame in the dual-link communication architecture. The seen frame and the real-time frame represent specific single frames within the total latency of the seen frame and the total latency of the real-time frame, respectively. A synchronization clock provides a unified time reference for all data streams entering the video stream processing software. The main field-of-view link is a communication channel specifically configured in the dual-link communication architecture for transmitting high-quality, high-reliability, but relatively high-latency full-field-of-view video streams. Its purpose is to ensure the driver has a stable and clear main visual image, which is the basis for the driver's routine observation and operation. The probe link is a pre-emptive mode communication channel specifically configured in the dual-link communication architecture for transmitting specific data with the lowest latency, aiming to quickly obtain the vehicle's closest real-time status information. Total latency is a physical quantity representing the actual time elapsed from information generation to reception. Specifically, the configuration goal of the main field-of-view link is to ensure the driver obtains a high-quality, high-reliability main field-of-view image. The vehicle encodes the high-definition video captured by the camera, and the encoded data stream is encapsulated through a standard network protocol stack and transmitted through a 5G channel with quality of service guarantees. The entire transmission process involves complex encoding operations, data packaging, network queuing and jitter, decoding buffering, etc., resulting in a relatively long total latency from the moment the vehicle-mounted camera captures the image to the video stream processing software in the cabin receiving it. The probe link is configured to achieve extremely low latency. When the vehicle receives a data request from the probe link, a simplified encoding scheme is used. For example, only the luminance component of the video frame is extracted and encoded quickly, or uncompressed raw data is transmitted directly when bandwidth is sufficient. This simplified encoded data is given the highest network priority and transmitted through the vehicle-mounted communication unit, minimizing network queuing and retransmissions. Therefore, the total latency of the data on the probe link is extremely short.

[0064] The observed frame, the total latency of the observed frame, the real-time frame, the total latency of the real-time frame, and the timing deviation are integrated to obtain a reference data pair.

[0065] Step S12: Perform time-series prediction on the reference data to obtain the predicted real-time frame representing the theoretical current image.

[0066] In the practice of predictive compensation for remote-controlled driving, simply obtaining past-view frames and present-time frames is insufficient to determine whether the scene changes from the past to the present are reasonable. Therefore, a dynamic model capable of simulating the inertial development of the physical world is constructed to generate a theoretical present view without any unexpected events, i.e., a predicted real-time frame.

[0067] The specific generation process of the predicted real-time frame is as follows: In the video stream processing software at the cockpit, a lightweight time-series prediction AI model is invoked. This AI model is preferably a convolutional long short-term memory (LSTM) network because it can simultaneously capture the spatial features and dynamic changes in the temporal series of an image, and its computational load is smaller than that of large models, thus meeting the real-time requirements of remote-controlled driving. The AI ​​model receives the seen frame as the starting point for prediction and issues a precise instruction to predict forward by one time step. This time step is a time length equal to the temporal deviation. The AI ​​model ultimately outputs a predicted image with the exact same total time dimension as the seen frame, i.e., the predicted real-time frame. This predicted real-time frame is the image deduced by the AI ​​model based on the motion patterns and physical inertia of the scene exhibited by the input seen frame, assuming no unexpected events have occurred, after the temporal deviation. For example, if a car is moving to the right at a constant speed in the total time delay of the seen frame, the AI ​​model will calculate the motion vector, deduce the distance the car should travel within the temporal deviation, and render the car at the corresponding position in the predicted real-time frame.

[0068] Step S13: Perform pixel-level subtraction on the predicted real-time frame to obtain the temporal bias field used to quantify causal break.

[0069] After obtaining the predicted real-time frame representing the theoretical present scene, the difference between the predicted real-time frame and the actual present is still quantified. The magnitude and distribution of this difference directly reflect the degree to which the real world deviates from the physical inertia prediction, and are the core basis for identifying causal break events. Therefore, it is necessary to materialize the abstract causal break into an analyzable temporal deviation field, which is a single-channel grayscale image matrix. The purpose is to quantify the deviation between the real world and the model prediction in two-dimensional space. In the temporal deviation field, the brightness value of each pixel directly corresponds to the degree of difference between the real scene and the predicted scene at the corresponding spatial location.

[0070] Specifically: In the video stream processing software at the cockpit end, real-time frames are decoded to obtain a real-time frame image matrix. This matrix is ​​a two-dimensional digital matrix, where each element represents a pixel. The real-time frame image matrix is ​​a color image, and each element contains three components: red (R), green (G), and blue (B). The predicted real-time frame is the predicted image output by the model, which is itself a two-dimensional digital matrix with the same size as the real-time frame image matrix, i.e., the predicted real-time frame image matrix. Pixel-by-pixel subtraction is performed on both the real-time frame image matrix and the predicted real-time frame image matrix. A weighted average method is used to convert the two color image matrices into a single-channel grayscale image matrix. For example, the three components R, G, and B of the color image are assigned weights of 0.299, 0.587, and 0.114, respectively. The above transformation process is applied to the real-time frame image matrix and the predicted real-time frame image matrix, respectively, to obtain the real-time frame grayscale matrix and the predicted real-time frame grayscale matrix. The real-time frame grayscale matrix is ​​a two-dimensional matrix, where each pixel represents the grayscale value at a corresponding spatial location in the real-time frame. Similarly, the predicted real-time frame grayscale matrix is ​​also a two-dimensional matrix, where each pixel represents the grayscale value at a corresponding spatial location in the predicted real-time frame. The grayscale value of any pixel (i,j) in the temporal deviation field is calculated using the pixel coordinates (a, b) of the real-time frame grayscale matrix and the pixel coordinates (c, d) of the predicted real-time frame grayscale matrix. That is, the grayscale value (i,j) in the temporal deviation field. Where i and j represent the row index and column index of the pixel in the matrix of the temporal deviation field, respectively, and similarly a, b, c, and d are also represented by i and j. The row and column indices of the corresponding pixels in the matrix.

[0071] In the temporal deviation field, the grayscale values ​​of pixels are differentiated, such as... Figure 2 As shown, if the grayscale value of a pixel is greater than or equal to 0 and less than or equal to a preset low grayscale threshold, the pixel is determined to be in a stable range. When the grayscale value of a pixel is equal to 0, it means that the grayscale value of the real-time frame grayscale matrix is ​​exactly equal to the grayscale value of the predicted real-time frame grayscale matrix, implying that the development of the real world is completely consistent with the prediction of the AI ​​model. When the grayscale value of a pixel is greater than 0 and less than or equal to the low grayscale threshold, it indicates a negligible normal deviation caused by sensor thermal noise, slight image compression artifacts, or minor jitter in the AI ​​model prediction. Pixels in the stable range have not undergone any meaningful unexpected changes. The low grayscale threshold is a small integer grayscale value used to distinguish between negligible background noise and potentially meaningful weak signals, obtained based on statistical analysis of the temporal deviation field in static or smooth motion scenes. For example, the low grayscale threshold is set to 5. If the grayscale value of a pixel is greater than the low grayscale threshold and less than or equal to a preset significant grayscale threshold, the pixel is determined to be in a region of interest. The significant grayscale threshold is used to distinguish between a gradual, albeit not yet drastic, change process and a clear, sudden event requiring immediate analysis. For example, the significant grayscale threshold is set to 30. Pixels within the attention interval represent observable but not yet drastic differences between the real and predicted scenes, potentially indicating instability or an ongoing gradual change process. If a pixel's grayscale value is greater than the significant grayscale threshold, the pixel is determined to be within a singularity interval. Pixels within a singularity interval represent significant and drastic differences between the grayscale values ​​of the real and predicted scenes. The higher the pixel's grayscale value, the greater the difference, indicating that the AI ​​model's prediction has completely failed, and a sudden event has occurred in the real world that the AI ​​model could not have anticipated. Examples include the intrusion of something new, emergency braking, or a sharp turn.

[0072] Step S14: Perform density clustering analysis on the time series deviation field to identify causal singularity regions, parameterize the causal singularity regions, and obtain a structured geometric descriptor.

[0073] A threshold filter is applied to the temporal deviation field to select only pixels that are determined to be in the singularity interval, i.e., pixels with gray values ​​greater than the significant gray value threshold. The interference of pixels in the stable interval and the interval of interest is excluded. The pixels with gray values ​​greater than the significant gray value threshold are then integrated into a sample point set.

[0074] Density clustering analysis is performed on the sample point set. This analysis includes neighborhood and core points. The neighborhood defines the area surrounding a pixel, representing the set of pixels whose Euclidean distance from the pixel is no greater than the neighborhood radius. The neighborhood radius is set based on the pixel area occupied by a typical causal break event in the temporal deviation field. For example, the neighborhood radius is set to 3. Spatially, the neighborhood is a circle centered at each pixel's coordinates with a radius equal to the neighborhood radius. Core points are defined as pixels within each neighborhood whose number exceeds a preset minimum number of neighbors; if the number of such pixels exceeds this minimum, the pixel corresponding to the center of the neighborhood is considered a core point. Here, the minimum neighbor count of the core point represents the minimum neighbor density required for a pixel to become a core point, with the aim of filtering out sparse noise points. For example, the minimum neighbor count of the core point is set to 5. Traversing all pixels in the sample point set, pixels that meet the core point condition and all pixels in the neighborhood of the core point are aggregated into the same pixel cluster to complete the clustering operation, and finally one or more pixel clusters are obtained. Each pixel cluster is defined as a causal singularity region. The causal singularity region refers to a connected region in the temporal deviation field, which is composed of a group of spatially adjacent and sufficiently dense pixels located in the singularity interval, representing an independent causal break event.

[0075] The identified causal singularity regions are geometrically parameterized to generate a structured geometric descriptor. This descriptor includes centroid coordinates, a minimum bounding rectangle, and a shape ratio. The centroid coordinates represent the geometric center of the causal singularity region within the image. The minimum bounding rectangle represents the smallest rectangle that completely encloses the causal singularity region, including the coordinates of its top-left corner, width, and height. The shape ratio represents the basic shape characteristics of the causal singularity region. Specifically, the centroid coordinates are obtained by calculating the arithmetic mean of the row and column coordinates of the pixels within the causal singularity region. For example, the row coordinates of the centroid are obtained by adding the row coordinates of all pixels within the causal singularity region and dividing by the total number of pixels; the column coordinates are obtained similarly. The minimum bounding rectangle is determined by selecting the maximum and minimum values ​​of all pixel coordinates within the causal singularity region, i.e., the minimum row coordinate. Maximum row coordinates and the minimum value of column coordinates Maximum column coordinates The coordinates of the top left corner of the rectangle are The width w of the rectangle is calculated using the minimum and maximum values ​​of the column coordinates, i.e. The height H of the rectangle is calculated using the minimum and maximum values ​​of its row coordinates. The formulas for calculating the height and width of the rectangle both include a step of adding 1. This is because, based on the discrete nature of pixel coordinates, the true width and height of the rectangle in discrete space are accurately calculated. The shape ratio is calculated using the width and height of the smallest bounding rectangle, i.e. The geometric descriptor transforms an unstructured pixel region into a set of structured numerical values ​​that describe its location, extent, and shape. The aim is to simplify the complexity of subsequent analysis, enabling rapid and efficient classification and decision-making.

[0076] Step S10, through benchmark data pairs, predicted real-time frames, temporal deviation fields, and geometric descriptors, addresses the initial discovery problem in remote driving: the inherent time delay causes a disconnect between the driver's perception and the vehicle's actual state, and traditional prediction compensation methods fail when faced with unknown abrupt changes, leading to spatiotemporal causality collapse and intuition artifacts. It enables the discovery and precise location of causal break events without relying on any pre-defined observation structure. The construction of benchmark data pairs creatively transforms inherent network delays into an objective time scale, providing a unified reference system for all subsequent analyses. The calculation of the temporal deviation field materializes abstract spatiotemporal differences into an analyzable data field, making any form of unknown anomaly readily apparent. The generation of geometric descriptors successfully transforms unstructured pixel regions into structured data, providing precise and operable targets for subsequent causal probes, serving as the logical starting point for the entire risk identification and analysis process.

[0077] Step S20: Perform logical judgment on the geometric descriptor to obtain probe instructions, formulate vehicle-side adaptive encoding and transmission mechanism for probe instructions, obtain physical evidence data to verify the hypothesis, construct a physical signature template library to store typical physical events, and quantitatively compare the physical evidence data with the physical signature template library to obtain the final verification result confirming the nature of the causal break event.

[0078] Further, step S20 includes:

[0079] Step S21: Perform logical judgment on the geometric descriptor to obtain probe instructions.

[0080] The location and shape of the causal singularity region alone are insufficient to determine the specific physical event behind the accident. For example, a long, narrow singularity region could be caused by a vehicle braking suddenly ahead, or it could be caused by a long, narrow obstacle on the road. Therefore, to ascertain the true physical nature of the event, a probe instruction, a hypothesis with a clear physical orientation, is pre-generated. This probe instruction is a structured instruction designed to guide the vehicle-side to collect evidence data for verifying a specific physical event in the most efficient way. The probe instruction includes a question type identifier and target region parameters. The question type identifier is text or code that informs the vehicle-side which physical hypothesis needs to be verified. The target region parameters represent a set of specific values ​​that completely enclose the causal singularity region.

[0081] The probe command, within the video stream processing software at the cockpit, reads the geometric descriptor of the causal singularity region and performs logical judgments based on the morphological ratios in the geometric descriptor, such as... Figure 3 As shown, a structured probe instruction is generated. Specifically, if the read morphology ratio is greater than the preset braking morphology threshold, it indicates that the causal singularity region is a horizontal, thin strip due to sudden deceleration or emergency braking of the moving object. The generated probe instruction is: the problem type identifier is "Braking Hypothesis Verification Request," and the target region parameter of the probe instruction is set to the parameter in the minimum bounding rectangle of the geometric descriptor. Here, "Braking Hypothesis Verification Request" is an instruction identifier meaning that the vehicle-side is requested to verify whether there is a violent linear deceleration physical phenomenon in the target region. The target region refers to a two-dimensional spatial range uniquely determined by the target region parameter in the real-time image of the vehicle-side camera, aiming to provide the vehicle-side with a precise spatial operating range. The braking morphology threshold is obtained by analyzing the morphology ratio distribution of causal singularities generated in a large number of emergency braking scenarios, selecting a critical value that can effectively distinguish most braking events. For example, the braking morphology threshold is set to 5. If the morphology ratio is within the preset square morphology range and the centroid coordinates are located to the side and front of the moving object, the problem type identifier of the generated probe instruction is "Steering Hypothesis Verification Request," and the target region parameter of the probe instruction is set to the parameter in the minimum bounding rectangle of the geometric descriptor. Among them, the steering hypothesis verification request is an instruction identifier, which means that the vehicle side is requested to verify whether there is a significant rotational motion physical phenomenon in the target area; the square shape interval is a range of values, defined as the set of shape ratios that make the size difference of the causal singularity region in the horizontal and vertical directions insignificant. The purpose is to filter out the causal singularity regions caused by the lateral motion or rotation of the object. For example, the square shape interval is set to 0.7~1.5, that is, the shape ratio is less than or equal to 1.5 and greater than or equal to 0.7.

[0082] Step S22: Establish an adaptive encoding and transmission mechanism for the probe commands on the vehicle side to obtain physical evidence data to verify the hypothesis.

[0083] In order for probe commands to effectively guide the vehicle-mounted system in conducting accurate and efficient probes, the system needs to be able to understand and execute probe commands and transmit key verification information back at the fastest speed with minimal bandwidth overhead. Therefore, a vehicle-mounted adaptive encoding and transmission mechanism was developed. This mechanism transforms the probe link from a transmission channel used only for transmitting real-time frames into a dedicated data channel that can respond to probe commands and actively extract and transmit specific physical evidence data.

[0084] The vehicle-side adaptive encoding and transmission mechanism sends probe commands to the vehicle via the video stream processing software in the cockpit through the uplink, such as the low-bandwidth uplink channel of the main view link or probe link. After receiving the probe commands, the vehicle-side immediately adjusts the encoding and transmission strategy of the probe link, while the main view link maintains its original configuration target and continues to stably transmit high-quality full-view video streams to ensure that the driver's main visual perception is not interrupted by any probing behavior.

[0085] The encoding and transmission strategy of the probe link refers to the encoding method and transmission content of the data sent through the probe link. The adjustment of the encoding and transmission strategy of the probe link is switched according to the problem type identifier in the probe command. When the problem type identifier is a braking hypothesis verification request, the vehicle end no longer performs image encoding on the target area. Instead, in the latest frame image captured in real time by the vehicle camera, a one-dimensional virtual sampling band, i.e., the optical flow gradient sampling band, is defined in front of the minimum bounding rectangle according to the target area parameter in the probe command. For example, the optical flow gradient sampling band can be defined as a rectangular area with a length equal to the width w of the minimum bounding rectangle and a width of 3 pixels, located 10 pixels outward from the front edge of the minimum bounding rectangle. The optical flow calculation unit on the vehicle end quickly calculates the motion vector of all points within the optical flow gradient sampling band, extracts the velocity component of each motion vector in the vehicle's forward direction, and calculates the rate of change of the velocity component along the sampling band direction, i.e., the gradient. The gradient values ​​representing the rate of change of each point are arranged in spatial order on the sampling band to form an ordered one-dimensional gradient floating-point sequence. Ultimately, the probe link no longer transmits any pixel data, but only encodes and transmits this set of one-dimensional gradient floating-point sequences. A sudden braking event will produce a very steep, negatively sized trough in this set of physical evidence data. When the problem type identifier is a steering hypothesis verification request, the vehicle also stops transmitting images. Based on the target region parameter in the probe instruction, i.e., the minimum bounding rectangle, a local two-dimensional virtual sampling grid, i.e., the motion curl sampling grid, is defined with the centroid of the minimum bounding rectangle as the center. For example, the motion curl sampling grid is 20×20 pixels, where the centroid of the minimum bounding rectangle is calculated from the width and height of the minimum bounding rectangle, i.e., the centroid of the minimum bounding rectangle is... The optical flow calculation unit at the vehicle end calculates the motion vector of each pixel within the motion curl sampling grid. This motion vector is used to construct an optical flow vector field covering the entire grid. This optical flow vector field is a two-dimensional set of vectors, where each vector corresponds to a pixel within the grid. The vectors describe the instantaneous direction and velocity of the pixel's motion between two consecutive frames. Using the motion vector data in the optical flow vector field, the curl within the motion curl sampling grid is calculated using the central difference method. The curl, a concept in vector analysis, describes the rotational tendency of the vector field around the pixel. Curl calculation is performed on each pixel within the motion curl sampling grid, resulting in a two-dimensional matrix with the same size as the grid, i.e., the curl scalar matrix. Each element of this matrix represents the curl intensity of the pixel's rotation. Finally, the probe link no longer transmits any pixel data; instead, it only encodes and transmits this curl scalar matrix. A rapid turning action will appear as a region of curl peaks with significantly non-zero values ​​in the curl scalar matrix.

[0086] The gradient floating-point sequence and curl obtained by switching the encoding and transmission strategies of the above probe links according to the problem type identifier in the probe command are integrated to obtain physical evidence data.

[0087] Step S23: Construct a physical signature template library that stores typical physical events. Use a quantification mechanism to compare the physical evidence data with the physical signature template library to obtain the final verification result that confirms the nature of the causal break event.

[0088] In order for physical evidence data to be transformed into a final judgment on physical events, the cockpit needs to have the ability to interpret raw numerical values ​​and assess their degree of conformity with expected assumptions. Therefore, a quantification mechanism is developed, which compares real-time acquired physical evidence data with a preset theoretical model to output a structured final verification result that includes event type and similarity.

[0089] In the video stream processing software at the cockpit, a physical signature template library is built and dynamically maintained internally. This library is a database storing idealized physical evidence data models corresponding to various typical physical events. Each model in the database is called a physical signature template. The physical signature template is a standardized data structure intended as a theoretical benchmark for comparison with real-time acquired physical evidence data. The specific construction process of the physical signature template library is as follows: Vehicle dynamics simulation software simulates various typical physical events of a remotely controlled vehicle under standard test scenarios, ultimately calculating the gradient floating-point number sequence corresponding to each standard braking event. This gradient floating-point number sequence, directly generated from a single standard event simulation, is defined as a physical signature template and stored in the physical signature template library. The same operation is performed on other standard physical events such as steering, thereby constructing a template library containing physical signature templates for multiple events.

[0090] After receiving physical evidence data from the vehicle via the probe link, the video stream processing software in the cockpit traverses all templates in the physical signature template library based on the question type identifier in the probe command sent to the vehicle. It matches the question type identifier with the template's question type identifier. For example, for both steering hypothesis verification requests and braking hypothesis verification requests, all physical signature templates matching the question type identifier are extracted. The similarity between the real-time physical evidence data and each extracted physical signature template is calculated. The physical event represented by the physical signature template with the highest similarity is adopted as the verified physical event. Here, a physical event refers to a definite correspondence between each physical signature template and a standard physical event that serves as the source of its generation when the physical signature template library is constructed. A verified physical event is structured data used to provide a final qualitative description of the dynamic event confirmed through physical law verification. For example, if the template is generated by simulating a -0.8G emergency braking event, then the template represents the -0.8G emergency braking physical event, and this -0.8G emergency braking event type is adopted as the verified physical event confirmed in this verification. The event type is the name or identifier of a verified physical event, with the aim of providing clear qualitative information.

[0091] In the similarity calculation, for a gradient floating-point sequence, the matching cost between the real-time gradient floating-point sequence and the gradient floating-point sequence in the template is calculated using a cost accumulation matrix. For example, assuming the real-time gradient floating-point sequence has N points and the template gradient floating-point sequence has M points, an N x M matrix is ​​created. Each element F(k, y) in the matrix represents the minimum accumulated cost required to align the first k points of the real-time sequence with the first y points of the template sequence, where k and y represent the row and column indices of the matrix, respectively. , The value of element F(k, y) equals the numerical difference between the real-time gradient floating-point sequence and the template gradient floating-point sequence at point (k, y), plus the minimum value among adjacent elements F(k-1, y), F(k, y-1), and F(k-1, y-1). Starting from F(1, 1), the entire matrix is ​​filled, and the value of the final element F(N, M) in the lower right corner is the total cost required for complete alignment of the real-time gradient floating-point sequence and the template gradient floating-point sequence. Similarity is defined as the reciprocal of this total cost; the smaller the total cost, the higher the similarity. For the curl scalar matrix, similarity is calculated through convolution. The smaller template curl scalar matrix is ​​used as a convolution kernel, and a two-dimensional convolution operation is performed on the larger real-time curl matrix to obtain a result matrix. Each element in the result matrix represents the matching response intensity of the template at that position. The maximum matching response intensity is the similarity. The larger the maximum value of the matching response strength, the more likely it is that there is a region in the real-time physical evidence data that is highly consistent with the shape and numerical distribution of the physical signature template.

[0092] The adopted and verified physical events are used as event types. Event types and similarity scores are integrated to obtain a structured final verification result. This final verification result is a data structure containing a qualitative description and quantitative assessment of the current causal break event, namely, the event type and similarity score.

[0093] Step S20, through probe commands, physical evidence data, and final verification results, solves the technical challenge of how to uncover the underlying physical nature of a causal singularity with minimal bandwidth cost and maximum speed. It achieves in-depth analysis, moving from "what's wrong" to "why it's wrong," providing clear event types and reliability evidence for subsequent analysis. The generation of probe commands reduces an open, global analysis problem to a closed, local feature verification problem, significantly improving analysis efficiency. The vehicle-side adaptive encoding and transmission mechanism upgrades the probe link from a passive image transmission channel to an intelligent data channel capable of extracting and transmitting condensed physical evidence on demand, achieving ultimate optimization of communication resources. The output of the final verification results endows the original physical signals with clear semantics and quantified credibility, completing the crucial transformation from "data acquisition" to "final decision-making."

[0094] Step S30: Based on the final verification result, a frame rendering mechanism with dynamic distortion effect is created for the driver, and the final distortion intensity used to adjust the frame rendering mechanism is calculated. The frame rendering mechanism is adjusted according to the final distortion intensity to generate the final display screen used to remind the driver.

[0095] Further, step S30 includes:

[0096] Step S31: Based on the final verification result, create a frame rendering mechanism with dynamic distortion effect for the driver.

[0097] To transform the final verification result from structured data into a visual signal intuitively perceived by the driver, the information needs to be effectively transmitted to the driver while avoiding interference with the driver's central focal vision to prevent intuitive artifacts. Therefore, a frame rendering mechanism is developed. This mechanism is achieved by constructing a dynamic visual tunnel that acts on the driver's peripheral vision. This dynamic visual tunnel is a subtle dynamic distortion effect associated with physical events applied to the background area of ​​the seen frame without modifying foreground objects. The aim is to bypass the driver's central vision for recognition and directly communicate with the driver's peripheral visual system, which is more sensitive to motion and changes in light and shadow, thereby subconsciously guiding the driver's attention and anticipation. Peripheral vision refers to the vast area of ​​the human retina excluding the fovea; this area is extremely sensitive to motion and changes in light and shadow but has weaker ability to recognize details and colors.

[0098] The specific construction process of the frame rendering mechanism is as follows: In the rendering engine at the cockpit end, the rendering engine obtains the final verification result and the minimum bounding rectangle associated with the final verification result. Based on the event type in the final verification result, a dynamic distortion effect is applied to the background portion of the seen frame. This dynamic distortion effect is applied with a preset initial distortion amplitude, which is a dimensionless value representing the basic effect or visibility of the dynamic distortion effect before any similarity information adjustment. The purpose is to ensure that even with the highest similarity, there is still an extremely weak visual effect that only serves as a confirmation at a subconscious level. The initial distortion amplitude is set based on visual perception threshold experiments in human factors engineering to determine a minimum effect amplitude that is neither easily perceived by the driver's central focal vision nor by the peripheral visual system. For example, the initial distortion amplitude is set to 0.05. Specifically, if the event type is confirmed by the physical laws of the braking hypothesis verification request, the rendering engine applies a weak radial blur centered on the centroid of the minimum bounding rectangle on the background image behind the minimum bounding rectangle of the causal singularity region. The subtle radial blur creates a centrifugal visual flow in the driver's peripheral vision. This centrifugal visual flow simulates the visual experience in the real physical world when the relative speed between the driver and an object in front increases sharply, subconsciously inducing the driver to anticipate slowing down or avoiding the obstacle. If the event type is confirmed by the physical laws of the steering hypothesis verification request, the rendering engine determines it as a rotational motion event. For example, assuming the object turns left, the rendering engine applies a slight leftward horizontal motion blur to the background area to the right of the minimum bounding rectangle of the causal singularity region. This subtle horizontal motion blur creates a holistic translational visual flow in the driver's peripheral vision that is consistent with the object's turning direction. This visual flow conforms to the laws of relative motion of distant reference objects observed by the driver when the vehicle turns in the real physical world, thus intuitively conveying the object's turning intention to the driver.

[0099] Step S32: Calculate the final distortion intensity used to adjust the frame rendering mechanism, adjust the frame rendering mechanism according to the final distortion intensity, and generate the final display screen used to remind the driver.

[0100] To ensure that the guidance effect generated by the framework rendering mechanism accurately reflects the reliability of event judgments to the driver and avoids over-guiding the driver under conditions of high uncertainty, the initial distortion amplitude defined in the dynamic distortion effect of the framework rendering mechanism is adjusted based on the similarity in the final verification result, and the final distortion amplitude is calculated. The final distortion amplitude is a dynamically distorted amplitude that is ultimately applied to the rendering after real-time adjustment.

[0101] Specifically: The rendering engine on the cockpit side obtains the similarity from the final verification result, combines it with the initial distortion amplitude, and calculates the final distortion intensity. The formula for the final distortion intensity is: The maximum distortion amplitude represents the maximum visual visibility allowed by the dynamic distortion effect. Based on ergonomic experiments, an upper limit was determined that effectively alerts the driver subconsciously without being overly exaggerated and distracting. For example, the maximum distortion amplitude is set to 0.3. Q represents an adjustment index used to adjust the non-linear relationship between the amplitude and similarity; for example, the adjustment index is set to 2. The adjustment logic of the final distortion intensity formula is as follows: when the similarity is high, the calculated final distortion amplitude is close to the initial distortion amplitude, and the distortion effect is extremely weak, only serving a subconscious confirmation function; when the similarity is low, the calculated final distortion amplitude is close to the maximum distortion amplitude, and the distortion effect is relatively more obvious, subconsciously reminding the driver to pay more attention. After obtaining the final distortion intensity, the rendering engine performs the final compositing operation. The rendering engine merges the background portion with the dynamic distortion effect adjusted by the final distortion amplitude with the foreground object in the seen frame that has not been modified, generating the final display image used to remind the driver.

[0102] The final displayed image is a complete video image presented to the driver after being processed by the frame rendering mechanism. The purpose is to transmit the verification results of causal break events to the driver in a way that conforms to human intuition, without interfering with the driver's perception of key foreground objects, such as other vehicles and pedestrians, through subtle manipulation of the background visual flow. This repairs the perceptual break caused by time delay and guides the driver's subconscious to make predictions.

[0103] Step S30, through a framework rendering mechanism and the final display screen, solves the core human-computer interaction problem of how to present the verification results of sudden events in a way that effectively conveys information without interfering with the driver's flow state, thereby avoiding the interference of intuition artifacts. It achieves non-intrusive guidance and repair of the driver's intuitive model. The construction of dynamic visual tunnels in the framework rendering mechanism, by cleverly manipulating the driver's peripheral vision (which is not easily perceived) rather than the intrusive central vision, pioneers a subconscious communication paradigm based on cognitive psychology. Adjusting the initial distortion amplitude based on similarity transforms "uncertainty" into a perceptible visual guidance amplitude, achieving "honest communication" in human-computer interaction, respecting and stimulating the driver's final judgment. The final display screen, while ensuring the absolute realism of key foreground objects, seamlessly integrates complex causal verification conclusions into the driver's intuitive perception, fundamentally repairing the perceptual break caused by time delay.

[0104] Example 2

[0105] This embodiment, based on Embodiment 1, provides a remote-controlled driving platform based on dual-link redundancy and adaptive coding, such as... Figure 4 As shown, it includes:

[0106] Singularity identification module: used to acquire heterogeneous time series frames to construct reference data pairs, perform time series prediction on the reference data pairs to obtain the predicted real-time frames that represent the theoretical current picture, perform pixel-level subtraction operation on the predicted real-time frames to obtain the time series deviation field that quantifies the causal break, and perform density clustering analysis on the time series deviation field to obtain a structured geometric descriptor.

[0107] The probe verification module is used to perform logical judgments on the geometric descriptor, obtain probe instructions, formulate vehicle-side adaptive encoding and transmission mechanisms for the probe instructions, obtain physical evidence data to verify the hypothesis, construct a physical signature template library to store typical physical events, and quantitatively compare the physical evidence data with the physical signature template library to obtain the final verification result confirming the nature of the causal break event.

[0108] Image rendering module: Based on the final verification results, it creates a frame rendering mechanism with dynamic distortion effect for the driver, calculates the final distortion intensity for adjusting the frame rendering mechanism, adjusts the frame rendering mechanism according to the final distortion intensity, and generates the final display screen to remind the driver.

[0109] In the singularity identification module, the process involves acquiring heterogeneous time-series frames to construct a reference data pair, performing time-series prediction on the reference data pair to obtain a predicted real-time frame representing the theoretical current scene, performing pixel-level subtraction on the predicted real-time frame to obtain a time-series bias field that quantifies causal breakage, and performing density clustering analysis on the time-series bias field to obtain a structured geometric descriptor, including:

[0110] Step S11: Obtain the heterogeneous time-series frame construction reference data pair for the dual-link communication architecture;

[0111] Step S12: Perform time-series prediction on the reference data to obtain the predicted real-time frame representing the theoretical current image.

[0112] Step S13: Perform pixel-level subtraction on the predicted real-time frame to obtain the temporal bias field used to quantify causal breakage.

[0113] Step S14: Perform density clustering analysis on the time series deviation field to identify causal singularity regions, parameterize the causal singularity regions, and obtain a structured geometric descriptor.

[0114] In the probe verification module, logical judgments are performed on the geometric descriptor to obtain probe instructions. An adaptive encoding and transmission mechanism is established for the probe instructions to obtain physical evidence data for verifying the hypothesis. A physical signature template library storing typical physical events is constructed. The physical evidence data is quantitatively compared with the physical signature template library to obtain the final verification result confirming the nature of the causal break event, including:

[0115] Step S21: Perform logical judgment on the geometric descriptor to obtain probe instructions;

[0116] Step S22: Establish an adaptive encoding and transmission mechanism for the probe commands on the vehicle side to obtain physical evidence data to verify the hypothesis;

[0117] Step S23: Construct a physical signature template library that stores typical physical events. Use a quantification mechanism to compare the physical evidence data with the physical signature template library to obtain the final verification result that confirms the nature of the causal break event.

[0118] In the image rendering module, based on the final verification result, a frame rendering mechanism with dynamic distortion effect is created for the driver, and the final distortion intensity used to adjust the frame rendering mechanism is calculated. The frame rendering mechanism is then adjusted according to the final distortion intensity to generate the final display screen used to remind the driver, including:

[0119] Step S31: Based on the final verification result, create a frame rendering mechanism with dynamic distortion effect for the driver;

[0120] Step S32: Calculate the final distortion intensity used to adjust the frame rendering mechanism, adjust the frame rendering mechanism according to the final distortion intensity, and generate the final display screen used to remind the driver.

[0121] The methods and systems of this application may be implemented in many ways. For example, they may be implemented by software, hardware, firmware, or any combination of software, hardware, and firmware. The above-described order of steps for the method is for illustrative purposes only, and the steps of the method of this application are not limited to the order specifically described above, unless otherwise specifically stated.

[0122] In addition, the parts of the technical solutions provided in the embodiments of this application that are consistent with the implementation principles of the corresponding technical solutions in the prior art have not been described in detail, so as to avoid excessive elaboration.

[0123] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above descriptions are merely specific embodiments of the present invention and are not intended to limit the invention. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for remote telepiloting based on dual link redundancy and adaptive coding, characterized in that, The method comprises: obtaining a heterogeneous timing frame construction reference data pair, performing timing prediction on the reference data pair to obtain a predicted real-time frame representing a theoretical present picture, performing pixel-level subtraction operation on the predicted real-time frame to obtain a quantified causal rupture timing deviation field, performing density clustering analysis on the timing deviation field to obtain a structured geometric descriptor; performing logical judgment on the geometric descriptor to obtain a probe instruction, formulating a vehicle-end adaptive coding and transmission mechanism for the probe instruction, obtaining physical evidence data of a verification hypothesis, constructing a physical signature template library storing typical physical events, quantitatively comparing the physical evidence data with the physical signature template library to obtain a final check result confirming the nature of a causal rupture event; based on the final check result, creating a frame rendering mechanism with dynamic distortion effect for the driver, and calculating a final distortion strength for adjusting the frame rendering mechanism, adjusting the frame rendering mechanism according to the final distortion strength, and generating a final display picture for reminding the driver.

2. The method for remote tele-driving based on dual-link redundancy and adaptive coding according to claim 1, characterized in that, The construction method of the reference data pair comprises: starting a synchronous clock in a video stream processing software at the cockpit end, and there are two independent communication links, i.e. a main field of view link and a probe link, in the double-link communication architecture of the video stream processing software; due to the essential difference between the configuration targets of the main field of view link and the probe link, the total time delay of the data transmission of the two links is different, the frame decoded from the main field of view link is marked as a seen frame, the total time delay of the main field of view link is marked as the total time delay of the seen frame, the frame obtained from the probe link is marked as a real-time frame, and the total time delay of the probe link is marked as the total time delay of the real-time frame; the deviation between the total time delay of the seen frame and the total time delay of the real-time frame is expressed as a timing deviation; integrating the seen frame, the total time delay of the seen frame, the real-time frame, the total time delay of the real-time frame and the timing deviation to obtain the reference data pair.

3. The method for remote tele-driving based on dual-link redundancy and adaptive coding according to claim 2, characterized in that, The construction method of the timing deviation field comprises: decoding the real-time frame in the video stream processing software at the cockpit end to obtain a real-time frame image matrix, the real-time frame image matrix is a two-dimensional digital matrix, each element in the real-time frame image matrix represents a pixel point, and each element contains three component values of red, green and blue; the predicted real-time frame is a predicted image output by the video stream processing software, is a two-dimensional digital matrix with the same size as the real-time frame image matrix, and is expressed as a predicted real-time frame image matrix; performing pixel-by-pixel subtraction on the real-time frame image matrix and the predicted real-time frame image through the three component values to convert the real-time frame image matrix and the predicted real-time frame image into single-channel gray image matrices, i.e. a real-time frame gray matrix and a predicted real-time frame gray matrix, the gray image matrix is a two-dimensional matrix, and each pixel point represents the gray value of the gray image matrix at the corresponding spatial position; the timing deviation field is also a single-channel gray image matrix, and the gray value of each pixel point in the timing deviation field is obtained through the real-time frame gray matrix and the predicted real-time frame gray matrix.

4. The method for remote tele-driving based on dual-link redundancy and adaptive coding according to claim 3, characterized in that, The timing deviation field comprises: the gray value corresponding to the timing deviation field is obtained by subtracting the gray value of the corresponding pixel point position in the predicted real-time frame gray matrix from the gray value of the corresponding pixel point position in the real-time frame gray matrix and then taking the absolute value; If the gray value of the pixel point is greater than or equal to 0 and less than or equal to a preset low gray threshold, the pixel point is determined to be in a stable interval; if the gray value of the pixel point is greater than the low gray threshold and less than or equal to a preset significant gray threshold, the pixel point is determined to be in a focus interval; and if the gray value of the pixel point is greater than the significant gray threshold, the pixel point is determined to be in a singularity interval.

5. The method for remote tele-driving based on dual-link redundancy and adaptive coding according to claim 4, characterized in that, The method for obtaining the geometric descriptor comprises: Only the pixel points determined to be in the singularity interval in the time deviation field are selected, and the pixel points with a gray value greater than the significant gray threshold are integrated into a sample point set; Density clustering analysis is performed on the sample point set, the density clustering analysis comprises a neighborhood and a core point, and the neighborhood is a circle with a coordinate of each pixel point in the time deviation field as a center and a preset neighborhood radius as a radius; In each neighborhood, if the number of pixel points is greater than a preset minimum core point neighbor number, the pixel point corresponding to the center of the neighborhood is determined to be a core point; All pixel points in the sample point set are traversed, the pixel point corresponding to the core point and all pixel points in the neighborhood where the core point is located are aggregated into a same pixel cluster, clustering is completed, and one or more pixel clusters are finally obtained, each pixel cluster is defined as a causal singularity region; The identified causal singularity region is geometrically parameterized to generate a structured geometric descriptor, and the geometric descriptor comprises a centroid coordinate, a minimum circumscribed rectangle and a shape ratio.

6. The method for remote tele-driving based on dual-link redundancy and adaptive coding according to claim 5, characterized in that, The probe instruction comprises: In the video stream processing software at the cockpit end, the geometric descriptor of the causal singularity region is read, logical judgment is performed according to the shape ratio in the geometric descriptor, and a structured probe instruction is generated; If the read shape ratio is greater than a preset braking shape threshold, it is indicated that the causal singularity region is a transverse and slender strip due to sudden deceleration or emergency braking of the moving object, and the generated probe instruction is that a problem type identifier is a braking hypothesis verification request, and a target region parameter of the probe instruction is set as a parameter in the minimum circumscribed rectangle in the geometric descriptor, the target region parameter is a group of values capable of completely enclosing the causal singularity region; If the shape ratio is in a preset square shape interval and the centroid coordinate is located at a side front of the moving object, a problem type identifier of the generated probe instruction is a steering hypothesis verification request, and a target region parameter of the probe instruction is set as a parameter in the minimum circumscribed rectangle in the geometric descriptor.

7. The method for remote tele-driving based on dual-link redundancy and adaptive coding according to claim 6, characterized in that, The physical evidence data comprises: When the problem type identifier is the braking hypothesis verification request, a one-dimensional optical flow gradient sampling band is defined in a latest image captured in real time by a vehicle-mounted camera according to the target region parameter, an optical flow calculation unit at the vehicle end calculates a change rate of all points in the optical flow gradient sampling band along a direction of the sampling band, that is, a gradient, gradient values representing the change rates of the points are arranged in a spatial order on the sampling band to form an ordered one-dimensional gradient floating point sequence; When the problem type identifier is a steering assumption verification request, the vehicle end defines a local motion vorticity sampling grid according to the target area parameter in the probe instruction, i.e., the minimum bounding rectangle, and constructs an optical flow vector field through the motion vorticity sampling grid, and calculates the vorticity in the optical flow vector field grid using the optical flow vector field; The gradient float number sequence and the vorticity are integrated to obtain physical evidence data.

8. The method for remote tele-driving based on dual-link redundancy and adaptive coding according to claim 7, characterized in that, The method for obtaining the final verification result comprises: In the video stream processing software at the cockpit end, a physical signature template library is constructed and dynamically maintained inside the software, the physical signature template library is a database storing idealized physical evidence data models corresponding to various typical physical events, and each model in the database is referred to as a physical signature template; When the video stream processing software receives the physical evidence data, the video stream processing software traverses all templates in the physical signature template library according to the problem type identifier sent, matches the problem type identifier with the problem type identifier of the template, extracts all physical signature templates matched with the problem type identifier, and calculates the similarity between the real-time physical evidence data and each extracted physical signature template, and adopts the physical event represented by the physical signature template with the highest similarity as the verified physical event; The adopted verified physical event is taken as an event type, and the event type and the similarity are integrated to obtain a structured final verification result.

9. The method for remote tele-driving based on dual-link redundancy and adaptive coding according to claim 8, characterized in that, The method for obtaining the final display screen comprises: After the rendering engine at the cockpit end obtains the final verification result and the minimum bounding rectangle associated with the final verification result, a dynamic distortion effect is applied to the background part of the seen frame according to the event type in the final verification result, and the dynamic distortion effect is applied at a preset initial distortion amplitude; After the rendering engine at the cockpit end obtains the similarity in the final verification result, the similarity is combined with the initial distortion amplitude to calculate a final distortion strength, and after the rendering engine obtains the final distortion strength, a final composition operation is performed, and the rendering engine fuses the background part with the dynamic distortion effect adjusted by the final distortion amplitude with the foreground object in the seen frame without any modification to generate a final display screen for reminding the driver.

10. Remote telepiloting platform based on dual-link redundancy and adaptive coding for implementing the remote telepiloting method based on dual-link redundancy and adaptive coding according to any one of claims 1 to 9, characterized in that, The platform comprises: a singularity recognition module for obtaining a heterogeneous time sequence frame construction benchmark data pair, performing time sequence prediction on the benchmark data pair to obtain a predicted real-time frame representing a theoretical present picture, performing a pixel-level subtraction operation on the predicted real-time frame to obtain a quantized causal rupture time sequence deviation field, and performing density clustering analysis on the time sequence deviation field to obtain a structured geometric descriptor; a probe verification module for performing logical judgment on the geometric descriptor to obtain a probe instruction, formulating a vehicle end adaptive coding and transmission mechanism for the probe instruction, obtaining physical evidence data of a verification assumption, constructing a physical signature template library storing typical physical events, quantitatively comparing the physical evidence data with the physical signature template library, and obtaining a final verification result confirming the nature of a causal rupture event. The image rendering module is configured to create a frame rendering mechanism with dynamic distortion effect for the driver based on the final check result, calculate a final distortion intensity for adjusting the frame rendering mechanism, adjust the frame rendering mechanism according to the final distortion intensity, and generate a final display picture for reminding the driver.

Images