Vehicle-mounted immersive virtual reality interaction display method based on big data

Abstract

The application discloses a vehicle-mounted immersive virtual reality interactive display method based on big data, relates to the technical field of vehicle-mounted virtual reality, and comprises the following steps: collecting vehicle operation state, driver operation behavior and environment perception data, constructing a driving behavior data set, and establishing a knowledge graph; a time sequence model is used to predict driving intention and probability distribution in a future time window, and a confidence parameter is calculated; candidate scene resources are matched by querying the knowledge graph, priority is determined based on the confidence, and asynchronous caching is performed; when the vehicle enters an associated geographical trigger area or meets a corresponding trigger condition, scene resources are called for real-time rendering, a virtual scene is registered and aligned with a vehicle coordinate system, interactive instructions are generated based on operation behavior to drive content evolution, and VR interactive display content dynamically adapted to a driving situation is generated. The application solves the problems of high scene switching delay and disconnection between content and situation in the prior art, and realizes immersive real-time interaction.

Description

Technical Field

[0001] This invention relates to the field of in-vehicle virtual reality technology, specifically to an in-vehicle immersive virtual reality interactive display method based on big data. Background Technology

[0002] With the rapid development of smart cockpit technology, in-vehicle immersive virtual reality (VR) interactive displays have become an important direction for enhancing the driving experience. They provide passengers with entertainment and information services integrated with the driving context by constructing virtual scenes. Currently, mainstream in-vehicle VR systems mainly use local storage of fixed scene content or cloud-based real-time streaming, switching and displaying scenes based on user manual selection or simple geographical location triggers. These solutions typically collect basic vehicle operating data through onboard sensors and combine it with a pre-set scene resource library for content matching and rendering output.

[0003] However, existing technical solutions have significant shortcomings in practical applications: On the one hand, existing systems mostly adopt passive response mechanisms, which cannot predictively analyze driving intentions based on big data of driving behavior, resulting in high latency and rendering stuttering during VR scene switching, making it difficult to meet the low-latency real-time requirements in the in-vehicle environment; on the other hand, existing scene matching mostly uses static key-value mapping or simple rule bases, lacking the ability to perform multi-dimensional semantic association and dynamic reasoning of driving intentions, environmental features, and user preferences, and cannot adapt to the contextual changes of complex driving situations; in addition, the virtual scene and the vehicle's physical space coordinate system lack a real-time registration mechanism, and an effective interactive loop is not formed between driving operation behavior and VR content, resulting in insufficient immersion and a disconnect in interaction;

[0004] In conclusion, existing in-vehicle VR display technologies struggle to achieve dynamic adaptation between driving scenarios and virtual reality content, predictive preloading of scene resources, and immersive real-time interaction, thus hindering the practicality and user experience of in-vehicle virtual reality systems.

[0005] In view of this, the present invention proposes a method for in-vehicle immersive virtual reality interactive display based on big data. Summary of the Invention

[0006] To address the aforementioned technical issues, this solution provides a big data-based in-vehicle immersive virtual reality interactive display method. This technical solution resolves the problems of high latency during VR scene switching, rendering stuttering, difficulty in meeting the low-latency real-time requirements of in-vehicle environments, inability to adapt to complex driving context changes, insufficient immersion, and fragmented interaction.

[0007] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0008] Big data-based in-vehicle immersive virtual reality interactive display methods include:

[0009] S1. Collect vehicle operating status, driver operation behavior and environmental perception data through the vehicle computing unit, obtain user's historical virtual reality usage data, construct a large dataset of driving behavior, and establish a knowledge graph mapping driving intention, environmental characteristics, user preferences and virtual reality scene resources.

[0010] S2. Based on a large dataset of driving behavior, use a time-series machine learning model to extract behavioral features, predict the types of driving intentions and their probability distributions within a future preset time window, and calculate the corresponding confidence parameters.

[0011] S3. Query the mapping knowledge graph according to the driving intention category, match candidate virtual reality scene resources, and determine the preloading priority based on the confidence parameter;

[0012] S4. Based on the preloading priority, the candidate scene resources are asynchronously transmitted to the storage buffer of the vehicle computing unit;

[0013] S5. Based on the environmental perception data, determine the geographic trigger area associated with the driving intention category. When the vehicle enters the geographic trigger area, or when the vehicle status meets the trigger conditions corresponding to the driving intention category, call the scene resources in the storage buffer for real-time rendering, register and align the virtual scene coordinate system with the vehicle physical space coordinate system, and generate interactive control commands based on the driver's operation behavior to drive the evolution of the virtual scene content, generating immersive virtual reality interactive display content that dynamically adapts to the driving situation.

[0014] Preferably, in step S1, the step of constructing the driving behavior big data set is as follows:

[0015] The system collects vehicle speed, acceleration, steering wheel angle, and brake pedal travel data through the vehicle CAN bus interface; it also collects driver eye movement trajectory, head posture, and hand operation behavior data through onboard sensors; it collects road type, traffic flow, and weather environment data through the environmental perception module; and it acquires users' historical virtual reality usage data.

[0016] The vehicle operating status, driver operation behavior, environmental perception data, and user historical virtual reality usage data are used as multi-source heterogeneous data. These data are then processed by timestamp alignment, outlier removal, and normalization to generate the driving behavior big data set.

[0017] Preferably, in step S1, the steps for establishing a knowledge graph mapping driving intentions, environmental features, user preferences, and virtual reality scene resources are as follows:

[0018] Extract entity nodes and relation edges from a big data set of driving behavior. The entity nodes include driving intention categories, environmental feature parameters, user preference tags, and virtual reality scene resource identifiers. The relation edges represent the semantic association between entity nodes.

[0019] A multidimensional semantic association network is constructed based on relational edges. The multidimensional semantic association network is configured to: respond to contextual changes in the current driving situation, perform multi-hop semantic reasoning to dynamically activate virtual reality scene resource nodes that are related to the driving intention category, environmental features, and user preferences.

[0020] Entity nodes, relation edges, and multidimensional semantic association networks are stored as graph-structured data to form the mapping knowledge graph.

[0021] Preferably, step S2 is as follows:

[0022] The driving behavior dataset is sliced ​​into time sequences according to the preset time window, and vehicle dynamics features, driving operation sequence features and environmental context features are extracted respectively to construct a multi-dimensional behavior feature vector sequence.

[0023] The multi-dimensional behavioral feature vector sequence is input into the time-series machine learning model, and the encoder performs feature compression and the decoder performs probabilistic decoding to output the conditional probability distribution of driving intention categories within a future preset time window.

[0024] The temporal machine learning model includes at least one of temporal convolutional networks, long short-term memory networks, or sequence prediction networks based on attention mechanisms.

[0025] The confidence parameter is calculated by weighted fusion of the information entropy of the conditional probability distribution and the maximum a posteriori probability.

[0026] Preferably, step S3 is as follows:

[0027] Starting with the driving intention category as the starting node, the mapping knowledge graph is traversed, and candidate virtual reality scene resource nodes are activated based on the semantic association path between entity nodes to generate a candidate scene resource set.

[0028] The confidence parameter is compared with a preset grading threshold, which includes a first threshold and a second threshold. When the confidence parameter is greater than or equal to the first threshold, the corresponding candidate scene resource is marked as high priority and full preloading is performed. When the confidence parameter is less than the first threshold but greater than or equal to the second threshold, it is marked as low priority and incremental preloading is performed. The first threshold ranges from 0.75 to 0.90, and the second threshold ranges from 0.40 to 0.60.

[0029] Based on the ratio of the data volume of candidate scene resources to the remaining capacity of the storage buffer, and combined with the remaining duration of the preset time window, the actual preloading sequence of candidate scene resources of each priority is determined.

[0030] Preferably, step S4 is as follows:

[0031] An asynchronous transmission queue is established based on preloading priority. Candidate scene resources marked as high priority are inserted at the head of the asynchronous transmission queue for priority transmission, while candidate scene resources marked as low priority are inserted at the tail for sequential transmission.

[0032] The storage buffer is divided into hierarchical storage areas corresponding to the preload priority. High-priority resources that have been transferred are stored in the fast access area, and low-priority resources are stored in the spare cache area.

[0033] Monitor the transmission completion status of resources in each candidate scenario, generate resource ready identifiers, and release the storage space occupied by ready low-priority resources according to the first-in-first-out principle when the storage capacity of the fast access area or the backup cache area reaches the saturation threshold.

[0034] Preferably, in step S5, when the vehicle enters the geographic trigger area, or the vehicle status meets the trigger condition corresponding to the driving intention category, the scene resources in the storage buffer are invoked for real-time rendering, and the steps are as follows:

[0035] Based on environmental perception data, the current geographical location of the vehicle is obtained, and it is determined whether the vehicle has entered the geographical trigger area corresponding to the driving intention category. The geographical trigger area is an electronic fence area generated with the target path node as the center.

[0036] Based on the vehicle speed and acceleration parameters in the vehicle's operating state, determine whether the vehicle state triggering conditions corresponding to the driving intention category are met.

[0037] When a vehicle enters the geographical trigger area or when the vehicle status triggers a condition, scene resources with resource readiness flags are read from the storage buffer, triggering the real-time rendering process.

[0038] Preferably, in step S5, the registration and alignment of the virtual scene coordinate system with the vehicle physical space coordinate system is carried out as follows:

[0039] Based on vehicle operating status and environmental perception data, a vehicle physical space coordinate system with the vehicle's center of mass as the origin is established.

[0040] Establish a virtual scene coordinate system based on spatial reference information of virtual reality scene resources;

[0041] The virtual scene coordinate system is mapped to the vehicle's physical space coordinate system through coordinate transformation, and the relative pose of the virtual scene coordinate system is updated synchronously based on the real-time changes in the vehicle's operating status.

[0042] Preferably, in step S5, interactive control commands are generated based on the driver's operating behavior to drive the evolution of virtual scene content, generating immersive virtual reality interactive display content that dynamically adapts to the driving situation. The steps are as follows:

[0043] The steering wheel angle parameter in the driver's operation behavior is mapped to the horizontal rotation command of the virtual scene's view, and the accelerator opening and brake pedal travel parameters are mapped to the vertical pitch command of the virtual scene's view.

[0044] The viewing angle of the virtual scene is adjusted in real time according to interactive control commands;

[0045] Based on the adjusted viewing perspective, interactive objects in the virtual scene are driven to perform content evolution corresponding to the driving scenario, generating immersive virtual reality interactive display content.

[0046] Preferably, during the execution of S1 to S5, a security monitoring step is performed in parallel, the security monitoring step including:

[0047] Driver attention state parameters are extracted based on the driver's operating behavior, and vehicle safety state parameters are extracted based on the vehicle's operating status and environmental perception data.

[0048] An attention judgment benchmark is determined based on the driving behavior big data dataset, and a safety judgment benchmark is determined based on the vehicle operating status and the environmental perception data.

[0049] The driver's attention status parameters are compared with the attention judgment benchmark, and the vehicle safety status parameters are compared with the safety judgment benchmark.

[0050] When the driver's attention status parameters deviate from the attention judgment benchmark, or the vehicle safety status parameters deviate from the safety judgment benchmark, a safety interruption command is generated.

[0051] According to the security interruption instruction, the rendering of virtual reality interactive display content is prohibited before the execution of S5, or the output of virtual reality interactive display content is terminated immediately during the execution of S5.

[0052] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0053] This invention proposes a driving intention prediction method based on a temporal machine learning model. It utilizes a large dataset of driving behavior to extract behavioral features and predict the categories and probability distribution of driving intentions within a preset time window, achieving advance quantitative prediction of driving intentions and providing data support for predictive scheduling of scene resources. It proposes a multi-dimensional mapping knowledge graph of driving intentions, environmental features, user preferences, and virtual reality scene resources. Through multi-dimensional semantic association networks, it performs multi-hop semantic reasoning, achieving dynamic activation and accurate matching of scene resources in complex driving situations. It proposes a resource scheduling mechanism based on confidence parameters to determine preloading priorities, combined with asynchronous transmission queues and hierarchical storage areas, achieving predictive preloading and low-latency invocation of scene resources, significantly reducing VR scene switching latency. Finally, it proposes a dual triggering mechanism based on geographical triggering areas and vehicle state triggering conditions from environmental perception data. Combined with real-time registration of the virtual scene and the vehicle's physical space coordinate system and interactive mapping of driving operation behaviors, it achieves immersive interactive display that integrates virtual and real spaces and dynamically adapts to driving situations. Attached Figure Description

[0054] Figure 1 This is a flowchart of the method of the present invention. Detailed Implementation

[0055] The following description is intended to disclose the invention and enable those skilled in the art to implement it. The preferred embodiments described below are merely examples, and other obvious variations will occur to those skilled in the art.

[0056] Reference Figure 1 As shown, the in-vehicle immersive virtual reality interactive display method based on big data includes:

[0057] S1. Collect vehicle operating status, driver operation behavior and environmental perception data through the vehicle computing unit, obtain user's historical virtual reality usage data, construct a large dataset of driving behavior, and establish a knowledge graph mapping driving intention, environmental characteristics, user preferences and virtual reality scene resources.

[0058] In step S1, the steps for constructing the driving behavior big data set are as follows:

[0059] The system collects vehicle speed, acceleration, steering wheel angle, and brake pedal travel data through the vehicle CAN bus interface; it also collects driver eye movement trajectory, head posture, and hand operation behavior data through onboard sensors; it collects road type, traffic flow, and weather environment data through the environmental perception module; and it acquires users' historical virtual reality usage data.

[0060] The vehicle operating status, driver operation behavior, environmental perception data, and user historical virtual reality usage data are used as multi-source heterogeneous data. These data are then processed by timestamp alignment, outlier removal, and normalization to generate the driving behavior big data set.

[0061] In step S1, the steps for establishing a knowledge graph mapping driving intentions, environmental features, user preferences, and virtual reality scene resources are as follows:

[0062] Extract entity nodes and relation edges from a big data set of driving behavior. The entity nodes include driving intention categories, environmental feature parameters, user preference tags, and virtual reality scene resource identifiers. The relation edges represent the semantic association between entity nodes.

[0063] Four types of entity nodes are extracted from the driving behavior dataset:

[0064] Driving Intent Category Nodes: Sliding window clustering (window length 30 seconds, step size 5 seconds) is performed on vehicle speed, acceleration, steering wheel angle, and brake pedal travel data collected from the vehicle's CAN bus. Combined with road segment type labels in the navigation route planning data, an intent label set is generated through K-Means clustering, including categories such as "high-speed cruising", "congested following", "curving", "ramp merging", and "service area approach". Each intent category is assigned a unique node identifier.

[0065] Environmental feature parameter nodes: Structured parameters are extracted from road type, traffic flow and meteorological environmental data collected by the environmental perception module, including {road type: highway / urban road / mountain road / rural road}, {traffic flow level: smooth / slow / congested}, {meteorological conditions: sunny / rain / fog / snow}, {light conditions: day / dusk / night}, and the combination of these parameters forms environmental feature composite nodes;

[0066] User preference tag node: Analyze the scene selection records, single usage duration and active exit frequency in the user's historical virtual reality usage data, extract content type preferences (natural scenery / science fiction / games / movies), interaction intensity preferences (strong interaction / weak interaction / pure viewing), motion sickness sensitivity level (high sensitivity / medium sensitivity / low sensitivity), and form user profile nodes.

[0067] Virtual Reality Scene Resource Identifier Node: Metadata is annotated for scene files in the VR scene resource library to generate unique resource identifiers (such as SC_001 to SC_N). The annotation attributes include scene theme, spatial complexity level (level 1-5), motion intensity level (level 1-5), rendering load level (low / medium / high) and resource data volume.

[0068] Relationship edge extraction: Based on the co-occurrence frequency statistics of historical driving behavior big data sets, three types of relationship edges are extracted:

[0069] Direct mapping edge (intent → scene): Statistically count the frequency of users selecting scene resources under a specific driving intent category. Establish direct mapping edges for intent-scene pairs with a frequency greater than 5%, and initialize the edge weight to the normalized frequency value.

[0070] Environment Modulation Edge (Environment → Scene): Statistics are collected on the adjustment records of rendering parameters of each scene resource under a specific combination of environmental feature parameters (such as automatically reducing scene motion intensity on rainy days). Modulation edges are established between environmental feature nodes and scene resource nodes, with the edge weight being the co-occurrence probability of environment and scene.

[0071] Preference Weighted Edge (Preference → Scene): Statistically analyze the historical matching degree between user preference tags and scene resource identifiers, establish weighted edges between preference nodes and scene nodes, and the edge weight is the user's personalized matching score.

[0072] A multidimensional semantic association network is constructed based on relational edges. The multidimensional semantic association network is configured to: respond to contextual changes in the current driving situation, perform multi-hop semantic reasoning to dynamically activate virtual reality scene resource nodes that are related to the driving intention category, environmental features, and user preferences.

[0073] The execution logic for multi-hop semantic reasoning is as follows:

[0074] When a contextual change in the current driving situation is detected (such as a switch in driving intention category or a jump in environmental feature parameters exceeding a preset threshold), a multi-hop semantic reasoning process is triggered:

[0075] Inference starting point: The current driving intention category node is taken as the starting node, and its 128-dimensional embedding vector is used as the query vector;

[0076] Hop Limit: The maximum number of hops is set to K=3. The first hop traverses from the driving intention node to the directly associated environmental feature node; the second hop traverses from the environmental feature node to the user preference node; the third hop traverses from the user preference node to the virtual reality scene resource node. Paths exceeding 3 hops are not considered to control inference complexity and ensure real-time performance.

[0077] Activation threshold: The path activation probability is defined as the product of the attention coefficients of each edge on the path. The path activation threshold is set to 0.15. Only when the cumulative activation probability of a certain inference path is ≥0.15 is the scene resource node at the end of that path marked as active and included in the candidate scene resource set.

[0078] Path search algorithm: Weighted breadth-first search (WBFS) is used. A priority queue is constructed, with each element representing (current node, cumulative activation probability, current hop count). Starting from the initial node, neighboring nodes are expanded layer by layer, and the cumulative activation probability of each expanded path is calculated, eliminating branches below the activation threshold. For scene resource nodes reached on the 3rd hop, they are sorted in descending order of cumulative activation probability to generate a candidate scene resource set.

[0079] Dynamic update mechanism: After each VR scene display is completed, the consistency feedback between the user's actual selection data and the inference recommendation results is collected. The node embedding vector and attention weight are updated by comparing the learning loss function to achieve online incremental optimization of the knowledge graph.

[0080] Entity nodes, relation edges, and multidimensional semantic association networks are stored as graph-structured data to form the mapping knowledge graph.

[0081] The storage format for graph structure data is as follows:

[0082] The mapping knowledge graph is stored using a property graph model. Nodes are stored in the format {node ID, node type, attribute dictionary}, where the attribute dictionary contains all structured attributes of the entity node. Edges are stored in the format {starting node ID, ending node ID, relation type, edge weight, attention coefficient}. The underlying layer uses a graph database (such as Neo4j or a self-developed adjacency list structure) for persistent storage, while also maintaining a sparse adjacency matrix in memory for real-time inference computation.

[0083] S2. Based on a large dataset of driving behavior, use a time-series machine learning model to extract behavioral features, predict the types of driving intentions and their probability distributions within a future preset time window, and calculate the corresponding confidence parameters.

[0084] Step S2 is as follows:

[0085] The driving behavior dataset is sliced ​​into time sequences according to the preset time window, and vehicle dynamics features, driving operation sequence features and environmental context features are extracted respectively to construct a multi-dimensional behavior feature vector sequence.

[0086] The multi-dimensional behavioral feature vector sequence is input into the time-series machine learning model, and the encoder performs feature compression and the decoder performs probabilistic decoding to output the conditional probability distribution of driving intention categories within a future preset time window.

[0087] The temporal machine learning model includes at least one of temporal convolutional networks, long short-term memory networks, or sequence prediction networks based on attention mechanisms.

[0088] The confidence parameter is calculated by weighted fusion of the information entropy of the conditional probability distribution and the maximum a posteriori probability.

[0089] The training and prediction of a time-series machine learning model are as follows:

[0090] The driving behavior dataset is time-series sliced, with a historical observation window length of 60 seconds, a future prediction window length of 30 seconds (i.e., the preset time window), a step size of 5 seconds, and an overlap rate of 91.7%. Within each observation window, vehicle dynamics features (vehicle speed, acceleration, yaw rate, 4D), driving operation sequence features (steering wheel angle, accelerator / brake pedal opening, 4D), and environmental context features (road type, traffic flow, weather and lighting conditions, 14D) are extracted and concatenated into a single-time-step 22-dimensional vector, forming a 12×22-dimensional multi-dimensional behavioral feature vector sequence.

[0091] A sequence prediction network based on an attention mechanism is used as the temporal machine learning model. The encoder consists of four stacked Transformer layers, each containing an eight-head self-attention mechanism and a 256-dimensional feedforward network, which compresses the input sequence into a 256-dimensional feature vector; the decoder consists of two Transformer layers, which interact with the intent category embedding through a cross-attention mechanism to output decoded features.

[0092] At the end of the decoder, a Softmax layer is used for probabilistic decoding to output the conditional probability distribution P of the driving intention categories in the next 30 seconds, satisfying that the sum of the probabilities of each intention category is 1.

[0093] The confidence parameter is calculated based on the conditional probability distribution: the maximum posterior probability is extracted, and the distribution information entropy is calculated and then normalized to convert it into an information entropy deterministic index; the maximum posterior probability is multiplied by a first weighting coefficient, the information entropy deterministic index is multiplied by a second weighting coefficient, and the products of the two are summed to obtain the confidence parameter, wherein the first weighting coefficient is 0.6 and the second weighting coefficient is 0.4.

[0094] S3. Query the mapping knowledge graph according to the driving intention category, match candidate virtual reality scene resources, and determine the preloading priority based on the confidence parameter;

[0095] Step S3 is as follows:

[0096] Starting with the driving intention category as the starting node, the mapping knowledge graph is traversed, and candidate virtual reality scene resource nodes are activated based on the semantic association path between entity nodes to generate a candidate scene resource set.

[0097] The confidence parameter is compared with a preset grading threshold, which includes a first threshold and a second threshold. When the confidence parameter is greater than or equal to the first threshold, the corresponding candidate scene resource is marked as high priority and full preloading is performed. When the confidence parameter is less than the first threshold but greater than or equal to the second threshold, it is marked as low priority and incremental preloading is performed. The first threshold ranges from 0.75 to 0.90, and the second threshold ranges from 0.40 to 0.60.

[0098] Based on the ratio of the data volume of candidate scene resources to the remaining capacity of the storage buffer, and combined with the remaining duration of the preset time window, the actual preloading sequence of candidate scene resources of each priority is determined.

[0099] The specific criteria for calibrating the preloading priority grading threshold are as follows:

[0100] Based on 12,856 valid driving behavior samples collected by the test fleet over a continuous 6 months, a large dataset of driving behavior was constructed. After being temporally sliced ​​according to a preset time window (set to 30 seconds), a temporal convolutional network was used to extract behavioral features and predict driving intention categories, and output conditional probability distributions.

[0101] The prediction results were backtested: using the actual actions performed by the driver as the truth labels, the prediction accuracy was statistically analyzed under different confidence intervals. The experiment showed that when the confidence parameter was ≥0.85, the prediction accuracy reached 91.3%; when the confidence parameter was in the range of 0.50-0.85, the prediction accuracy was 72.6%; when the confidence parameter was <0.50, the prediction accuracy dropped to 48.1%, close to the level of random guessing.

[0102] Further analysis of the trade-off between preloading success rate and cache utilization: The average data size for full preloading of scene resources was set at 120MB, and the average data size for incremental preloading was set at 35MB. With an onboard computing unit storage buffer capacity of 512MB, if all samples with a confidence level ≥ 0.50 were fully preloaded, the cache overflow probability was 34.7%. If the threshold for full preloading was raised to a confidence level ≥ 0.80, the cache overflow probability decreased to 8.2%, and the rendering readiness rate of high-priority resources reached 94.5%.

[0103] Based on three metrics—prediction accuracy, buffer overflow probability, and rendering readiness—a weighted scoring method was used to determine the tiered thresholds: the first threshold was set to 0.80 (full preloading was performed when confidence level ≥ 0.80), and the second threshold was set to 0.50 (incremental preloading was performed when confidence level < 0.80). Validation was conducted in three typical scenarios: urban roads, highways, and mountain roads. The average scene switching latency under this threshold combination was 87ms, a 62.3% reduction compared to the control group without a tiered preloading mechanism.

[0104] S4. Based on the preloading priority, the candidate scene resources are asynchronously transmitted to the storage buffer of the vehicle computing unit;

[0105] Step S4 is as follows:

[0106] An asynchronous transmission queue is established based on preloading priority. Candidate scene resources marked as high priority are inserted at the head of the asynchronous transmission queue for priority transmission, while candidate scene resources marked as low priority are inserted at the tail for sequential transmission.

[0107] The storage buffer is divided into hierarchical storage areas corresponding to the preload priority. High-priority resources that have been transferred are stored in the fast access area, and low-priority resources are stored in the spare cache area.

[0108] Monitor the transmission completion status of resources in each candidate scenario, generate resource ready identifiers, and release the storage space occupied by ready low-priority resources according to the first-in-first-out principle when the storage capacity of the fast access area or the backup cache area reaches the saturation threshold.

[0109] The hierarchical management of storage buffers is as follows:

[0110] The onboard computing unit is divided into a 512MB storage buffer, which is further divided into hierarchical storage areas according to preload priority. The fast access area occupies 30% (approximately 154MB) and uses a contiguous address mapping method to store high-priority scene resources to reduce random read latency; the spare cache area occupies 70% (approximately 358MB) and uses a paging mapping method to store low-priority scene resources, with dynamic space allocation achieved through page table management.

[0111] The saturation threshold is set to 80% of the total capacity of the tiered storage area. When the used capacity of the fast access area or the backup cache area reaches this saturation threshold, a resource release mechanism is triggered. The saturation threshold is determined based on historical load statistics: the peak occupancy rate of the storage buffer during seven consecutive days of operation is counted, and 90% of the peak occupancy rate is taken as the baseline value of the saturation threshold, so as to reserve 20% of the burst buffer space.

[0112] The first-in-first-out (FIFO) release strategy is executed as follows: a low-priority resource queue is established based on the resource readiness timestamp, and the earliest ready and unused low-priority resources are released first; before release, data integrity verification is performed, and the integrity of the resource file is confirmed by comparing the cyclic redundancy check code. If the verification passes, the storage space is released; if the verification fails, the resource is marked as corrupt and the release is skipped. The queue is then traversed to select the next earliest ready resource until the used capacity drops below the saturation threshold.

[0113] S5. Based on the environmental perception data, determine the geographic trigger area associated with the driving intention category. When the vehicle enters the geographic trigger area, or when the vehicle status meets the trigger conditions corresponding to the driving intention category, call the scene resources in the storage buffer for real-time rendering, register and align the virtual scene coordinate system with the vehicle physical space coordinate system, and generate interactive control commands based on the driver's operation behavior to drive the evolution of the virtual scene content, generating immersive virtual reality interactive display content that dynamically adapts to the driving situation.

[0114] In step S5, when the vehicle enters the geographic trigger area, or the vehicle status meets the trigger condition corresponding to the driving intention category, the scene resources in the storage buffer are invoked for real-time rendering. The steps are as follows:

[0115] Based on environmental perception data, the current geographical location of the vehicle is obtained, and it is determined whether the vehicle has entered the geographical trigger area corresponding to the driving intention category. The geographical trigger area is an electronic fence area generated with the target path node as the center.

[0116] Based on the vehicle speed and acceleration parameters in the vehicle's operating state, determine whether the vehicle state triggering conditions corresponding to the driving intention category are met.

[0117] When a vehicle enters the geographical trigger area or when the vehicle status triggers a condition, scene resources with resource readiness flags are read from the storage buffer, triggering the real-time rendering process.

[0118] In step S5, the virtual scene coordinate system is registered and aligned with the vehicle physical space coordinate system, and the steps are as follows:

[0119] Based on vehicle operating status and environmental perception data, a vehicle physical space coordinate system with the vehicle's center of mass as the origin is established.

[0120] Establish a virtual scene coordinate system based on spatial reference information of virtual reality scene resources;

[0121] The virtual scene coordinate system is mapped to the vehicle's physical space coordinate system through coordinate transformation, and the relative pose of the virtual scene coordinate system is updated synchronously based on the real-time changes in the vehicle's operating status.

[0122] Mathematical implementation of coordinate system registration:

[0123] Based on the attitude angle data in the vehicle's operating state, a vehicle physical space coordinate system is established with the vehicle's center of mass as the origin, with the vehicle's forward direction as the X-axis, the left side as the Y-axis, and the vertical upward direction as the Z-axis, and the scale unit as meters;

[0124] A virtual scene coordinate system is established based on the spatial reference information of virtual reality scene resources, with the geometric center of the scene as the origin and the reference axis initially aligned with the vehicle's physical space coordinate system.

[0125] The virtual scene coordinate system is mapped to the vehicle physical space coordinate system by coordinate transformation: a three-dimensional rotation matrix is ​​constructed based on roll angle, pitch angle and yaw angle, and a translation vector is constructed based on the vehicle center of gravity offset. The coordinates of the virtual scene nodes are transformed by the rotation matrix and superimposed with the translation vector to obtain the mapped coordinates in the vehicle physical space coordinate system.

[0126] The coordinate update cycle is set to be synchronized with the vehicle CAN bus data refresh rate. Every 50 milliseconds, the rotation matrix and translation vector are recalculated based on the latest vehicle attitude angle data to correct the relative pose of the virtual scene coordinate system in real time.

[0127] In step S5, interactive control commands are generated based on the driver's operating behavior to drive the evolution of virtual scene content, generating immersive virtual reality interactive display content that dynamically adapts to the driving situation. The steps are as follows:

[0128] The steering wheel angle parameter in the driver's operation behavior is mapped to the horizontal rotation command of the virtual scene's view, and the accelerator opening and brake pedal travel parameters are mapped to the vertical pitch command of the virtual scene's view.

[0129] The viewing angle of the virtual scene is adjusted in real time according to interactive control commands;

[0130] Based on the adjusted viewing perspective, interactive objects in the virtual scene are driven to perform content evolution corresponding to the driving scenario, generating immersive virtual reality interactive display content.

[0131] The mapping of interactive control commands and the interactive object drivers are as follows:

[0132] The steering wheel angle parameter in the driver's operation behavior is mapped to the horizontal rotation command of the virtual scene: the effective range of the steering wheel angle is set to ±540 degrees, corresponding to the horizontal rotation angular velocity of the virtual camera to ±60 degrees / second, and a linear scaling factor of 0.111 (i.e. 60 / 540) is used for mapping; dead zone compensation is set in the range where the absolute value of the steering wheel angle is less than 5 degrees, and zero rotation command is output in this range to eliminate the view shaking caused by hand tremors;

[0133] The throttle opening and brake pedal travel parameters are mapped to the longitudinal pitch command of the virtual scene's view: the throttle opening percentage of 0%-100% corresponds to a pitch angular velocity of -20 degrees / second to +30 degrees / second, using piecewise linear mapping, with the 0%-50% opening range corresponding to a linear increase from -20 degrees / second to 0 degrees / second, and the 50%-100% opening range corresponding to a linear increase from 0 degrees / second to +30 degrees / second; the brake pedal travel percentage of 0%-100% is directly mapped to a linear decrease from 0 degrees / second to -40 degrees / second, simulating the downward pressure effect of the view during braking;

[0134] Interactive objects in the virtual scene are defined as dynamic environmental elements (such as drifting clouds and flowing road light strips) and contextual response elements (such as taillights of vehicles ahead and roadside signs). Each interactive object has a built-in state machine that executes state transitions after receiving interactive control commands: when the horizontal rotation command lasts for more than 2 seconds, the cloud element changes from a static state to a flowing state, with the flow direction opposite to the view rotation direction; when the vertical pitch command is positive and the throttle opening is greater than 70%, the road light strip element changes from a dim state to an accelerated flowing light state, with the flowing light rate being positively correlated with the pitch angular velocity, realizing real-time linkage between driving operations and the evolution of virtual scene content.

[0135] During the execution of S1 to S5, a security monitoring step is performed in parallel, the security monitoring step including:

[0136] Driver attention state parameters are extracted based on the driver's operating behavior, and vehicle safety state parameters are extracted based on the vehicle's operating status and environmental perception data.

[0137] An attention judgment benchmark is determined based on the driving behavior big data dataset, and a safety judgment benchmark is determined based on the vehicle operating status and the environmental perception data.

[0138] The driver's attention status parameters are compared with the attention judgment benchmark, and the vehicle safety status parameters are compared with the safety judgment benchmark.

[0139] When the driver's attention status parameters deviate from the attention judgment benchmark, or the vehicle safety status parameters deviate from the safety judgment benchmark, a safety interruption command is generated.

[0140] According to the security interruption instruction, the rendering of virtual reality interactive display content is prohibited before the execution of S5, or the output of virtual reality interactive display content is terminated immediately during the execution of S5.

[0141] The criteria for safety monitoring and the interruption mechanism are as follows:

[0142] Based on the aforementioned driving behavior big data dataset, the attention judgment benchmark is statistically determined as follows: the gaze point distribution entropy value and head posture pitch angle and yaw angle data of the driver's eye movement trajectory under historical normal driving conditions are extracted, and their mean and standard deviation are calculated. The mean plus twice the standard deviation is used as the upper limit for judging attention distraction. When the real-time eye movement trajectory entropy value exceeds the upper limit or the abnormal head posture deflection lasts for more than 3 seconds, the driver's attention state parameters are judged to deviate from the attention judgment benchmark.

[0143] Based on the vehicle's operating status and environmental perception data, a safety judgment benchmark is determined: the vehicle speed and acceleration safety thresholds are dynamically adjusted according to the current road type and weather conditions. When the environmental perception data determines that it is rainy / foggy weather or a mountain road, the longitudinal acceleration safety threshold is tightened from 0.3g on a regular road surface to 0.2g, and the lateral acceleration safety threshold is tightened from 0.4g to 0.25g. When the vehicle speed exceeds 80% of the maximum speed limit for the current road type, a deviation judgment of the vehicle safety status parameters is triggered.

[0144] When the driver's attention state parameter deviates from the attention determination benchmark, or the vehicle safety state parameter deviates from the safety determination benchmark, a safety interruption command is generated. The safety interruption command has the highest execution priority, directly preempts the current virtual reality rendering process, prohibits the start of the rendering process before S5 is executed, or immediately freezes the virtual scene coordinate system update and terminates the screen output during the execution of S5. At the same time, the current virtual scene state parameters are saved to a non-volatile storage area. When the vehicle safety state is restored and the driver's attention falls back into the determination benchmark range, the saved state parameters are called to restore the virtual reality interactive display.

[0145] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the claimed invention.

Claims

1. A vehicle-mounted immersive virtual reality interactive display method based on big data, characterized in that, Includes the following steps: S1. Collect vehicle operating status, driver operation behavior and environmental perception data through the vehicle computing unit, obtain user's historical virtual reality usage data, construct a large dataset of driving behavior, and establish a knowledge graph mapping driving intention, environmental characteristics, user preferences and virtual reality scene resources. S2. Based on a large dataset of driving behavior, use a time-series machine learning model to extract behavioral features, predict the types of driving intentions and their probability distributions within a future preset time window, and calculate the corresponding confidence parameters. S3. Query the mapping knowledge graph according to the driving intention category, match candidate virtual reality scene resources, and determine the preloading priority based on the confidence parameter; S4. Based on the preloading priority, the candidate scene resources are asynchronously transmitted to the storage buffer of the vehicle computing unit; S5. Based on the environmental perception data, determine the geographic trigger area associated with the driving intention category. When the vehicle enters the geographic trigger area, or when the vehicle status meets the trigger conditions corresponding to the driving intention category, call the scene resources in the storage buffer for real-time rendering, register and align the virtual scene coordinate system with the vehicle physical space coordinate system, and generate interactive control commands based on the driver's operation behavior to drive the evolution of the virtual scene content, generating immersive virtual reality interactive display content that dynamically adapts to the driving situation.

2. The in-vehicle immersive virtual reality interactive display method based on big data according to claim 1, characterized in that, In step S1, the steps for constructing the driving behavior big data set are as follows: The system collects vehicle speed, acceleration, steering wheel angle, and brake pedal travel data through the vehicle CAN bus interface; it also collects driver eye movement trajectory, head posture, and hand operation behavior data through onboard sensors; it collects road type, traffic flow, and weather environment data through the environmental perception module; and it acquires users' historical virtual reality usage data. The vehicle operating status, driver operation behavior, environmental perception data, and user historical virtual reality usage data are used as multi-source heterogeneous data. These data are then processed by timestamp alignment, outlier removal, and normalization to generate the driving behavior big data set.

3. The in-vehicle immersive virtual reality interactive display method based on big data according to claim 1, characterized in that, In step S1, the steps for establishing a knowledge graph mapping driving intentions, environmental features, user preferences, and virtual reality scene resources are as follows: Extract entity nodes and relation edges from a big data set of driving behavior. The entity nodes include driving intention categories, environmental feature parameters, user preference tags, and virtual reality scene resource identifiers. The relation edges represent the semantic association between entity nodes. A multidimensional semantic association network is constructed based on relational edges. The multidimensional semantic association network is configured to: respond to contextual changes in the current driving situation, perform multi-hop semantic reasoning to dynamically activate virtual reality scene resource nodes that are related to the driving intention category, environmental features, and user preferences. Entity nodes, relation edges, and multidimensional semantic association networks are stored as graph-structured data to form the mapping knowledge graph.

4. The in-vehicle immersive virtual reality interactive display method based on big data according to claim 1, characterized in that, Step S2 is as follows: The driving behavior dataset is sliced ​​into time sequences according to the preset time window, and vehicle dynamics features, driving operation sequence features and environmental context features are extracted respectively to construct a multi-dimensional behavior feature vector sequence. The multi-dimensional behavioral feature vector sequence is input into the time-series machine learning model, and the encoder performs feature compression and the decoder performs probabilistic decoding to output the conditional probability distribution of driving intention categories within a future preset time window. The temporal machine learning model includes at least one of temporal convolutional networks, long short-term memory networks, or sequence prediction networks based on attention mechanisms. The confidence parameter is calculated by weighted fusion of the information entropy of the conditional probability distribution and the maximum a posteriori probability.

5. The in-vehicle immersive virtual reality interactive display method based on big data according to claim 1, characterized in that, Step S3 is as follows: Starting with the driving intention category as the starting node, the mapping knowledge graph is traversed, and candidate virtual reality scene resource nodes are activated based on the semantic association path between entity nodes to generate a candidate scene resource set. The confidence parameter is compared with a preset grading threshold, which includes a first threshold and a second threshold. When the confidence parameter is greater than or equal to the first threshold, the corresponding candidate scene resource is marked as high priority and full preloading is performed. When the confidence parameter is less than the first threshold but greater than or equal to the second threshold, it is marked as low priority and incremental preloading is performed. The first threshold ranges from 0.75 to 0.90, and the second threshold ranges from 0.40 to 0.

60. Based on the ratio of the data volume of candidate scene resources to the remaining capacity of the storage buffer, and combined with the remaining duration of the preset time window, the actual preloading sequence of candidate scene resources of each priority is determined.

6. The in-vehicle immersive virtual reality interactive display method based on big data according to claim 1, characterized in that, Step S4 is as follows: An asynchronous transmission queue is established based on preloading priority. Candidate scene resources marked as high priority are inserted at the head of the asynchronous transmission queue for priority transmission, while candidate scene resources marked as low priority are inserted at the tail for sequential transmission. The storage buffer is divided into hierarchical storage areas corresponding to the preload priority. High-priority resources that have been transferred are stored in the fast access area, and low-priority resources are stored in the spare cache area. Monitor the transmission completion status of resources in each candidate scenario, generate resource ready identifiers, and release the storage space occupied by ready low-priority resources according to the first-in-first-out principle when the storage capacity of the fast access area or the backup cache area reaches the saturation threshold.

7. The in-vehicle immersive virtual reality interactive display method based on big data according to claim 6, characterized in that, In step S5, when the vehicle enters the geographic trigger area, or the vehicle status meets the trigger condition corresponding to the driving intention category, the scene resources in the storage buffer are invoked for real-time rendering. The steps are as follows: Based on environmental perception data, the current geographical location of the vehicle is obtained, and it is determined whether the vehicle has entered the geographical trigger area corresponding to the driving intention category. The geographical trigger area is an electronic fence area generated with the target path node as the center. Based on the vehicle speed and acceleration parameters in the vehicle's operating state, determine whether the vehicle state triggering conditions corresponding to the driving intention category are met. When a vehicle enters the geographical trigger area or when the vehicle status triggers a condition, scene resources with resource readiness flags are read from the storage buffer, triggering the real-time rendering process.

8. The in-vehicle immersive virtual reality interactive display method based on big data according to claim 1, characterized in that, In step S5, the virtual scene coordinate system is registered and aligned with the vehicle physical space coordinate system, and the steps are as follows: Based on vehicle operating status and environmental perception data, a vehicle physical space coordinate system with the vehicle's center of mass as the origin is established. Establish a virtual scene coordinate system based on spatial reference information of virtual reality scene resources; The virtual scene coordinate system is mapped to the vehicle's physical space coordinate system through coordinate transformation, and the relative pose of the virtual scene coordinate system is updated synchronously based on the real-time changes in the vehicle's operating status.

9. The in-vehicle immersive virtual reality interactive display method based on big data according to claim 1, characterized in that, In step S5, the process of generating interactive control commands based on driver operation behavior to drive the evolution of virtual scene content and generate immersive virtual reality interactive display content that dynamically adapts to the driving situation is as follows: The steering wheel angle parameter in the driver's operation behavior is mapped to the horizontal rotation command of the virtual scene's view, and the accelerator opening and brake pedal travel parameters are mapped to the vertical pitch command of the virtual scene's view. The viewing angle of the virtual scene is adjusted in real time according to interactive control commands; Based on the adjusted viewing perspective, interactive objects in the virtual scene are driven to perform content evolution corresponding to the driving scenario, generating immersive virtual reality interactive display content.

10. The in-vehicle immersive virtual reality interactive display method based on big data according to claim 1, characterized in that, During the execution of S1 to S5, a security monitoring step is performed in parallel, the security monitoring step including: Driver attention state parameters are extracted based on the driver's operating behavior, and vehicle safety state parameters are extracted based on the vehicle's operating status and environmental perception data. An attention judgment benchmark is determined based on the driving behavior big data dataset, and a safety judgment benchmark is determined based on the vehicle operating status and the environmental perception data. The driver's attention status parameters are compared with the attention judgment benchmark, and the vehicle safety status parameters are compared with the safety judgment benchmark. When the driver's attention status parameters deviate from the attention judgment benchmark, or the vehicle safety status parameters deviate from the safety judgment benchmark, a safety interruption command is generated. According to the security interruption instruction, the rendering of virtual reality interactive display content is prohibited before the execution of S5, or the output of virtual reality interactive display content is terminated immediately during the execution of S5.

Images