Vehicle motion situation visualization method and device

Abstract

The invention relates to a vehicle motion situation visualization method and device, and belongs to the crossing field of man-machine interaction and intelligent traffic systems, and the method comprises the steps: responding to an automatic driving instruction, obtaining the real-time driving information of a vehicle, and obtaining the motion situation of the vehicle according to the historical driving data and the real-time driving information of the vehicle; predicting the motion conditions of the vehicle and the pre-appearing objects around the vehicle to obtain a multi-target motion prediction result, and performing trajectory alignment processing on the multi-target motion prediction result based on a real-time driving path in the real-time planning decision data to obtain a future time sequence trajectory layer; and performing visual display on the future time sequence track layer according to a preset visual display mode, and synchronously displaying the real-time driving path. Through the method disclosed by the invention, the transparency and credibility of automatic driving are improved, takeover misoperation caused by information lag is reduced, and the driving safety is enhanced.

Description

Technical Field

[0001] This disclosure relates to the intersection of human-computer interaction and intelligent transportation systems. Specifically, this disclosure relates to a method and apparatus for visualizing the motion status of vehicles. Background Technology

[0002] After completing environmental perception, path planning, and motion control, existing autonomous driving systems typically need to feed the results back to the driver or a remote monitoring terminal. Limited by display methods, current mainstream solutions only provide raw data such as the vehicle's instantaneous position, speed, and acceleration, or at most a static planned path. The future motion trends of surrounding vehicles, pedestrians, and other targets are still presented as lists, icons, or brief text prompts, lacking both continuous representation over time and the ability to intuitively compare them with the vehicle's planned path on the same screen. This fragmented and low-dimensional information output method forces drivers to integrate multi-source data and deduce future scenarios during supervision or takeover, increasing cognitive load and reaction delay, which is detrimental to driving safety. Summary of the Invention

[0003] The technical problem to be solved by this disclosure is to provide a method and apparatus for visualizing the motion status of vehicles, aiming to solve at least one of the above-mentioned technical problems.

[0004] In a first aspect, this disclosure provides a method for visualizing the motion state of a vehicle to solve the aforementioned technical problems. The method includes: In response to autonomous driving commands, the system acquires real-time driving information of the vehicle, including real-time vehicle data, real-time road condition data, and real-time planning and decision-making data. Based on the vehicle's historical driving data and real-time driving information, the movement of the vehicle and the objects that are expected to appear around the vehicle is predicted, and multi-target motion prediction results are obtained. The multi-target motion prediction results include the vehicle's trajectory at the current moment and the motion state information of the objects that are expected to appear within the vehicle's target range. Based on the real-time driving path in the real-time planning and decision data, the trajectory alignment processing of the multi-target motion prediction results is performed to obtain the future time series trajectory layer. The future time series trajectory layer is used to characterize the position sequence and motion trend of the vehicle and the pre-appearing object in the future preset time period. The future time-series trajectory layer is visualized according to the preset visualization display method, and the real-time driving path is displayed simultaneously.

[0005] The beneficial effects of this disclosure are as follows: Based on acquiring three types of information—real-time vehicle data, real-time traffic data, and real-time planning and decision-making data—historical driving data is further introduced to predict the multi-target motion of vehicles and anticipated objects. Subsequently, using the real-time driving path in the real-time planning and decision-making data as a unified benchmark, spatiotemporal alignment is performed on each predicted future trajectory to generate a future time-series trajectory layer. Finally, this layer is visualized synchronously with the real-time driving path. Through the combination of the above features, the originally scattered instantaneous perception results are transformed into a continuous trajectory sequence within the same spatiotemporal framework. Drivers only need to observe the overlay image once to simultaneously obtain the planned direction of their own vehicle and the future movements of surrounding targets, eliminating the need for manual integration of multi-source information or mental deduction. This reduces cognitive load, shortens judgment time, and reduces the risk of misoperation due to trajectory misunderstanding.

[0006] Based on the above technical solution, the present disclosure can be further improved as follows.

[0007] Furthermore, the above-mentioned visualization of the future time-series trajectory layer according to the preset visualization display method includes: By using augmented reality (AR) devices and simultaneous localization and mapping (SLAM) technology, an AR coordinate system is constructed in the real scene view; Transform the future time-series trajectory layer from the world coordinate system to the AR coordinate system to obtain spatially aligned time-series trajectory data; In the AR coordinate system, different types of traffic participants represented by time-series trajectory data are rendered with corresponding and distinctive visual icons and overlaid on the real scene view containing the real-time driving path.

[0008] The beneficial effect of adopting the above-mentioned further solution is that by constructing a coordinate system that is strictly aligned with the real scene through AR devices + SLAM, and rendering and overlaying the future trajectory layer directly onto the real scene view with differentiated visual icons according to object type, users can see the vivid trajectory and the outline of traffic participants moving with the scene without changing their perspective, realizing WYSIWYG augmented reality navigation, which greatly improves information intuitiveness and driving trust.

[0009] Furthermore, in the AR coordinate system, different types of traffic participants represented by the time-series trajectory data are rendered with corresponding and distinctive visual icons, and overlaid on the real-world scene view containing the real-time driving path, including: Based on the time offset of each trajectory point in the time-series trajectory data relative to the current time, the dynamically decreasing transparency coefficient corresponding to each trajectory point in the time-series trajectory data is determined according to the preset transparency-time decay function. By associating each dynamically decreasing transparency coefficient with its corresponding trajectory point, enhanced trajectory data with dynamic transparency attributes is obtained. The enhanced trajectory data with dynamic transparency properties is rendered as a semi-transparent layer and overlaid on the real-world scene view that contains the real-time driving path displayed on the current page. The dynamically decreasing transparency coefficient is used to visually represent the confidence decay of trajectory prediction.

[0010] The beneficial effect of adopting the above-mentioned further solution is that by using the "transparency-time decay function" to assign dynamic transparency that decreases over time to each point of the future trajectory, the distant trajectory gradually disappears and the nearby trajectory becomes brighter. Users can directly perceive the decrease in prediction confidence over time from the visual brightness at a glance, avoiding the mistake of taking the low-confidence trajectory in the distant future as a high-confidence reference, thus improving the safety of decision-making and the readability of interface information.

[0011] Furthermore, after predicting the motion of the vehicle and surrounding objects based on historical driving data and real-time driving information to obtain multi-target motion prediction results, the above method also includes: Based on the multi-target motion prediction results, determine whether the pre-appearing object appears within the vehicle's preset range; When it is determined that a pre-occurring object appears within the preset range of the vehicle, decision-making process information is generated based on the motion state information of the pre-occurring object. Visualize the decision-making process information to obtain visualized decision-making process information; Based on a pre-defined visualization method, information about the visual decision-making process is displayed through a combination of a graphical user interface and voice prompts. The decision-making process information includes the current decision, the basis for the current decision, alternative decisions, the reasons for the alternative decisions, the final decision, and the reasons for the final decision.

[0012] The beneficial effect of adopting the above-mentioned further solution is that after trajectory prediction, it can determine in real time whether the pre-occurring object appears within the preset range. Once it appears, it can immediately generate and visualize the complete decision-making process information of "current-alternative-final", which is presented in a graphical and voice manner. Through the decision-making process information, users not only know what the system should do, but also understand why it should do so and why other options were abandoned. This significantly enhances the explainability of autonomous driving and reduces trust anxiety and accidental takeover caused by the black box of decision-making.

[0013] Furthermore, the above-mentioned visualization processing of decision-making process information yields visualized decision-making process information, including: Decision tree flowcharts and / or decision heatmaps are generated based on decision process information. The decision tree flowcharts and / or decision heatmaps include priority weights, risk scores, and key scenario parameters and condition thresholds that trigger decisions. Access the cloud database to obtain decision statistics of other vehicles in similar historical scenarios, and generate statistical comparison charts based on the decision statistics. The statistical comparison charts include the statistical comparison results of planning decision data and historical decision sets. Based on the decision tree flowchart and / or decision heatmap and statistical comparison chart, determine the information for visualizing the decision process.

[0014] The beneficial effect of adopting the above-mentioned further solution is that by parsing the decision-making process into a decision tree / decision heatmap and overlaying it with a statistical comparison chart of historical success rates and accident rates of similar scenarios in the cloud, users can intuitively see the relative position of the current decision in risk score, weight distribution and historical big data, thereby further understanding why the system chooses this strategy, reducing the sense of "black box", and improving trust and acceptance of autonomous driving decisions.

[0015] Furthermore, before performing trajectory alignment processing on the multi-target motion prediction results based on the real-time driving path in the real-time planning and decision data, the above method also includes: Real-time monitoring to check whether the pre-existing object appears within the preset range of the vehicle; When a pre-appearing object appears within the vehicle's preset range, acquire environmental perception data of the pre-appearing object within the preset range; Augmented reality (AR) technology is used to overlay environmental perception data onto a real-world scene view containing the real-time driving path in a visual form. The environmental perception data includes relevant information about the objects to be observed within a preset range and / or weather status information.

[0016] The beneficial effect of adopting the above-mentioned further solution is that when a pre-appearing object appears within the vehicle's preset range, the environmental perception data of the object is immediately superimposed onto the real scene view as a visual icon using AR technology, and displayed on the same screen as the future trajectory layer. Users can see "what the target is, where it is, and what environmental factors affect it" at once without switching perspectives, reducing information fragmentation and improving environmental perception efficiency and decision-making confidence.

[0017] Furthermore, after visualizing the future time-series trajectory layer according to a preset visualization method and simultaneously displaying the real-time driving path, the above method also includes: User feedback information is collected in real time through at least one interaction modality, including visual modality, voice modality and touch modality; Identify users' intervention intentions and emotional states based on feedback information; Based on the intervention intention and emotional state, the vehicle's current driving strategy is adjusted to obtain an adjusted driving strategy, which is then used to control the vehicle.

[0018] The beneficial effects of adopting the above-mentioned further solutions are that they can collect multimodal feedback from vision, voice, and touch in real time and identify the user's intervention intention and emotional state. Based on this, the current driving strategy can be dynamically adjusted, allowing users to correct or confirm autonomous driving behavior in a natural way in an instant, while reducing excessive prompts or false interventions through emotional perception, thus balancing safety redundancy and user comfort.

[0019] Furthermore, after obtaining the trajectory of the motion from the current moment to the next moment, the above method also includes: Based on the trajectory of the vehicle in the next moment and the driving state of the vehicle at the current moment, the next driving action of the vehicle is predicted. Prompt the user to perform the next driving action through at least one prompting mode; Among them, at least one prompting mode is a preset prompting mode or a prompting mode selected by the user, and the prompting mode includes at least one of visual prompting, voice prompting and tactile prompting.

[0020] The beneficial effect of adopting the above-mentioned further solution is that, based on the generated motion trajectory of the next moment and the current driving status, the next driving action can be predicted in advance and proactively informed to the user through various prompt modes such as vision, voice or touch, so that the user can have psychological expectations before the vehicle executes the action, reduce the sense of surprise, and improve the predictability of autonomous driving and the comfort of riding.

[0021] Furthermore, the above methods also include: In response to a user's request to replay the target information, a preset replay interface is invoked; Receive playback instructions from the user based on the playback interface. The playback instructions are used to specify the target playback time and / or the target playback scenario. In response to a playback command, target information corresponding to the target playback time and / or target playback scenario is retrieved from a historical data file; the historical data file contains replayable data, including environmental perception data, decision-making process information, and feedback information corresponding to each driving process of the vehicle. The acquired target information is displayed synchronously in the playback interface; The target information includes at least one of the following: environmental perception data, weather condition data, driving decision information, and feedback information.

[0022] The beneficial effects of adopting the above-mentioned further solutions are that, in response to user playback requests, environmental perception, weather, decision-making and feedback information at any time can be quickly retrieved and synchronously reproduced from historical data files, helping users or developers to review the entire autonomous driving process afterward, improving system transparency and traceability, and providing real-scene data support for algorithm optimization.

[0023] Secondly, in order to solve the above-mentioned technical problems, this disclosure also provides a vehicle motion status visualization device, which includes: The acquisition module is used to respond to autonomous driving commands and acquire real-time driving information of the vehicle. The real-time driving information includes real-time vehicle data, real-time road condition data, and real-time planning and decision data. The prediction module is used to predict the movement of the vehicle and the surrounding objects based on the vehicle's historical driving data and real-time driving information, and to obtain multi-target motion prediction results. The multi-target motion prediction results include the vehicle's trajectory at the current moment and the motion state information of the objects to be predicted within the vehicle's target range. The future time-series trajectory layer determination module is used to perform trajectory alignment processing on the multi-target motion prediction results based on the real-time driving path in the real-time planning and decision data to obtain the future time-series trajectory layer. The future time-series trajectory layer is used to characterize the position sequence and motion trend of the vehicle and the pre-appearing object in the future preset time period. The display module is used to visualize the future time-series trajectory layer according to the preset visualization display method, and simultaneously display the real-time driving path.

[0024] Thirdly, in order to solve the above-mentioned technical problems, this disclosure also provides an electronic device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the vehicle motion status visualization method of this disclosure.

[0025] Fourthly, in order to solve the above-mentioned technical problems, this disclosure also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the vehicle motion state visualization method of this disclosure.

[0026] Fifthly, in order to solve the above-mentioned technical problems, this disclosure also provides a vehicle, including a motion situation visualization device.

[0027] Additional aspects and advantages of this disclosure will be set forth in part in the description which follows, and will become apparent from the description or may be learned by practice of this disclosure. Attached Figure Description

[0028] To more clearly illustrate the technical solutions in the embodiments of this disclosure, the accompanying drawings used in the description of the embodiments of this disclosure will be briefly introduced below.

[0029] Figure 1 This is a flowchart illustrating a method for visualizing the motion state of a vehicle according to an embodiment of this disclosure. Figure 2 A schematic diagram of the structure of a vehicle motion status visualization device provided in one embodiment of this disclosure; Figure 3 This is a schematic diagram of the structure of an electronic device provided in one embodiment of the present disclosure. Detailed Implementation

[0030] The principles and features of this disclosure are described below. The examples given are for illustrative purposes only and are not intended to limit the scope of this disclosure.

[0031] The technical solutions of this disclosure and how they solve the aforementioned technical problems will be described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of this disclosure will now be described with reference to the accompanying drawings.

[0032] In this disclosure, all acquisition of signals, information, or data is carried out in strict compliance with the relevant data protection laws and policies of the country where the location is situated, and with the authorization of the owner of the relevant device.

[0033] The solution provided in this disclosure can be applied to any application scenario that requires visualization of vehicle motion status. The solution provided in this disclosure can be executed by any electronic device, such as an in-vehicle control terminal.

[0034] This disclosure provides a possible implementation, such as... Figure 1 As shown, a flowchart of a method for visualizing the motion status of a vehicle is provided. For ease of description, the method provided in this embodiment will be described below using an on-board control terminal as the execution subject as an example. Figure 1 The flowchart shown indicates that the method may include the following steps: S10 responds to autonomous driving commands by acquiring real-time driving information of the vehicle, including real-time vehicle data, real-time road condition data, and real-time planning and decision data. S20, based on the vehicle's historical driving data and real-time driving information, predict the movement of the vehicle and the objects that are expected to appear around the vehicle, and obtain multi-target motion prediction results. The multi-target motion prediction results include the vehicle's trajectory at the current moment and the motion state information of the objects that are expected to appear within the target range of the vehicle. S30, based on the real-time driving path in the real-time planning and decision data, performs trajectory alignment processing on the multi-target motion prediction results to obtain the future time-series trajectory layer. The future time-series trajectory layer is used to characterize the position sequence and motion trend of the vehicle and the pre-appearing object in the future preset time period. S40 visualizes the future time-series trajectory layer according to the preset visualization display method and simultaneously displays the real-time driving path.

[0035] The method disclosed herein acquires real-time vehicle data, road condition data, and planning decision data in a single response to autonomous driving commands. Based on historical driving information, it predicts the vehicle's trajectory at the next moment and the motion state of objects expected to appear within the target range. The predicted results are then aligned with the real-time planning decision data to generate a future time-series trajectory layer. Finally, this layer and the real-time driving path are presented synchronously in a preset visualization method, allowing users to see "how the vehicle is going to go" and "how surrounding objects are going to move" on the same interface. This exposes potential conflicts in advance, improves the transparency and trust of autonomous driving, reduces misoperation due to information lag, and enhances driving safety. At the same time, presenting the "future trajectory layer" and the "real-time path" at the same time and on the same screen avoids users having to split the screen or piece together information in their minds. Users can immediately compare the planned line with the actual movement trend, detect deviations or conflicts in advance, reduce cognitive load, and shorten reaction time.

[0036] The present disclosure will be further described below with reference to the following specific embodiments. In this embodiment, the vehicle motion situation visualization method can be applied to an autonomous driving system. The vehicle motion situation visualization method provided in this embodiment may include the following steps: S10 responds to autonomous driving commands by acquiring real-time driving information of the vehicle, including real-time vehicle data, real-time road condition data, and real-time planning and decision data. Autonomous driving commands refer to control signals generated internally or triggered externally by the vehicle's autonomous driving system to initiate or maintain autonomous driving mode and drive subsequent perception, prediction, planning, and visualization processes. These include, but are not limited to, user pressing the "autonomous driving" button, automatic activation after the system detects an autonomous driving section, remote driving requests sent from the cloud, or platooning instructions issued by the dispatch platform. Once the command takes effect, the vehicle enters autonomous driving mode and continuously outputs real-time driving information to support motion situation visualization. In autonomous driving technology, real-time driving information refers to a complete set of dynamic data synchronously collected and used to instantly update the motion situation after the vehicle enters autonomous driving mode. Real-time vehicle data refers to the vehicle's pose (latitude and longitude, heading, yaw rate), speed, acceleration, steering angle, brake / throttle opening, and chassis status. Real-time road condition data refers to vehicle status information collected and received in real-time by various sensors (such as cameras, radar, lidar, etc.) and communication devices (such as vehicle-to-everything (V2X) technology) during driving. Real-time planning and decision data refers to the driving path, speed curve, expected lane change / turn / stop instructions, and their confidence levels generated instantly by the autonomous driving system based on the above inputs.

[0037] S20, based on the vehicle's historical driving data and real-time driving information, predict the movement of the vehicle and the objects that are expected to appear around the vehicle, and obtain multi-target motion prediction results. The multi-target motion prediction results include the vehicle's trajectory at the current moment and the motion state information of the objects that are expected to appear within the target range of the vehicle. Among them, machine learning algorithms (such as LSTM or CNN) can be used to analyze the vehicle's historical driving data and real-time driving information to predict the movement of objects that may appear around the vehicle in the next moment (e.g., within the next 5-10 seconds). The objects that may appear refer to objects that may appear, such as pedestrians and vehicles.

[0038] The trajectory of a vehicle at the next moment can be represented as a series of coordinate points, describing the vehicle's travel path over a period of time. Motion state information can include the object's position, velocity, acceleration, direction of motion, etc., used to describe the object's motion trend over a period of time.

[0039] Historical driving data can include: vehicle data, road condition data, and planning decision data for the past period.

[0040] S30, based on the real-time driving path in the real-time planning and decision data, performs trajectory alignment processing on the multi-target motion prediction results to obtain the future time-series trajectory layer. The future time-series trajectory layer is used to characterize the position sequence and motion trend of the vehicle and the pre-appearing object in the future preset time period. Trajectory alignment processing refers to the spatiotemporal registration of the future trajectories (the position sequence of the vehicle and surrounding pre-emerging objects) in the multi-target motion prediction results with the real-time driving path of the vehicle given by the real-time planning and decision data, under the same coordinate system, the same time reference, and the same velocity profile. Each future trajectory reflects the vehicle's motion trajectory in the next moment from the current moment, as well as the motion state information of pre-emerging objects within the vehicle's target range.

[0041] In S30 above, based on the real-time driving path in the real-time planning and decision data, a possible implementation method for aligning the multi-target motion prediction results to obtain the future time-series trajectory layer is as follows: By placing each future trajectory in the multi-objective motion prediction results and the real-time driving path given by the real-time planning and decision data in the same world coordinate system, and using the current time as the reference, the trajectory points in the future time period are unified with timestamps, resampled with velocity-curvature and spatially interpolated, so that the predicted trajectory and the planned path are fully registered in the spatiotemporal dimension, forming a future time series trajectory layer that can be directly superimposed and displayed.

[0042] S40 visualizes the future time-series trajectory layer according to the preset visualization display method and simultaneously displays the real-time driving path.

[0043] Optionally, the display can be optimized using a lightweight rendering engine (such as WebGL) to improve visualization efficiency and reduce rendering latency.

[0044] The preset visualization display methods include: assigning a fixed main color according to the object category and distinguishing between vehicles (thick solid line), other vehicles (medium-thick dashed line), and pedestrians (thin dotted line) by line type / line width. At the same time, the probability of conflict is presented by transparency-time decay, 3D height mapping speed, and thermal gradient, supplemented by arrow direction, rectangle or standing figure icon, red flashing of high-risk sections and speed / TTC text labels, so that the trajectory direction, credibility and risk intensity are clear at a glance in the same AR screen.

[0045] Optionally, in the above S40, one possible way to visualize the future time-series trajectory layer according to a preset visualization display method is as follows: S401 constructs an AR coordinate system in a real-world scene view by using augmented reality (AR) devices and simultaneous localization and mapping (SLAM) technology. S402 transforms the future time-series trajectory layer from the world coordinate system to the AR coordinate system to obtain spatially aligned time-series trajectory data; S403, in the AR coordinate system, renders different types of traffic participants represented by time-series trajectory data with corresponding and distinctive visual icons, and overlays them on the real scene view containing the real-time driving path.

[0046] The AR coordinate system refers to a three-dimensional coordinate system established in the real-world scene view using Simultaneous Localization and Mapping (SLAM) algorithms, based on real-time visual and depth data collected by augmented reality (AR) devices. Its origin, axes, and scale correspond one-to-one with the physical world. The purpose of converting the future time-series trajectory layer from the world coordinate system to the AR coordinate system is to eliminate origin displacement, axis rotation, and scale differences between the world coordinate system and the AR view. This ensures that each point in the future time-series trajectory layer corresponds one-to-one with the real-world spatial location in the AR coordinate system, guaranteeing that the virtual trajectory can be precisely superimposed on the actual roads, vehicles, and pedestrians seen on the windshield or HUD with millimeter-level accuracy, achieving an augmented reality effect of "what you see is what you walk."

[0047] Optionally, the distinctive visual icons refer to AR symbols that are pre-defined for different traffic participants (such as bicycles, passenger cars, trucks, pedestrians, two-wheeled vehicles, traffic signs, etc.) and have significant differences in shape, color, line width, transparency, or dynamic flashing frequency. For example, bicycles use solid blue arrows, pedestrians use red standing figures, trucks use orange squares, and traffic signs use yellow diamonds. Furthermore, the transparency or size of similar icons gradually changes with the prediction confidence level, so that users can instantly identify the object category and confidence level in the overlay view.

[0048] Optionally, in S403 above, one possible way to render different types of traffic participants represented by the time-series trajectory data in the AR coordinate system using corresponding and distinctive visual icons, and overlay them on a real-world view containing the real-time driving path, is as follows: S4031, Based on the time offset of each trajectory point in the time-series trajectory data relative to the current time, determine the dynamically decreasing transparency coefficient corresponding to each trajectory point in the time-series trajectory data according to the preset transparency-time decay function. S4032, associate each dynamically decreasing transparency coefficient with its corresponding trajectory point to obtain enhanced trajectory data with dynamic transparency attributes; S4033 renders the enhanced trajectory data with dynamic transparency properties as a semi-transparent layer and overlays it onto the real scene view containing the real-time driving path displayed on the current page. The dynamically decreasing transparency coefficient is used to visually represent the confidence decay of trajectory prediction.

[0049] The dynamically decreasing transparency coefficient refers to a value that gradually decreases according to a preset function (such as linear, exponential, or Gaussian decay) as the time step increases, based on the current moment. It is used to make the trajectory points fade further away from "now", so as to intuitively present the trend of the prediction confidence decreasing over time in the same AR view, and avoid users mistaking the low confidence trajectory in the distant future as a high confidence reference in the near future.

[0050] Associating each dynamically decreasing transparency coefficient with its corresponding trajectory point means calculating a transparency value for each point according to a preset decay function based on the time interval between the trajectory point and the current moment. This value is then written as an attribute field of the point into the trajectory data, so that each trajectory point is displayed according to its unique transparency value during rendering, thus achieving a "fader as it gets farther away" confidence visualization effect. The association method can be pointers, key-value pairs, or database association.

[0051] In this disclosed solution, after predicting the motion of the vehicle and surrounding objects based on historical driving data and real-time driving information to obtain multi-target motion prediction results, the method further includes: Based on the multi-target motion prediction results, it is determined whether a pre-appearing object appears within the preset range of the vehicle. Specifically, the current position of the vehicle can be used as the center, and the trajectory point sequence of each pre-appearing object in the multi-target motion prediction results within a preset time period (such as 5s) can be compared with a preset spatial threshold (a rectangular or fan-shaped area composed of longitudinal Δx in front, lateral Δy on both sides, and height Δz). If any trajectory point falls into the threshold range, it is determined that a pre-appearing object has appeared within the preset range.

[0052] When it is determined that a pre-occurring object appears within the preset range of the vehicle, decision-making process information is generated based on the motion state information of the pre-occurring object. The decision-making process information is visualized to obtain visualized decision-making process information. The visualized decision-making process information refers to converting the complete logical chain of "current decision - decision basis - alternative plan - alternative reason - final decision - final reason" generated based on the multi-objective motion prediction results into graphical elements such as decision tree flowchart, decision heat map, and statistical comparison chart.

[0053] Based on a pre-defined visualization method, the system displays visualized decision-making process information through a combination of a graphical user interface and voice prompts. The pre-defined visualization method refers to a set of fixed rules for presenting visualized decision-making process information to users, including: displaying the "current-alternative-final" decision branches and key condition thresholds using a decision tree flowchart; using a decision heatmap to indicate the risk score and priority weight of each branch with varying color intensity; overlaying historical success rates and accident rates of similar scenarios in the cloud using a statistical comparison chart in the form of a line graph or radar; and simultaneously playing corresponding voice explanations, enabling users to quickly understand the system's decision-making logic through graphics, text, and sound within the same interface.

[0054] The aforementioned decision-making process information includes the current decision, the basis for the current decision, alternative decisions, the reasons for the alternative decisions, the final decision, and the reasons for the final decision.

[0055] Specifically, decision-making process information refers to the information provided to the user regarding the next driving step when a potential obstacle appears during autonomous driving. Both the current decision and alternative decisions refer to driving strategies. When a potential obstacle appears, different driving strategies can be offered to the user based on both the current decision and alternative decisions. The current decision refers to the immediate decision made at a specific moment. Alternative decisions refer to other possible options considered when making the current decision. The final decision refers to the decision selected and executed after comprehensive evaluation. The basis for the current decision refers to the specific data and rules upon which the current decision is based. The reasons for alternative decisions refer to the specific reasons for choosing or not choosing alternative decisions, and the reasons for the final decision refer to the specific reasons for choosing the final decision.

[0056] As an example, a vehicle is traveling on a highway, and a slow-moving vehicle appears ahead. Current decision: Maintain the current lane and reduce speed. Basis for the current decision: The vehicle ahead is traveling below the safe speed threshold, there are no other vehicles interfering in the current lane, the distance between the two vehicles is less than the safe distance threshold, and traffic rules require maintaining the current lane. Alternative decision: Change lanes to overtake (if the left lane is safe and there are no other vehicles), maintain the current speed, and wait for the vehicle ahead to accelerate. Final decision: Change lanes to overtake. Reasons for the final decision: The left lane is safe and there are no other vehicles. Changing lanes to overtake allows for a faster return to a safe speed.

[0057] Optionally, the above-mentioned visualization processing of decision-making process information to obtain visualized decision-making process information includes: Decision tree flowcharts and / or decision heatmaps are generated based on decision process information. The decision tree flowcharts and / or decision heatmaps include priority weights, risk scores, and key scenario parameters and condition thresholds that trigger decisions. Access the cloud database to obtain decision statistics of other vehicles in similar historical scenarios, and generate statistical comparison charts based on the decision statistics. The statistical comparison charts include the statistical comparison results of planning decision data and historical decision sets. Based on the decision tree flowchart and / or decision heatmap and statistical comparison chart, determine the information for visualizing the decision process.

[0058] Among them, priority weight refers to the relative importance value pre-assigned to each decision branch, which is used to determine which branch to execute first when there are multiple conflicting objectives. The higher the value, the higher the priority.

[0059] Risk scoring refers to a score of 0-1 or 0-100 that quantifies the likelihood of a collision, violation, or decrease in comfort occurring on each decision path in the future. The higher the score, the greater the risk.

[0060] The key scenario parameters and condition thresholds that trigger the decision refer to the core environmental variables and their critical values ​​that must be met to start the corresponding decision branch. For example, the relative distance to the vehicle in front is ≤30 m, TTC is ≤2.5 s, and the radius of curvature of the lane line is ≤100 m. When the real-time detection value exceeds or falls below the threshold, the corresponding decision node is triggered.

[0061] Optionally, one way to determine the visualized decision-making process information based on the decision tree flowchart and / or decision heatmap and statistical comparison chart is as follows: First, the decision tree flowchart is parsed to generate a "node-weight-risk" triple table. Then, the decision heatmap color scale array is bound to the triple table by coordinates to obtain a data package containing color attributes. Subsequently, a cloud interface is called to extract historical success rate and accident rate curves and their percentile rankings that match the current scene parameters from the statistical comparison chart. Finally, the data package and ranking results are packaged into visualized decision process information according to a preset JSON template. The decision heatmap color scale array refers to a one-dimensional numerical sequence obtained by mapping the risk value of each pixel in the heatmap to a 0-255 grayscale or RGB color channel and expanding it in row-first, column-second order. The array index corresponds one-to-one with the image coordinates, used to quantify the "risk-color" correspondence and quickly match it with decision node data.

[0062] Optionally, the above statistical comparison chart can be a line chart or a radar chart.

[0063] Optionally, before performing trajectory alignment processing on the multi-objective motion prediction results based on the real-time driving path in the real-time planning and decision data, the method further includes: Real-time monitoring to check whether the pre-existing object appears within the preset range of the vehicle; When a pre-appearing object appears within the vehicle's preset range, acquire environmental perception data of the pre-appearing object within the preset range; Augmented reality (AR) technology is used to overlay environmental perception data onto a real-world scene view containing the real-time driving path in a visual form. The environmental perception data includes relevant information about the objects to be observed within a preset range and / or weather status information.

[0064] Environmental perception data refers to all raw or fused information about the surrounding traffic environment that vehicles collect in real time through sensors such as cameras, radar, and lidar. This includes the position, speed, acceleration, and type of other vehicles; the pose and motion of traffic participants such as pedestrians, two-wheeled vehicles, and animals; the status of road boundaries, lane lines, traffic signs, and traffic lights; as well as road conditions, weather-related features (such as rain, snow, water accumulation, and icing), and visibility parameters. This data is used to support prediction, decision-making, and visualization. Weather condition information refers to quantitative data describing the current driving environment's meteorological conditions that vehicles acquire in real time through sensors for rainfall, illumination, temperature, humidity, fog, wind speed, and external meteorological services. This data includes precipitation type and intensity, visibility, road surface slippage coefficient, temperature, humidity, wind speed, and fog concentration. This data is used for prediction, decision-making, weather effect rendering, and safety warnings.

[0065] Optionally, the method further includes: Based on weather information, 3D rendering technology is used to simulate weather effects and display them to the user in the vehicle, while simultaneously adjusting the display attributes of various objects displayed on the current interface.

[0066] The system introduces weather effect simulation display. In rainy or snowy weather, 3D rendering technology can be used to dynamically simulate weather effects (such as raindrop and snowflake animations) on the central control screen or windshield, and the display attributes of various objects displayed on the current interface (such as reflectivity and pedestrian umbrella models) can be adjusted simultaneously.

[0067] Furthermore, the method also includes: Obtain the weather parameters corresponding to the weather status data, and adjust the weather effects based on the weather parameters.

[0068] In this disclosed solution, weather parameters obtained by sensors (such as rain sensors and humidity sensors) can be combined to adjust the details of the simulated weather effect in real time. For example, raindrop density can be linked to vehicle speed to enhance realism.

[0069] Optionally, if the weather status information includes abnormal weather status information, the method further includes: Adjust the display attributes of each object displayed on the current screen based on the abnormal weather status information.

[0070] Optionally, after visualizing the future time-series trajectory layer according to a preset visualization display method and synchronously displaying the real-time driving path, the method further includes: User feedback information is collected in real time through at least one interaction modality, including visual modality, voice modality and touch modality; Identify users' intervention intentions and emotional states based on feedback information; Based on the intervention intention and emotional state, the vehicle's current driving strategy is adjusted to obtain an adjusted driving strategy, which is then used to control the vehicle.

[0071] Intervention intent refers to the user's subjective desire to immediately or soon change the current autonomous driving behavior, conveyed to the system through visual, voice, or touch interaction modalities. This includes takeover requests, deceleration, lane changing, stopping, canceling overtaking instructions, or confirming continued execution. Emotional state refers to the user's emotional category and intensity identified in real time through facial expressions, voice tone, body movements, or physiological signals. This includes levels such as trust, anxiety, anger, calmness, fear, and fatigue, used to determine the user's acceptance of the current autonomous driving behavior and trigger corresponding prompts or strategy adjustments.

[0072] Optionally, one implementation of adjusting the vehicle's current driving strategy based on the intervention intention and emotional state to obtain the adjusted driving strategy may include: In response to the recognition of a user's negative intention to intervene, the current decision-making behavior is suspended or modified; In response to recognizing a user's anxiety or worry, the transparency of the system's decision-making information is improved; In response to recognizing a user's angry emotional state, the system reduces the frequency of prompts to the user.

[0073] Optionally, after obtaining the trajectory of the motion at the next moment from the current moment, the method further includes: Based on the trajectory of the vehicle in the next moment and the driving state of the vehicle in the current moment, the next driving action of the vehicle is predicted. The next driving action refers to the instantaneous driving behavior that will be executed within a few seconds, which is predicted based on the trajectory of the vehicle in the next moment and the driving state of the vehicle in the current moment. Specifically, it includes lane changing, left turning, right turning, straight acceleration, deceleration, stopping or obstacle avoidance steering, which is used to remind the user in advance of the operation that the vehicle is about to perform.

[0074] Prompt the user to perform the next driving action through at least one prompting mode; Among them, at least one prompting mode is a preset prompting mode or a prompting mode selected by the user, and the prompting mode includes at least one of visual prompting, voice prompting and tactile prompting.

[0075] Optionally, the method further includes: In response to a user's request to replay the target information, a preset replay interface is invoked; Receive playback instructions from the user based on the playback interface. The playback instructions are used to specify the target playback time and / or the target playback scenario. In response to a playback command, target information corresponding to the target playback time and / or target playback scenario is retrieved from a historical data file; the historical data file contains replayable data, including environmental perception data, decision-making process information, and feedback information corresponding to each driving process of the vehicle. The acquired target information is displayed synchronously in the playback interface; The target information includes at least one of the following: environmental perception data, weather condition data, driving decision information, and feedback information.

[0076] This disclosed solution can be applied to autonomous driving systems, and the following beneficial effects can be achieved through this solution: 1. Enhance user trust in autonomous driving systems: By displaying the decision-making process, environmental perception results, and vehicle behavior intentions of the autonomous driving system in real time, users can clearly understand the system's behavioral logic and reduce distrust of "black box" operations.

[0077] 2. Enhance user experience and interaction satisfaction: Through multimodal interaction design (visual, auditory, tactile) and personalized settings, the system can provide users with an intuitive and convenient interactive experience.

[0078] 3. Improve driving safety: By displaying the vehicle's behavioral intentions in advance and providing an emergency takeover mechanism, the system can help users react correctly in critical moments and avoid potential dangers.

[0079] 4. Optimize the decision-making capabilities of the autonomous driving system: By recording user feedback and historical decision-making data, the system can continuously optimize its decision-making algorithm and improve the intelligence level of autonomous driving.

[0080] 5. Enhance system transparency and explainability: By displaying the decision-making basis and alternative decisions, the system can explain its behavioral logic to users, meeting users' needs for transparency and explainability.

[0081] 6. Supports multiple application scenarios: This system is suitable for various driving scenarios and user groups, and has broad application prospects.

[0082] 7. Reduce user learning costs: Through intuitive interface design and multimodal interaction, the system can reduce user learning costs and enable them to quickly master the operation methods.

[0083] 8. Enhance the social acceptance of autonomous driving systems: By improving the transparency of the system and the user experience, this technology can enhance public acceptance and recognition of autonomous driving technology.

[0084] Based on and Figure 1 Using the same principle as the method shown, this disclosure also provides a vehicle motion status visualization device 20, such as... Figure 2 As shown, the vehicle motion situation visualization device 20 may include an acquisition module 210, a prediction module 220, and a display module 230, wherein: The acquisition module 210 is used to acquire real-time driving information of the vehicle in response to autonomous driving instructions. The real-time driving information includes real-time vehicle data, real-time road condition data, and real-time planning and decision data. The prediction module 220 is used to predict the movement of the vehicle and the objects that are expected to appear around the vehicle based on the vehicle's historical driving data and real-time driving information, and to obtain multi-target motion prediction results. The multi-target motion prediction results include the vehicle's trajectory at the current moment and the motion state information of the objects that are expected to appear within the target range of the vehicle. The future time-series trajectory layer determination module 230 is used to perform trajectory alignment processing on the multi-target motion prediction results based on the real-time driving path in the real-time planning and decision data to obtain the future time-series trajectory layer. The future time-series trajectory layer is used to characterize the position sequence and motion trend of the vehicle and the pre-appearing object in a future preset time period. Display module 240 is used to visualize the future time-series trajectory layer according to a preset visualization display method, and simultaneously display the real-time driving path.

[0085] Optionally, when the display module 240 visualizes the future time-series trajectory layer according to a preset visualization display method, it is specifically used for: By using augmented reality (AR) devices and simultaneous localization and mapping (SLAM) technology, an AR coordinate system is constructed in the real scene view; Transform the future time-series trajectory layer from the world coordinate system to the AR coordinate system to obtain spatially aligned time-series trajectory data; In the AR coordinate system, different types of traffic participants represented by time-series trajectory data are rendered with corresponding and distinctive visual icons and overlaid on the real scene view containing the real-time driving path.

[0086] Optionally, when the aforementioned display module 240 renders different types of traffic participants represented by the time-series trajectory data as corresponding and distinctive visual icons in the AR coordinate system, and overlays them on a real-world scene view containing the real-time driving path, it is specifically used for: Based on the time offset of each trajectory point in the time-series trajectory data relative to the current time, the dynamically decreasing transparency coefficient corresponding to each trajectory point in the time-series trajectory data is determined according to the preset transparency-time decay function. By associating each dynamically decreasing transparency coefficient with its corresponding trajectory point, enhanced trajectory data with dynamic transparency attributes is obtained. The enhanced trajectory data with dynamic transparency properties is rendered as a semi-transparent layer and overlaid on the real-world scene view that contains the real-time driving path displayed on the current page. The dynamically decreasing transparency coefficient is used to visually represent the confidence decay of trajectory prediction.

[0087] Optionally, after predicting the motion of the vehicle and surrounding objects based on historical driving data and real-time driving information to obtain multi-target motion prediction results, the device further includes: a decision module, used for: Based on the multi-target motion prediction results, determine whether the pre-appearing object appears within the vehicle's preset range; When it is determined that a pre-occurring object appears within the preset range of the vehicle, decision-making process information is generated based on the motion state information of the pre-occurring object. Visualize the decision-making process information to obtain visualized decision-making process information; Based on a pre-defined visualization method, information about the visual decision-making process is displayed through a combination of a graphical user interface and voice prompts. The decision-making process information includes the current decision, the basis for the current decision, alternative decisions, the reasons for the alternative decisions, the final decision, and the reasons for the final decision.

[0088] Optionally, when the above decision-making module performs visualization processing on the decision-making process information to obtain visualized decision-making process information, it is specifically used for: Decision tree flowcharts and / or decision heatmaps are generated based on decision process information. The decision tree flowcharts and / or decision heatmaps include priority weights, risk scores, and key scenario parameters and condition thresholds that trigger decisions. Access the cloud database to obtain decision statistics of other vehicles in similar historical scenarios, and generate statistical comparison charts based on the decision statistics. The statistical comparison charts include the statistical comparison results of planning decision data and historical decision sets. Based on the decision tree flowchart and / or decision heatmap and statistical comparison chart, determine the information for visualizing the decision process.

[0089] Optionally, before performing trajectory alignment processing on the multi-target motion prediction results based on the real-time driving path in the real-time planning and decision data, the device further includes: an environmental perception data display module, used for: Real-time monitoring to check whether the pre-existing object appears within the preset range of the vehicle; When a pre-appearing object appears within the vehicle's preset range, acquire environmental perception data of the pre-appearing object within the preset range; Augmented reality (AR) technology is used to overlay environmental perception data onto a real-world scene view containing the real-time driving path in a visual form. The environmental perception data includes relevant information about the objects to be observed within a preset range and / or weather status information.

[0090] Optionally, after visualizing the future time-series trajectory layer according to a preset visualization method and simultaneously displaying the real-time driving path, the device further includes: a multimodal module, used for: User feedback information is collected in real time through at least one interaction modality, including visual modality, voice modality and touch modality; Identify users' intervention intentions and emotional states based on feedback information; Based on the intervention intention and emotional state, the vehicle's current driving strategy is adjusted to obtain an adjusted driving strategy, which is then used to control the vehicle.

[0091] Optionally, after obtaining the motion trajectory for the next moment from the current moment, the device further includes: a prompting module, used for: Based on the trajectory of the vehicle in the next moment and the driving state of the vehicle at the current moment, the next driving action of the vehicle is predicted. Prompt the user to perform the next driving action through at least one prompting mode; Among them, at least one prompting mode is a preset prompting mode or a prompting mode selected by the user, and the prompting mode includes at least one of visual prompting, voice prompting and tactile prompting.

[0092] Optionally, the device further includes: a playback module for: In response to a user's request to replay the target information, a preset replay interface is invoked; Receive playback instructions from the user based on the playback interface. The playback instructions are used to specify the target playback time and / or the target playback scenario. In response to a playback command, target information corresponding to the target playback time and / or target playback scenario is retrieved from a historical data file; the historical data file contains replayable data, including environmental perception data, decision-making process information, and feedback information corresponding to each driving process of the vehicle. The acquired target information is displayed synchronously in the playback interface; The target information includes at least one of the following: environmental perception data, weather condition data, driving decision information, and feedback information.

[0093] The vehicle motion situation visualization device of this disclosure embodiment can execute the vehicle motion situation visualization method provided in this disclosure embodiment. The implementation principle is similar. The actions performed by each module and unit in the vehicle motion situation visualization device in each embodiment of this disclosure are corresponding to the steps in the vehicle motion situation visualization method in each embodiment of this disclosure. For detailed functional descriptions of each module of the vehicle motion situation visualization device, please refer to the descriptions of the corresponding vehicle motion situation visualization methods shown above, which will not be repeated here.

[0094] The aforementioned vehicle motion situation visualization device can be a computer program (including program code) running on a computer device, such as an application software; the device can be used to execute the corresponding steps in the method provided in the embodiments of this disclosure.

[0095] In some embodiments, the vehicle motion situation visualization device provided in this disclosure can be implemented using a combination of hardware and software. As an example, the vehicle motion situation visualization device provided in this disclosure can be a processor in the form of a hardware decoding processor, which is programmed to execute the vehicle motion situation visualization method provided in this disclosure. For example, the processor in the form of a hardware decoding processor can be one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), or other electronic components.

[0096] In other embodiments, the vehicle motion status visualization device provided in this disclosure can be implemented in software. Figure 2 A vehicle motion situation visualization device stored in a memory is shown. It can be software in the form of programs and plug-ins, and includes a series of modules, including an acquisition module 210, a prediction module 220 and a display module 230, for implementing the vehicle motion situation visualization method provided in the embodiments of this disclosure.

[0097] The modules described in the embodiments of this disclosure can be implemented in software or hardware. The names of the modules are not, in some cases, intended to limit the functionality of the module itself.

[0098] Based on the same principles as the methods shown in the embodiments of this disclosure, the embodiments of this disclosure also provide an electronic device, which may include, but is not limited to: a processor and a memory; the memory for storing computer programs; and the processor for executing the methods shown in any embodiment of this disclosure by invoking the computer programs.

[0099] In one alternative embodiment, an electronic device is provided, such as Figure 3 As shown, Figure 3 The illustrated electronic device 4000 includes a processor 4001 and a memory 4003. The processor 4001 and the memory 4003 are connected, for example, via a bus 4002. Optionally, the electronic device 4000 may further include a transceiver 4004, which can be used for data interaction between the electronic device and other electronic devices, such as sending and / or receiving data. It should be noted that in practical applications, the transceiver 4004 is not limited to one type, and the structure of the electronic device 4000 does not constitute a limitation on the embodiments of this disclosure.

[0100] Processor 4001 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It may implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with this disclosure. Processor 4001 may also be a combination that implements computational functions, such as including one or more microprocessor combinations, a combination of a DSP and a microprocessor, etc.

[0101] Bus 4002 may include a pathway for transmitting information between the aforementioned components. Bus 4002 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. Bus 4002 can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 3 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0102] The memory 4003 may be ROM (Read Only Memory) or other types of static storage devices capable of storing static information and instructions, RAM (Random Access Memory) or other types of dynamic storage devices capable of storing information and instructions, or EEPROM (Electrically Erasable Programmable Read Only Memory), CD-ROM (Compact Disc Read Only Memory) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited thereto.

[0103] The memory 4003 stores application code (computer program) that executes the present invention, and its execution is controlled by the processor 4001. The processor 4001 executes the application code stored in the memory 4003 to implement the content shown in the foregoing method embodiments.

[0104] Among these, electronic devices can also be terminal devices. Figure 3 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.

[0105] This disclosure provides a computer-readable storage medium storing a computer program that, when run on a computer, enables the computer to execute the corresponding content in the aforementioned method embodiments.

[0106] According to another aspect of this disclosure, a computer program product or computer program is also provided, comprising computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the methods provided in the various embodiments described above.

[0107] Computer program code for performing the operations of this disclosure can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0108] It should be understood that the flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of methods and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0109] The computer-readable storage medium provided in this disclosure can be, for example,—but is not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this disclosure, a computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0110] The aforementioned computer-readable storage medium carries one or more programs, which, when executed by the electronic device, cause the electronic device to perform the method shown in the above embodiments.

[0111] The above description is merely a preferred embodiment of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features disclosed in this disclosure that have similar functions.

Claims

1. A method for visualizing the motion status of a vehicle, characterized in that, include: In response to autonomous driving commands, the vehicle obtains real-time driving information, which includes real-time vehicle data, real-time road condition data, and real-time planning and decision data. Based on the vehicle's historical driving data and real-time driving information, the movement of the vehicle and the objects that are expected to appear around the vehicle are predicted to obtain a multi-target motion prediction result. The multi-target motion prediction result includes the vehicle's trajectory at the current moment and the motion state information of the objects that are expected to appear within the target range of the vehicle. Based on the real-time driving path in the real-time planning and decision data, the multi-target motion prediction results are processed by trajectory alignment to obtain a future time-series trajectory layer. The future time-series trajectory layer is used to characterize the position sequence and motion trend of the vehicle and the pre-appearing object in a future preset time period. The future time-series trajectory layer is visualized according to a preset visualization display method, and the real-time driving path is displayed simultaneously.

2. The method according to claim 1, characterized in that, The step of visualizing the future time-series trajectory layer according to a preset visualization display method includes: By using augmented reality (AR) devices and simultaneous localization and mapping (SLAM) technology, an AR coordinate system is constructed in the real scene view; The future time-series trajectory layer is transformed from the world coordinate system to the AR coordinate system to obtain spatially aligned time-series trajectory data; In the AR coordinate system, the different types of traffic participants represented by the time-series trajectory data are rendered with corresponding and distinguishable visual icons and overlaid on the real scene view containing the real-time driving path.

3. The method according to claim 2, characterized in that, In the AR coordinate system, the different types of traffic participants represented by the time-series trajectory data are rendered with corresponding and distinguishable visual icons, and overlaid on a real-world view containing the real-time driving path, including: Based on the time offset of each trajectory point in the time-series trajectory data relative to the current time, the dynamically decreasing transparency coefficient corresponding to each trajectory point in the time-series trajectory data is determined according to the preset transparency-time decay function. By associating each dynamically decreasing transparency coefficient with its corresponding trajectory point, enhanced trajectory data with dynamic transparency attributes is obtained. The enhanced trajectory data with dynamic transparency properties is rendered as a semi-transparent layer and overlaid on the real scene view containing the real-time driving path displayed on the current page. The dynamically decreasing transparency coefficient is used to visually represent the confidence decay of trajectory prediction.

4. The method according to claim 1, characterized in that, After predicting the motion of the vehicle and surrounding pre-existing objects based on the vehicle's historical driving data and real-time driving information to obtain multi-target motion prediction results, the method further includes: Based on the multi-target motion prediction results, determine whether the pre-appearing object appears within the preset range of the vehicle; When it is determined that the pre-appearing object appears within a preset range of the vehicle, decision process information is generated based on the motion state information of the pre-appearing object; The decision-making process information is visualized to obtain visualized decision-making process information; Based on a preset visualization display method, the visualized decision-making process information is displayed through a combination of a graphical user interface and voice prompts; The decision-making process information includes the current decision, the basis for the current decision, alternative decisions, the reasons for the alternative decisions, the final decision, and the reasons for the final decision.

5. The method according to claim 4, characterized in that, The visualization processing of the decision-making process information to obtain visualized decision-making process information includes: A decision tree flowchart and / or decision heatmap are generated based on the decision process information. The decision tree flowchart and / or decision heatmap include priority weights, risk scores, and key scenario parameters and condition thresholds that trigger decisions. Access the cloud database to obtain decision statistics of other vehicles in similar historical scenarios, and generate a statistical comparison chart based on the decision statistics. The statistical comparison chart includes the statistical comparison results of planning decision data and historical decision sets. Based on the decision tree flowchart and / or decision heatmap, and the statistical comparison chart, determine the information for the visualized decision process.

6. The method according to any one of claims 1-3, characterized in that, Before performing trajectory alignment processing on the multi-objective motion prediction results based on the real-time driving path in the real-time planning and decision data, the method further includes: Real-time monitoring to determine whether the pre-appearing object appears within a preset range of the vehicle; When the pre-appearing object appears within a preset range of the vehicle, environmental perception data of the pre-appearing object appearing within the preset range is acquired; Augmented reality (AR) technology is used to overlay the environmental perception data onto a real scene view containing the real-time driving path in a visual form. The environmental perception data includes relevant information about the pre-occurring object within the preset range and / or weather status information.

7. The method according to any one of claims 1-3, characterized in that, After visualizing the future time-series trajectory layer according to a preset visualization display method and synchronously displaying the real-time driving path, the method further includes: User feedback information is collected in real time through at least one interaction modality, including visual modality, voice modality and touch modality; Based on the feedback information, the user's intervention intention and emotional state are identified; Based on the intervention intention and emotional state, the current driving strategy of the vehicle is adjusted to obtain an adjusted driving strategy, so as to control the vehicle based on the adjusted driving strategy.

8. The method according to any one of claims 1-3, characterized in that, After obtaining the trajectory of the motion at the next moment from the current moment, the method further includes: Based on the trajectory of the vehicle in the next moment and the driving state of the vehicle at the current moment, the next driving action of the vehicle is predicted. The user is prompted with at least one prompting mode to indicate the next driving action; The at least one prompting mode is a preset prompting mode or a user-selected prompting mode, and the prompting mode includes at least one of visual prompts, voice prompts and tactile prompts.

9. The method according to any one of claims 1-3, characterized in that, The method further includes: In response to a user's request to replay the target information, a preset replay interface is invoked; Receive playback instructions input by the user based on the playback interface, wherein the playback instructions are used to specify the target playback time and / or the target playback scene; In response to the playback command, target information corresponding to the target playback time and / or target playback scene is obtained from the historical data file; the historical data file is replayable data, including environmental perception data, decision-making process information and feedback information corresponding to each driving process of the vehicle; The acquired target information is simultaneously displayed in the playback interface; The target information includes at least one of the following: environmental perception data, weather condition data, driving decision information, and feedback information corresponding to the vehicle.

10. A vehicle motion status visualization device, characterized in that, include: The acquisition module is used to acquire real-time driving information of the vehicle in response to autonomous driving instructions. The real-time driving information includes real-time vehicle data, real-time road condition data, and real-time planning and decision data corresponding to the vehicle. The prediction module is used to predict the movement of the vehicle and the surrounding objects based on the vehicle's historical driving data and real-time driving information, and to obtain a multi-target motion prediction result. The multi-target motion prediction result includes the vehicle's trajectory at the current moment and the motion state information of the objects within the target range of the vehicle. The future time-series trajectory layer determination module is used to perform trajectory alignment processing on the multi-target motion prediction results based on the real-time driving path in the real-time planning and decision data to obtain a future time-series trajectory layer. The future time-series trajectory layer is used to characterize the position sequence and motion trend of the vehicle and the pre-appearing object in a future preset time period. The display module is used to visualize the future time-series trajectory layer according to a preset visualization display method, and simultaneously display the real-time driving path.

Images