A vehicle-mounted three-dimensional scene rendering method, device, storage medium and program product

Abstract

The application provides a vehicle-mounted three-dimensional scene rendering method, device, storage medium and program product. The method comprises: acquiring perception data of a target object, and constructing a nonlinear motion state space representing a motion process of the target object; performing prediction update on the target object based on a state transition model of the nonlinear motion state space, to obtain prediction state information and a prediction covariance matrix at a current rendering time; extracting variance information related to a position from the covariance matrix to calculate an uncertainty factor, and introducing the uncertainty factor into a graphics rendering pipeline to dynamically adjust at least one type of visual property of a three-dimensional model to realize uncertainty prompting; when new perception data of the target object is acquired, performing fusion update on the prediction state information, and performing smooth transition on a rendering result based on a correction amount to suppress rendering jump. The application realizes high-frame-rate continuous rendering display under the condition of low-frequency update of a sensor, and improves the explainability and safety of a rendering result.

Description

Technical Field

[0001] This application relates to the field of intelligent driving technology, and in particular to a vehicle-mounted 3D scene rendering method, device, storage medium, and program product. Background Technology

[0002] With the development of intelligent cockpits and driver assistance technologies, in-vehicle display systems typically integrate Scene Reconstruction (SR) functionality. This is used to perceive and reconstruct 3D objects in the vehicle's surrounding environment, displaying identified targets such as vehicles, pedestrians, and road obstacles as 3D graphics on the vehicle's screen, thereby enhancing the driver's perception of the surrounding environment. Existing SR display solutions usually rely on target detection and tracking results output by in-vehicle sensors such as LiDAR and cameras, using coordinate transformation and rendering pipelines to render and display target objects in real time.

[0003] In existing technologies, the data acquisition and fusion processing of vehicle-mounted sensors are typically limited by factors such as sampling period, computing resources, and transmission latency, resulting in a relatively low update frequency for the output target perception data, usually in the range of 10Hz to 20Hz. In contrast, vehicle infotainment screens have higher refresh rates, typically reaching 60Hz or higher. Therefore, when an in-vehicle display system directly uses sensor perception data to drive rendering, multiple rendering frame gaps exist between adjacent observation updates, causing the target object to appear jerky or stuttering on the screen, affecting rendering continuity and display stability.

[0004] To address the aforementioned frequency inconsistency issue, existing technologies typically employ interpolation methods (such as the commonly used linear interpolation) to generate intermediate frame rendering results between adjacent observation times, thereby improving image smoothness. However, in real-world driving scenarios, target vehicles often exhibit curvilinear or non-linear motion trajectories during turning, lane changes, and other maneuvers. The assumption of uniform linear motion implicit in linear interpolation is insufficient to accurately reflect the target's movement, easily leading to deviations between the generated predicted trajectory and the target's actual trajectory. This results in visual distortions such as rendering position drift and trajectory inconsistencies, reducing the accuracy and reliability of scene reconstruction. Furthermore, existing scene reconstruction rendering schemes often lack mechanisms to express the reliability of rendering results, making it difficult to provide users with intuitive confidence prompts when sensor observations are interfered with or prediction errors increase. This may affect the interpretability and safety of the display results.

[0005] Therefore, how to achieve high frame rate continuous display of target objects under low sensor update conditions, reduce rendering distortion under nonlinear motion conditions, and improve the credibility of rendering results has become a technical problem that urgently needs to be solved in this field. Summary of the Invention

[0006] To address the shortcomings of existing technologies, this application provides a method, device, storage medium, and program product for rendering in-vehicle 3D scenes, which at least solves the rendering distortion problem caused by the mismatch between the observation update frequency of in-vehicle sensors and the refresh frequency of the vehicle screen in existing technologies.

[0007] To achieve the above objectives and other advantages, some embodiments of this application provide the following aspects:

[0008] In a first aspect, some embodiments of this application provide a method for rendering a vehicle-mounted 3D scene, including:

[0009] Acquire the perception data of the target object, and construct a nonlinear motion state space characterizing the motion process of the target object based on the perception data;

[0010] Based on the state transition model corresponding to the nonlinear motion state space, the target object is predicted and updated to obtain the predicted state information of the target object at the current rendering time and the corresponding prediction covariance matrix.

[0011] Based on the variance information related to the position of the target object in the predicted covariance matrix, an uncertainty factor characterizing the visual uncertainty of the predicted state information is calculated.

[0012] Based on the uncertainty factor and the predicted state information, the three-dimensional model of the target object is rendered and displayed through the graphics rendering pipeline, wherein at least one type of visual attribute of the three-dimensional model is dynamically adjusted based on the uncertainty factor.

[0013] When new perception data of the target object is obtained, the predicted state information is fused and updated based on the new perception data, and the rendering result of the three-dimensional model is smoothly transitioned based on the correction amount generated by the fusion update to suppress rendering jumps.

[0014] Secondly, some embodiments of this application also provide an electronic device, the electronic device comprising:

[0015] One or more processors; and a memory storing computer program instructions that, when executed, cause the processors to perform the in-vehicle 3D scene rendering method as described above.

[0016] Thirdly, some embodiments of this application also provide a computer-readable storage medium having a computer program and / or instructions stored thereon, which, when executed by a processor, implement the vehicle-mounted 3D scene rendering method as described above.

[0017] Fourthly, some embodiments of this application also provide a computer program product, including a computer program and / or instructions, which, when executed by a processor, implement the vehicle-mounted 3D scene rendering method as described above.

[0018] Compared with existing technologies, the solution provided in this application acquires the perception data of the target object and constructs a nonlinear motion state space representing its motion process. Based on a state transition model, the target object is predicted and updated between adjacent rendering frames. This allows for continuous output of predicted state information matching the current rendering time even with low sensor update frequencies, enabling continuous rendering and display of the target object on the vehicle-mounted system at high refresh rates. Simultaneously, an uncertainty factor is calculated based on the variance information related to the target position in the prediction covariance matrix, and this uncertainty factor is introduced into the graphics rendering pipeline to dynamically adjust at least one type of visual attribute of the 3D model. When external environmental conditions deteriorate, leading to increased sensor observation noise, or when there is a delay in observation updates causing prediction errors to accumulate over time, the scene reconstruction rendering result will adaptively reduce the display clarity and visibility of the target model based on the uncertainty factor, resulting in a blurred, weakened, or transparent visual effect. This alerts the driver that the system's current confidence level has decreased, preventing blind trust in the screen rendering results and improving safety and interpretability in human-machine co-driving scenarios. Furthermore, upon acquiring new perception data, by performing a fusion update on the predicted state information and smoothly transitioning the rendering result based on the correction amount, the screen jumps during the switch from the predicted trajectory to the observed trajectory can be suppressed, achieving a flexible connection between observation updates and rendering output. Therefore, this application ensures display smoothness while also taking into account rendering accuracy and reliable expression capabilities under nonlinear motion conditions, making it suitable for application scenarios with high requirements for real-time performance and stability, such as in-vehicle 3D scene reconstruction rendering. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other implementation methods can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a flowchart illustrating a vehicle-mounted 3D scene rendering method provided in an embodiment of this application;

[0021] Figure 2 This is a schematic diagram of the processing flow of the vehicle-mounted 3D scene rendering method based on nonlinear motion state space provided in the embodiments of this application;

[0022] Figure 3This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0023] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0024] Some embodiments of this application relate to a method for rendering a vehicle-mounted 3D scene, see reference. Figure 1 As shown, the method may include the following steps:

[0025] Step S1: Acquire the perception data of the target object, and construct a nonlinear motion state space representing the motion process of the target object based on the perception data.

[0026] In this embodiment, the in-vehicle 3D scene rendering method is deployed and runs on an in-vehicle computing platform (e.g., a cockpit domain controller or an intelligent driving domain controller), which is communicatively connected to the in-vehicle perception system. The in-vehicle perception system may include sensors such as LiDAR, millimeter-wave radar, and cameras, and outputs target detection and tracking results through a perception fusion module. Target objects can be any identifiable and trackable object, such as surrounding vehicles, pedestrians, two-wheeled vehicles, or road obstacles. It should be noted that the system mentioned in this embodiment refers to a graphics rendering system or processing unit that performs 3D scene rendering and uncertainty cues. This system can be an in-vehicle display system or an in-vehicle computer. It acquires environmental data through sensors, processes and renders the 3D scene of the target object, and dynamically adjusts the rendering effect based on the uncertainty factor.

[0027] The refresh rate of in-vehicle displays (such as central control screens or instrument panel screens) is typically 60Hz or higher, while the update frequency of the target data output by perception fusion is typically 10Hz~20Hz, resulting in multiple rendering frame intervals between adjacent perception updates. To continuously obtain the state of the target object between rendering frames, this embodiment constructs a nonlinear motion state space, enabling the system to continuously predict and extrapolate the state of the target object between rendering frames, thereby providing continuous state input for high frame rate rendering.

[0028] In one optional embodiment, step S1 specifically includes:

[0029] Step S101: Extract motion observation information related to the target object from the sensing data.

[0030] The perception fusion module can output target detection and tracking results according to a preset update cycle. Each frame of perception data corresponds to a timestamp and contains multiple tracking entries for target objects. Each tracking entry includes at least a target identifier ID and motion information fields related to the target. The target object to be rendered is located based on the target identifier ID, and the motion observation information of the target object is read from the corresponding tracking entry.

[0031] Motion observation information includes at least one of the following observations: target object position observation information, velocity observation information, heading observation information, and heading angular velocity observation information. Taking a two-dimensional road scene as an example, the planar position coordinates of the target object in the vehicle coordinate system or the world coordinate system can be extracted. , ,speed heading angle and angular velocity of heading , where the subscript k represents the number of the sensing data frame output at each update time.

[0032] Furthermore, motion observation information can also include the height coordinates of the target object. To form a three-dimensional position observation ( , , This allows the target object to achieve correct spatial positioning and rendering even in scenarios with elevation changes, such as ramps, bridges, or overpasses.

[0033] Step S102: Construct an augmented state vector based on motion observation information to characterize the motion state of the target object. The augmented state vector includes at least: position coordinates, velocity, heading angle, and heading angular velocity.

[0034] Obtain motion observation information of the target object in the k-th frame of the perception data, such as position observation information. , Velocity observation information Heading angle observation information and heading angular velocity observation information The above observations are then encapsulated into augmented state vectors in a preset order. : By introducing into the state vector and The system can more accurately model the nonlinear motion of the target object, such as turning and changing lanes, in subsequent prediction updates, avoiding rendering trajectory drift caused by relying solely on the assumption of uniform linear motion.

[0035] In a further alternative embodiment, to adapt to different types of target objects or different precision requirements, the state components can be expanded or clipped when constructing the augmented state vector. For example, for 3D scene reconstruction display, the augmented state vector may further include height coordinates. For scenarios requiring characterization of the target acceleration process, the augmented state vector can further include acceleration. For scenarios requiring a more detailed description of the target attitude, the augmented state vector can further include attitude components such as roll angle or pitch angle. These extensions do not affect the basic structure of the augmented state vector in this embodiment, which at least includes position coordinates, velocity, heading angle, and heading angular velocity.

[0036] Step S103: Establish a nonlinear state transition model corresponding to the augmented state vector to form a nonlinear motion state space.

[0037] A state transition function f(·) is constructed based on the kinematic constraints of the target object to describe the state evolution relationship between adjacent discrete time points. This allows the state components in the augmented state vector, such as position coordinates, velocity, heading angle, and heading angular velocity, to be recursively updated according to a preset motion law. This incorporates the turning behavior of the target object into the state evolution process and avoids trajectory deviations caused by using only the assumption of uniform linear motion. The kinematic constraints can employ a constant turn rate and velocity (CTRV) model or other nonlinear kinematic models, ensuring that the position update of the target object under nonlinear motion conditions such as turning and lane changing exhibits a curvilinear extrapolation consistent with the heading change, thereby forming a nonlinear motion state space that characterizes the nonlinear motion process of the target object.

[0038] In a feasible implementation, when the kinematic constraints adopt the CTRV model, a corresponding CTRV state transition function can be constructed to ensure that the state satisfies:

[0039]

[0040] Where Δt is the time interval between adjacent discrete moments.

[0041] Based on augmented state vector ,exist At that time, the state recursion relation can be expressed as:

[0042]

[0043]

[0044]

[0045]

[0046]

[0047] Furthermore, in When the constant is 0 or near-zero, the state transition function can degenerate into a uniform linear form to ensure the continuity and stability of the state recursion. The state transition model established in this way is a nonlinear model, and its position recursion introduces trigonometric function relationships, enabling the system to reasonably extrapolate the nonlinear motion trajectory of the target object under low-frequency sensor updates, providing a state recursion basis for subsequent prediction updates.

[0048] Through the processing in steps S101 to S103, the motion observation information is uniformly encapsulated into an augmented state vector containing position coordinates, velocity, heading angle, and heading angular velocity, so that the motion state of the target object at each perception update moment can be expressed in a structured and recursive manner. Based on the augmented state vector, a corresponding nonlinear state transition model is established, so that the state evolution of the target object is introduced with steering constraints and curvilinear motion laws, thereby providing a reliable motion modeling foundation for subsequent prediction extrapolation of the target object's pose between adjacent rendering frames to more closely resemble the real motion trajectory.

[0049] Step S2: Based on the state transition model corresponding to the nonlinear motion state space, the target object is predicted and updated to obtain the predicted state information of the target object at the current rendering time and the corresponding prediction covariance matrix.

[0050] Since the refresh rate of the vehicle screen is usually high (e.g., 60Hz), while the update frequency of sensor perception data is relatively low (e.g., 10Hz~20Hz), high frame rate prediction can be performed based on the extended Kalman filter (EKF) time update mechanism at each rendering frame time during the time gap between two adjacent frames of perception data, so as to continuously output the target prediction pose on the rendering side.

[0051] In an optional embodiment, step S2 specifically includes:

[0052] Step S201: Based on the state transition model, extrapolate the state of the previous rendering time to the current rendering time according to the time interval between adjacent rendering times, so as to obtain the predicted state information of the target object at the current rendering time. When the target object turns, the state extrapolation includes updating the position of the target object according to the nonlinear turning relationship.

[0053] When there is a time interval Δt between adjacent rendering moments, the motion state of the target object is recursively predicted using a state transition model (such as the constant turning rate and velocity model CTRV), that is, based on the optimal state estimate of the previous rendering moment. As the initial state input, the prior predicted state corresponding to the current rendering frame is deduced using the CTRV kinematic equations. This allows us to obtain the predicted state information of the target object at the current rendering moment, which is used to characterize the motion state of the target object at the current rendering moment. The components of the prior predicted state include the predicted position coordinates, predicted velocity, predicted heading angle, and predicted heading angular velocity of the target object.

[0054] When the target object changes direction in the k-th frame (e.g.) or When the value exceeds a preset threshold, a trigonometric integral relationship related to the heading change is introduced into the position update, making the position increment recursively derived with the heading angle to obtain a predicted position that better conforms to the turning motion law. Preferably, in At that time, position extrapolation update can satisfy:

[0055]

[0056] And update the heading angle simultaneously to reflect the turning process:

[0057]

[0058] By applying the aforementioned position update mechanism to the state extrapolation process, the system can make the predicted trajectory exhibit curvilinear extrapolation that changes with the heading under nonlinear motion conditions such as turning and changing lanes of the target object. This avoids trajectory drift caused by linear interpolation or straight-line extrapolation, and improves motion consistency and display stability under high frame rate rendering.

[0059] In another alternative implementation, when or When the threshold is less than zero, the CTRV model can be degenerated into uniform linear extrapolation to improve numerical stability and ensure the continuity of state extrapolation.

[0060] Step S202: Based on the covariance propagation mechanism, the covariance matrix corresponding to the predicted state of the target object is updated to obtain the predicted covariance matrix corresponding to the predicted state information.

[0061] The system maintains the covariance matrix for the target object at the previous rendering time. The covariance matrix is ​​used to characterize the uncertainty of the state prediction error of the target object. It can be initialized when the target object first enters the tracking based on the observation accuracy parameters output by the perception module or the preset sensor noise calibration parameters, and is continuously recursively maintained as the state is updated during subsequent rendering.

[0062] When the state extrapolation is completed according to step S201 and the predicted state is obtained Then, covariance prediction updates are performed synchronously to obtain the prediction covariance matrix corresponding to the prediction state information. Preferably, the update of the prediction covariance matrix follows the time update propagation relationship of the Extended Kalman Filter (EKF):

[0063]

[0064] in, This is the linearized coefficient matrix (e.g., Jacobian matrix) of the state transition model at the current prediction point, used to reflect the impact of state recursion on error propagation; For matrix The transpose of the matrix; The process noise matrix includes, but is not limited to, acceleration disturbances, steering disturbances, environmental disturbances, and model errors, and is used to characterize the prediction uncertainty introduced by unmodeled disturbances during the target's motion.

[0065] Through the aforementioned covariance propagation mechanism, the predicted covariance matrix increases accordingly with the time interval Δt between rendering frames. This reflects the gradual accumulation of the target object's state prediction error during the extrapolation process within the time gap before new sensor data arrives. Based on the predicted covariance matrix, variance information related to the target position can be further extracted to generate an uncertainty factor for visually indicating the reliability of the prediction results.

[0066] Through the processing in steps S201-S202, the motion state of the target object is recursively extrapolated based on the state transition model according to the time interval between adjacent rendering moments. This enables the continuous acquisition of predicted state information matching the current rendering moment, even when the sensor observes low-frequency updates or there are observation gaps. This ensures continuous rendering output of the target 3D model on high refresh rate display devices. Simultaneously, the covariance matrix is ​​predicted and updated based on the covariance propagation mechanism to obtain the predicted covariance matrix. This can simultaneously characterize the uncertainty evolution law of the prediction error increasing with the accumulation of extrapolation time, providing a quantitative basis for the uncertainty factor of subsequent extraction of position variance information and generation of visual cues. This improves the interpretability and safety of the prediction results while ensuring rendering smoothness.

[0067] Step S3: Based on the variance information related to the target object's position in the prediction covariance matrix, calculate the uncertainty factor that characterizes the visual uncertainty of the prediction state information.

[0068] In an optional embodiment, step S3 specifically includes:

[0069] Step S301: Extract the variance components corresponding to the position state of the target object from the predicted covariance matrix. The variance components include at least the variance components in the first position direction and the variance components in the second position direction.

[0070] Predicting covariance matrix The second-order statistical properties used to characterize the error in predicting state estimation are represented by their diagonal elements, which reflect the variance of each state component. From the prediction covariance matrix... The diagonal elements corresponding to the position state of the target object are read and used as the position variance components, wherein the variance components include at least the variance component of the first position direction. and the variance component of the second position direction The first and second position directions can correspond to the lateral / vertical directions, X / Y directions, or other preset orthogonal coordinate axes in the vehicle coordinate system or world coordinate system. If the target object's position is three-dimensional, the variance component of the third position direction can be further extracted. .

[0071] Step S302: Perform a synthesis operation on the variance components to obtain the position uncertainty metric.

[0072] Since the prediction error of the target object's position may differ in different directions, it can be improved by... and Performing a composition operation yields a single scalar that characterizes the overall position prediction uncertainty. The composition operation can be performed using methods such as weighted summation, square root extraction, maximum value selection, or root mean square calculation, to adapt to different rendering strategies and engineering requirements.

[0073] In one implementation, the location uncertainty metric A weighted summation method can be used, as shown below:

[0074]

[0075] in, and This is the weighting coefficient, which can be set according to actual needs. When In this case, it can be understood as a weighted synthesis of uncertainties in two directions.

[0076] If the location uncertainty metric If the variance of the three-dimensional position needs to be considered, it can be extended to:

[0077]

[0078] in, , and This is the weighting coefficient, which can be set according to actual needs.

[0079] In another implementation, the location uncertainty metric It can be calculated by the sum of squares and the square root:

[0080]

[0081] By combining the variance components of multiple directions into a position uncertainty metric, we can obtain the uncertainty input for subsequent mapping, avoiding the distortion of prompts caused by using only the variance of a single direction as the basis for rendering.

[0082] Step S303: Based on the location uncertainty metric, generate the original uncertainty value according to the preset uncertainty mapping rule, wherein the original uncertainty value increases as the location uncertainty metric increases.

[0083] The position uncertainty metric obtained in step S302 As input, and according to a preset mapping function The original value of uncertainty was calculated. The original uncertainty value increases with the increase of the position uncertainty metric. This mapping function can be a linear mapping, a nonlinear mapping, a piecewise mapping, etc., and the specific mapping rules can be adjusted according to actual rendering requirements.

[0084] For example, in one implementation, a linear mapping relationship is used, as shown in the following formula:

[0085]

[0086] in, This is the proportionality coefficient. This is the bias coefficient. Adjusted by... and The system can control the original value of uncertainty as a function of the position uncertainty metric. The rate of change and baseline value. If the location uncertainty is large, the original value of the uncertainty... The visual cues will increase in size to visually demonstrate the reliability of the target object prediction.

[0087] Furthermore, to avoid unstable visual effects caused by excessively large variations in the location uncertainty metric, a segmented mapping rule can be employed. For example, when Less than the preset threshold When, uncertainty enhancement is not triggered, but when Once this threshold is exceeded, the intensity increases linearly, as indicated by the following formula:

[0088]

[0089] The above methods can effectively control the uncertainty factor. Enlarging and smoothing the transition during the rendering process makes the rendering effect more in line with the user's visual needs, while avoiding unnecessary fluctuations.

[0090] Step S304: Perform a limiting process on the original uncertainty value to obtain an uncertainty factor within a preset range.

[0091] To ensure that the uncertainty factor remains within a reasonable range in the rendering control parameters of the rendering pipeline, based on preset upper and lower limits... and For the original value of uncertainty Amplitude limiting is applied to achieve the final result. It remains within this range. The clipping formula is as follows:

[0092]

[0093] in, and These are the preset minimum and maximum values ​​of the uncertainty factor, typically 0 and 1. Limiting ensures that the uncertainty factor does not exceed the predetermined renderable range, thus avoiding extreme visual effects such as excessive transparency or unreasonable edge blurring.

[0094] In some implementations, the limiting value and The system can be dynamically adjusted according to specific rendering needs to adapt to different rendering environments. For example, when the prediction uncertainty of the target object is high, the system can automatically increase the maximum value of the uncertainty factor to enhance the visual effect; conversely, if the prediction uncertainty of the target object is low, the range of the uncertainty factor can be reduced.

[0095] In one specific embodiment, based on the target object in and Variance of predicted location of direction and The uncertainty factor is calculated using the following formula. :

[0096]

[0097] in, It is a preset proportional coefficient. and The target object is in and The variance of the predicted location in the direction; the Clamp function is used to adjust the uncertainty factor. Limited to the range [0.0, 1.0].

[0098] This formula allows us to calculate the uncertainty factor based on the positional uncertainty of the target object. And limit it to the range of [0.0, 1.0] to ensure the stability and accuracy of the rendering effect.

[0099] Through steps S301-S304, positional uncertainty information can be extracted from the motion state of the target object, and an uncertainty factor can be generated based on this. This factor reflects the prediction uncertainty of the target object, and through amplitude limiting processing, it is ensured that the factor is always within a reasonable range, thereby guaranteeing the stability of the rendering effect. When the uncertainty factor is high, the system can significantly indicate the current uncertainty, thereby improving the driver's perception of the system status, avoiding blindly trusting inaccurate rendering results, and enhancing the driver's safety judgment of the environment.

[0100] Step S4: Based on the uncertainty factor and the predicted state information, render and display the 3D model of the target object through the graphics rendering pipeline, wherein at least one type of visual attribute of the 3D model is dynamically adjusted based on the uncertainty factor.

[0101] In an optional embodiment, step S4 specifically includes:

[0102] Step S401: Determine the spatial pose parameters of the 3D model on the rendering interface based on the predicted state information, and write the spatial pose parameters into the transformation matrix of the graphics rendering pipeline to drive the geometric rendering of the 3D model.

[0103] Using data such as the target object's position, velocity, and orientation extracted from the predicted state information, the spatial pose parameters (including position coordinates, rotation angle, and scale) of the target object at the current rendering moment are calculated. These spatial pose parameters are used to describe the target object's position and orientation in three-dimensional space.

[0104] For example, suppose the system obtains the current position of the target object through a prediction model. and rotation angle These data will then serve as spatial pose parameters to describe the 3D geometry of the target object. The calculated spatial pose parameters are written into the transformation matrix of the graphics rendering pipeline, driving the geometric rendering of the 3D model in the rendering interface. This process aligns the local coordinate system of the 3D model with the global coordinate system through the transformation matrix, ensuring that the target object can be correctly displayed in the rendering interface.

[0105] Transformation matrix The specific form can be represented as follows:

[0106]

[0107] in, Let be the rotation matrix of the target object, representing the object's pose (e.g., rotation angle). (corresponding matrix) This is the position vector of the target object.

[0108] This transformation matrix is ​​used to convert the target object from the local coordinate system to the global coordinate system, thereby ensuring the accurate position and orientation of the object in the rendering interface. In this way, the predicted state of the target object can be accurately mapped to the 3D rendering interface, ensuring that the geometry and spatial pose of the target object are displayed correctly, thus providing an accurate foundation for subsequent rendering effects.

[0109] Step S402: Input the uncertainty factor as a rendering control parameter into the shader of the graphics rendering pipeline to dynamically adjust at least one type of visual attribute of the 3D model.

[0110] In an optional embodiment, step S402 specifically includes:

[0111] Step S4021: Input the uncertainty factor as a uniform variable into the shader.

[0112] The calculated uncertainty factor Shaders are input as unified variables into the graphics rendering pipeline. As part of the rendering pipeline, shaders are responsible for computation and rendering within the graphics processing unit. When the uncertainty factor... Once input into the shader, it will serve as a key control parameter, affecting the rendering effect of the 3D model.

[0113] Step S4022: Based on the uncertainty factor, generate a visual control parameter corresponding to at least one of the visual attributes.

[0114] The visual attributes of the 3D model include at least one of the following: transparency attribute, edge blur width attribute, and edge halo intensity attribute; the visual control parameters include at least one of the following: transparency control parameter for adjusting transparency attribute, exponential control parameter for adjusting edge blur width attribute, and intensity control parameter for adjusting edge halo intensity attribute.

[0115] According to the uncertainty factor Generate visual control parameters related to at least one type of visual attribute (such as transparency, edge blur width, edge halo intensity, etc.). For example, based on an uncertainty factor, transparency control parameters, and / or edge blur width control parameters, and / or edge halo intensity control parameters can be generated, corresponding to the transparency, blur effect, and edge halo intensity of the target object, respectively. These visual control parameters are obtained by linear interpolation (such as the mix function) based on the uncertainty factor. Dynamic generation ensures that as the uncertainty factor increases, the visual effects of the target object (such as transparency, blurring, and edge light intensity) will increase accordingly, thereby enhancing the uncertainty cues.

[0116] Step S4023: Based on the visual control parameters, dynamically adjust the corresponding visual attributes so that the display effect of the 3D model presents a stronger uncertainty indication when the uncertainty factor increases.

[0117] In an optional embodiment, step S4023 specifically includes:

[0118] The transparency attribute of the 3D model is dynamically adjusted based on transparency control parameters so that the transparency of the 3D model increases when the uncertainty factor increases; and / or

[0119] The edge blur width attribute of the 3D model is dynamically adjusted based on the exponential control parameter, so that the exponential control parameter decreases when the uncertainty factor increases, thereby increasing the edge blur width of the 3D model; and / or

[0120] The intensity of the halo at the edge of the 3D model is dynamically adjusted based on the intensity control parameter so that the intensity control parameter increases when the uncertainty factor increases, thereby enhancing the halo display intensity in the edge region of the 3D model.

[0121] Dynamic adjustment of transparency properties: based on transparency control parameters Dynamically adjust the transparency property of the 3D model. Specifically, as the uncertainty factor... Increase the transparency control parameter This also increases the transparency of the 3D model. In this way, as the prediction uncertainty of the target object increases, the transparency of the 3D model also increases, creating a stronger indication of uncertainty.

[0122] Dynamic adjustment of edge blur width: based on edge blur control parameters Dynamically adjust the edge blur width attribute of the 3D model. Similar to transparency control, the edge blur width will also be adjusted based on the uncertainty factor. Increase. When the uncertainty factor increases, the edge blur width control parameter... As the width of the edge blur increases, the edge blur radius also increases, thus increasing the degree of blurring in the edge areas of the 3D model, enhancing the blurring effect of object edges, and simulating a visual effect with higher uncertainty. This adjustment method is used to enhance visual uncertainty cues.

[0123] Dynamic adjustment of edge halo intensity: based on edge light intensity control parameters Dynamically adjust the edge halo intensity attribute of the 3D model. Specifically, when the uncertainty factor... When increased, the edge light intensity control parameter This increases the intensity of the light, thereby enhancing the halo effect at the edges of the 3D model. By increasing the intensity of the edge light, more obvious visual cues are provided, enhancing the visual representation of uncertainty.

[0124] In this embodiment, the three visual attributes—transparency, edge blur width, and edge light intensity—can be adjusted independently or in combination. Specifically, the system will adjust the settings based on the uncertainty factor. The size of the adjustment determines whether to adjust the transparency, edge blur width, or edge light intensity, or all three simultaneously. These adjustments can be made individually or, in some cases, simultaneously. For example, in situations of high uncertainty, the system may simultaneously enhance the transparency, blur width, and light intensity of the 3D model to comprehensively alert the driver to the high uncertainty. In other cases, the system may adjust only the transparency or edge blur to maintain a smooth image.

[0125] Step S403: Perform shader-based pixel shading and transparency blending processing, and output a 3D rendering result that corresponds to the spatial pose parameters and whose visual attributes have been adjusted according to the uncertainty factor.

[0126] To apply the uncertainty factor to the dynamic adjustment of visual attributes of 3D models, such as transparency, edge blur width, and edge light intensity, and to achieve real-time reflection of uncertainty in rendering, the shader calculates the Fresnel term. First, set the camera's line-of-sight vector. and surface normal direction vector The formula for calculating the dot product is as follows:

[0127]

[0128] Fresnel terms are calculated using the following formula:

[0129]

[0130] After calculating the Fresnel terms, the shader performs a smoothing process (using the smoothstep function) to reduce the harshness of the edge transitions, resulting in a softer rendering effect and smoother edge transitions.

[0131]

[0132] Shader combined with transparency control parameters Edge blur control parameters Edge light intensity control parameters and Fresnel The color and transparency of the target 3D model are synthesized. Specifically, the edge light color, Fresnel term, and light intensity control parameters work together to affect the brightness of the edge areas, while the base color is superimposed on the edge light color to ultimately output the target pixel color and transparency. The relevant calculation formulas are as follows:

[0133]

[0134]

[0135] in, This indicates the edge light color, used to display the highlight area at the edge of an object; This represents the final edge light color value; The base color of an object, usually the color of the object itself; This represents the final rendered color, which includes the RGB components of the color.

[0136] Basic transparency control amount With uncertainty factor Increase and decrease:

[0137]

[0138] `gl_FragColor` is a built-in variable in OpenGL used to output the calculation result to the frame buffer, which is then displayed on the screen. The `vec4` function is a built-in function in the OpenGL Shading Language (GLSL) used to create a four-dimensional vector. The final output pixel color and transparency are obtained by combining them in the following way:

[0139]

[0140] Through steps S401 to S403, the system can dynamically adjust the visual effects of the 3D model based on the predicted state information and uncertainty factor of the target object, optimizing the accuracy and intuitiveness of the rendering display. When the uncertainty factor increases, visual attributes such as the transparency, edge blur width, and edge light intensity of the 3D model are adjusted in real time to present a stronger indication of uncertainty. By adjusting these visual attributes, not only is a smooth transition in the rendering result ensured during display, avoiding abrupt changes in the image, but the driver can also intuitively perceive changes in the system's trust level, improving their responsiveness to environmental changes and helping them to identify potential risks in a timely manner. Therefore, in this way, while improving rendering accuracy, the interpretability of visual feedback and driving safety are enhanced.

[0141] Step S5: When new perception data of the target object is obtained, the predicted state information is fused and updated based on the new perception data, and the rendering result of the 3D model is smoothly transitioned based on the correction amount generated by the fusion update to suppress rendering jumps.

[0142] When sensors (such as radar, cameras, etc.) acquire new sensing data First, calculate the Kalman gain. The new sensed data is compared with the previously predicted state using Kalman gain. Fusion yields the optimal posterior estimate. This updates the target object's state information. Specifically, the target object's position, velocity, and other relevant state variables are updated to reflect the latest perception data. The calculation formula is as follows:

[0143]

[0144] Where H is the observation matrix.

[0145] After fusing and updating the state estimates, it is also necessary to update the corresponding covariance matrix. At this point, the covariance matrix will change, typically becoming smaller with the introduction of perceived data, indicating a reduction in uncertainty. The updated covariance matrix represents the degree of uncertainty in the target object's state. Its calculation formula is as follows:

[0146]

[0147] in, It is the covariance matrix of the previous time step.

[0148] After the state update, the rendering of the 3D model needs a smooth transition to avoid a direct jump from the predicted position to the observed position. This process is achieved through the following steps... Spherical interpolation (using the Slerp function) is used in frame rendering to smooth out corrections, allowing the 3D model to "gracefully" return to its new, realistic position during rendering. Spherical interpolation ensures that the transition is not abrupt, avoiding unnatural jump effects. The spherical interpolation formula can be expressed as follows:

[0149]

[0150] in, and They are two direction vectors. It is the interpolation factor. It is the angle between two vectors. Generate a smooth transition between two vectors.

[0151] Through the above process, step S5 not only achieves accurate fusion of the predicted state of the target object, but also optimizes the rendering effect of the 3D model through smooth transition technology, thereby avoiding screen jumps caused by the difference between prediction and observation, and ensuring the continuity and naturalness of the rendering results.

[0152] Figure 2 The processing flow of a vehicle-mounted 3D scene rendering method based on nonlinear motion state space is illustrated. This flow includes the following stages:

[0153] System initialization: Upon system startup, the nonlinear motion state space is first initialized, and initial state values ​​are set for the target object. The system constructs an augmented state vector of the target object based on known initial data (such as position, velocity, heading angle, etc.), and assigns initial values ​​to each state variable (such as position, velocity, heading angle).

[0154] Determining the Type of Trigger Signal: Upon receiving new sensor data, the first step is to determine the type of trigger signal. This determination, based on the data type of the received signal, determines the next processing step. There are two main types: When a sensor (such as radar or a camera) acquires new sensing data, it is identified as a sensor data update signal, and the process enters the extended Kalman filter measurement update stage. By calculating the Kalman gain, the new observation data is fused with the current prediction state, and the augmented state vector (such as position, velocity, heading angle, and heading angular velocity) is corrected. Simultaneously, the covariance matrix is ​​updated to reflect the latest prediction error, thereby reducing prediction uncertainty.

[0155] State Prediction: When no new sensor data arrives, it is identified as a rendering frame trigger update signal, and the extended Kalman filter time prediction stage begins. At this point, the state of the target object is predicted based on a nonlinear motion state space and a state transition model (such as a constant turning rate and velocity model). Between each rendering frame, the state estimate from the previous moment (i.e., the predicted state of the target object) is extrapolated to the current rendering moment. By calculating the time interval Δt, kinematic equations are used to deduce the target object's position, velocity, heading angle, and other state information, providing necessary data support for subsequent rendering updates.

[0156] Visual uncertainty calculation: The predicted state of the target object is updated using algorithms such as Kalman filtering. Based on the variance information related to the target position in the prediction covariance matrix, a visual uncertainty factor is calculated. This uncertainty factor reflects the reliability of the target state and is input into the graphics rendering pipeline as a rendering control parameter. The system dynamically adjusts the visual attributes of the target object (such as transparency, edge blur width, and edge light intensity) according to the magnitude of the uncertainty factor, thus intuitively demonstrating the reliability of the target object to the driver. For example, a larger uncertainty factor may lead to increased transparency of the 3D model, indicating uncertainty about the object.

[0157] Extracting Spatial Pose Parameters: This involves extracting the spatial pose parameters of the target object, such as position, velocity, and heading angle, from the augmented state vector. Spatial pose parameters are crucial information representing the target object's position and orientation in three-dimensional space. Extracting these parameters provides accurate spatial information about the target object for subsequent graphics rendering pipelines.

[0158] Graphics Rendering and Visual Effects Adjustment: Based on the aforementioned predicted state information and uncertainty factor, the state of the target object is passed to the graphics rendering pipeline for real-time rendering. By dynamically adjusting visual attributes such as the target object's transparency, edge blurring effect, and edge light intensity, the reliability of the target object is visually displayed during the rendering process. For example, increased transparency can indicate the system's predictive uncertainty, while edge blurring enhances the visual effect of this uncertainty indication.

[0159] Smooth Transition and Rendering Optimization: To avoid abrupt changes or jumps during rendering, techniques such as spherical interpolation are used to smoothly transition the state of the target object. Whenever the state of the target object changes, the system updates the rendering results frame by frame, ensuring a smooth transition between the predicted state and the measured result. This allows drivers to experience a natural and fluid display when viewing the 3D scene, avoiding abrupt changes in the image caused by state updates or measurement errors.

[0160] Final Rendering and Output Display: After all rendering calculations are completed, the final 3D rendering result is output to the in-vehicle display screen. This rendering result includes the motion state, shape, and visual effects of the target object, while dynamically displaying the uncertainty of the target object's state. This helps the driver understand the motion state of the target object and the reliability of the system's predictions in real time, thereby improving driving safety.

[0161] This processing flow enables the maintenance of high frame rate 3D rendering even with low-frequency sensor data updates. By dynamically adjusting visual effects, it can display the predictive reliability and uncertainty of target objects in real time, thereby improving the driver's environmental perception and enhancing driving safety.

[0162] In summary, the in-vehicle 3D scene rendering method provided in this application acquires the perception data of the target object and constructs a nonlinear motion state space representing its motion process. It then predicts and updates the target object based on a state transition model between adjacent rendering frames. This allows for continuous output of predicted state information matching the current rendering time even with low sensor update frequencies, enabling continuous rendering and display of the target object on the vehicle's infotainment system at high refresh rates. Simultaneously, an uncertainty factor is calculated based on the variance information related to the target position in the prediction covariance matrix, and this uncertainty factor is introduced into the graphics rendering pipeline to dynamically adjust at least one type of visual attribute of the 3D model. When external environmental conditions deteriorate, leading to increased sensor observation noise, or when there is a delay in observation updates causing prediction errors to accumulate over time, the scene reconstruction rendering result will adaptively reduce the display clarity and visibility of the target model based on the uncertainty factor, resulting in a blurred, weakened, or transparent visual effect. This alerts the driver that the system's current confidence level has decreased, preventing blind trust in the screen rendering results and improving safety and interpretability in human-machine co-driving scenarios. Furthermore, upon acquiring new perception data, by performing a fusion update on the predicted state information and smoothly transitioning the rendering result based on the correction amount, the screen jumps during the switch from the predicted trajectory to the observed trajectory can be suppressed, achieving a flexible connection between observation updates and rendering output. Therefore, this application ensures display smoothness while also taking into account rendering accuracy and reliable expression capabilities under nonlinear motion conditions, making it suitable for application scenarios with high requirements for real-time performance and stability, such as in-vehicle 3D scene reconstruction rendering.

[0163] The steps of the various methods described above are only for clarity. In practice, they can be combined into one step or some steps can be split into multiple steps. As long as they include the same logical relationship, they are all within the scope of protection of this application. Adding insignificant modifications or introducing insignificant designs to the algorithm or process, but without changing the core design of the algorithm and process, are also within the scope of protection of this application.

[0164] Furthermore, some embodiments of this application also provide an electronic device. The electronic device can be various forms of digital computer, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, etc. The electronic device can also be various forms of mobile devices, such as cellular phones, smartphones, wearable devices, and other similar computing devices.

[0165] The electronic device includes: one or more processors; and a memory storing computer program instructions, which, when executed, cause the processor to perform a vehicle-mounted 3D scene rendering method as provided in any one or more of the above embodiments. Figure 3An exemplary structural diagram of the electronic device is disclosed. The electronic device includes one or more processors 1101, a memory 1102, and interfaces for connecting the components, including high-speed interfaces and low-speed interfaces. The components are interconnected via different buses and can be mounted on a common motherboard or otherwise installed as needed. The processors can process instructions executed within the electronic device, including instructions stored in or on memory to display graphical information of a GUI on an external input / output device (such as a display device coupled to the interface). In some other embodiments, multiple processors and / or multiple buses can be used with multiple memories and multiple memory modules, if desired. Similarly, multiple electronic devices can be connected, each providing some of the necessary operations. The components, their connections and relationships, and their functions shown herein are merely examples and are not intended to limit the implementation of the present application described and / or claimed herein.

[0166] The electronic device may further include an input device 1103 and an output device 1104. The processor 1101, memory 1102, input device 1103, and output device 1104 may be connected via a bus or other means. Figure 3 Taking the example of a connection between China and Israel via a bus.

[0167] Input device 1103 can receive input numerical or character information, and generate key signal inputs related to user settings and function control of the electronic device, such as a touch screen, keypad, mouse, trackpad, touchpad, joystick, one or more mouse buttons, trackball, joystick, etc. Output device 1104 may include a display device, auxiliary lighting device (e.g., LED), and haptic feedback device (e.g., vibration motor). The display device may include, but is not limited to, a liquid crystal display, a light-emitting diode display, and a plasma display. In some embodiments, the display device may be a touch screen.

[0168] To provide interaction with the user, the electronic device can be a computer. The computer has: a display device (e.g., a cathode ray tube or LCD monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback); and input from the user can be received in any form (e.g., voice input or tactile input).

[0169] In this embodiment, a computer-readable medium stores a computer program / instructions, which, when executed by a processor, implement a vehicle-mounted 3D scene rendering method provided in any one or more of the above embodiments. The computer-readable medium may be included in the electronic device described in the above embodiments; or it may exist independently and not assembled into that device. The computer-readable medium carries one or more computer-readable instructions.

[0170] The memory 1102 can serve as a non-transitory computer-readable storage medium, used to store non-transitory software programs, non-transitory computer-executable programs, and modules. The processor 1101 executes various functional applications and data processing of the server by running the non-transitory software programs, instructions, and modules stored in the memory 1102, thereby implementing the program instructions / modules corresponding to the methods provided in any one or more of the embodiments described above in this application.

[0171] The memory 1102 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created based on the use of the electronic device. Furthermore, the memory 1102 may include high-speed random access memory and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, the memory 1102 may optionally include memory remotely located relative to the processor 1101, and these remote memories can be connected to the electronic device via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0172] It should be noted that the computer-readable medium described in this application can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. Computer-readable media can be, for example, but not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatuses, or devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to, electrical connections having one or more wires, portable computer disks, hard disks, random access memory, read-only memory, erasable programmable read-only memory, optical fibers, portable compact disk read-only memory, optical storage devices, magnetic storage devices, or any suitable combination thereof. In this application, a computer-readable medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0173] Computer-readable media include permanent and non-permanent, removable and non-removable media, which can store information by any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory, static random access memory, dynamic random access memory, other types of random access memory, read-only memory, electrically erasable programmable read-only memory, flash memory or other memory technologies, read-only optical discs, digital versatile optical discs or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transfer medium that can be used to store information accessible by a computing device.

[0174] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including local area networks (LANs) or wide area networks (WANs), or it can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0175] In the above embodiments, all or part of the implementation can be achieved through software, hardware, firmware, or any combination thereof. For example, it can be implemented using an application-specific integrated circuit (ASIC), a general-purpose computer, or any other similar hardware device. In some embodiments, the software program of this application can be executed by a processor to implement the above steps or functions. Similarly, the software program of this application (including related data structures) can be stored in a computer-readable recording medium, such as RAM memory, magnetic or optical drives, floppy disks, and similar devices. In addition, some steps or functions of this application can be implemented in hardware, for example, as circuitry that cooperates with a processor to perform the various steps or functions.

[0176] The computer program product provided in this application includes one or more computer programs / instructions. When executed by a processor, these computer programs / instructions generate, in whole or in part, the processes or functions described in this application. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium may be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state drive), etc.

[0177] The flowcharts or block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of devices, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-specific system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0178] The scope of this application is defined by the appended claims rather than the foregoing description, and is therefore intended to encompass all variations falling within the meaning and scope of equivalents of the claims. No reference numerals in the claims should be construed as limiting the scope of the claims. Furthermore, it is clear that the word "comprising" does not exclude other elements or steps, and the singular does not exclude the plural. Terms such as "first," "second," etc., are used only to distinguish descriptions and do not indicate any particular order, nor should they be construed as indicating or implying relative importance.

[0179] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily made by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims, and the above embodiments should be regarded as exemplary and non-limiting.

Claims

1. A method for rendering a vehicle-mounted 3D scene, characterized in that, include: Acquire the perception data of the target object, and construct a nonlinear motion state space characterizing the motion process of the target object based on the perception data; Based on the state transition model corresponding to the nonlinear motion state space, the target object is predicted and updated to obtain the predicted state information of the target object at the current rendering time and the corresponding prediction covariance matrix. Based on the variance information related to the position of the target object in the predicted covariance matrix, an uncertainty factor characterizing the visual uncertainty of the predicted state information is calculated. Based on the uncertainty factor and the predicted state information, the three-dimensional model of the target object is rendered and displayed through the graphics rendering pipeline, wherein at least one type of visual attribute of the three-dimensional model is dynamically adjusted based on the uncertainty factor. When new perception data of the target object is obtained, the predicted state information is fused and updated based on the new perception data, and the rendering result of the three-dimensional model is smoothly transitioned based on the correction amount generated by the fusion update to suppress rendering jumps.

2. The vehicle-mounted 3D scene rendering method according to claim 1, characterized in that, The step of constructing a nonlinear motion state space characterizing the motion process of the target object based on the perceived data includes: Extract motion observation information related to the target object from the sensing data; An augmented state vector is constructed based on the motion observation information to characterize the motion state of the target object. The augmented state vector includes at least: position coordinates, velocity, heading angle, and heading angular velocity. A nonlinear state transition model corresponding to the augmented state vector is established to form the nonlinear motion state space.

3. The vehicle-mounted 3D scene rendering method according to claim 1, characterized in that, The step of predicting and updating the target object based on the state transition model corresponding to the nonlinear motion state space to obtain the predicted state information of the target object and the corresponding prediction covariance matrix at the current rendering time includes: Based on the state transition model, the state of the previous rendering time is extrapolated to the current rendering time according to the time interval between adjacent rendering times, so as to obtain the predicted state information of the target object at the current rendering time. When the target object turns, the state extrapolation includes updating the position of the target object according to the nonlinear turning relationship. Based on the covariance propagation mechanism, the covariance matrix corresponding to the predicted state of the target object is predicted and updated to obtain the prediction covariance matrix corresponding to the predicted state information.

4. The vehicle-mounted 3D scene rendering method according to claim 1, characterized in that, The step of calculating an uncertainty factor characterizing the visual uncertainty of the predicted state information based on the variance information related to the position of the target object in the predicted covariance matrix includes: Extract the variance components corresponding to the position state of the target object from the predicted covariance matrix. The variance components include at least the variance components in the first position direction and the variance components in the second position direction. The variance components are synthesized to obtain a measure of positional uncertainty. Based on the location uncertainty metric, an initial uncertainty value is generated according to a preset uncertainty mapping rule, wherein the initial uncertainty value increases as the location uncertainty metric increases; A limiting process is performed on the original uncertainty value to obtain an uncertainty factor within a preset range.

5. The vehicle-mounted 3D scene rendering method according to claim 1, characterized in that, The step of rendering and displaying the 3D model of the target object through a graphics rendering pipeline based on the uncertainty factor and the predicted state information, wherein the step of dynamically adjusting at least one type of visual attribute of the 3D model based on the uncertainty factor includes: Based on the predicted state information, the spatial pose parameters of the 3D model in the rendering interface are determined, and the spatial pose parameters are written into the transformation matrix of the graphics rendering pipeline to drive the geometric rendering of the 3D model. The uncertainty factor is input as a rendering control parameter to the shader of the graphics rendering pipeline to dynamically adjust at least one type of visual attribute of the 3D model. Perform pixel shading and transparency blending based on the shader, and output a 3D rendering result that corresponds to the spatial pose parameters and whose visual attributes have been adjusted according to the uncertainty factor.

6. The vehicle-mounted 3D scene rendering method according to claim 5, characterized in that, The step of inputting the uncertainty factor as a rendering control parameter into the shader of the graphics rendering pipeline to dynamically adjust at least one type of visual attribute of the 3D model includes: The uncertainty factor is input as a uniform variable into the shader; Based on the uncertainty factor, generate visual control parameters corresponding to at least one of the visual attributes; Based on the aforementioned visual control parameters, the corresponding visual attributes are dynamically adjusted so that the display effect of the 3D model presents a stronger uncertainty indication when the uncertainty factor increases.

7. The vehicle-mounted 3D scene rendering method according to claim 6, characterized in that, The three-dimensional model has at least one type of visual attribute, including at least one of transparency attribute, edge blur width attribute, and edge halo intensity attribute; the visual control parameters include at least one of transparency control parameters for adjusting the transparency attribute, exponential control parameters for adjusting the edge blur width attribute, and intensity control parameters for adjusting the edge halo intensity attribute. The step of dynamically adjusting the corresponding visual attributes based on the visual control parameters includes: The transparency attribute of the 3D model is dynamically adjusted based on the transparency control parameters so that the transparency of the 3D model increases when the uncertainty factor increases; and / or The edge blur width attribute of the 3D model is dynamically adjusted based on the exponential control parameter, so that the exponential control parameter decreases when the uncertainty factor increases, thereby increasing the edge blur width of the 3D model; and / or The intensity of the halo at the edge of the 3D model is dynamically adjusted based on the intensity control parameter, so that the intensity control parameter increases when the uncertainty factor increases, thereby enhancing the halo display intensity at the edge of the 3D model.

8. An electronic device, characterized in that, The electronic device includes: One or more processors; and a memory storing computer program instructions, which, when executed, cause the processors to perform the vehicle-mounted 3D scene rendering method as described in any one of claims 1-7.

9. A computer-readable storage medium having a computer program and / or instructions stored thereon, characterized in that, When the computer program and / or instructions are executed by the processor, they implement the vehicle-mounted 3D scene rendering method as described in any one of claims 1-7.

10. A computer program product comprising a computer program and / or instructions, characterized in that, When the computer program and / or instructions are executed by the processor, they implement the vehicle-mounted 3D scene rendering method as described in any one of claims 1-7.

Images