Cockpit 4D image generation system based on end-cloud cooperation

Abstract

This invention relates to the field of cockpit imaging technology and discloses a cockpit 4D image generation system based on edge-cloud collaboration. The system comprises three main modules: edge-side data acquisition, cloud-side processing, and edge-side image generation. The edge-side data acquisition module includes a multi-sensor fusion unit that can generate multi-source cockpit data. After receiving this data, the cloud-side processing module outputs image prediction signals and risk warning signals through an image prediction unit. An image optimization unit processes the data and prediction signals to obtain the optimal rendering signal, and a rendering control unit defines constraints and outputs rendering parameters. In the edge-side image generation module, an adaptive rendering execution unit adjusts parameters according to the optimal rendering signal, the cockpit equipment control unit adjusts equipment based on parameters and multi-source data, and receives real-time status signals. A user interaction module processes warning and prediction signals and outputs a feedback index. This system optimizes the cockpit 4D image generation effect and intelligent experience through edge-cloud collaboration and multi-source data fusion.

Description

Technical Field

[0001] This invention relates to the field of cockpit imaging technology, specifically to a cockpit 4D image generation system based on edge-cloud collaboration. Background Technology

[0002] With the increasing demand for intelligent vehicles and upgraded cabin experiences, drivers and passengers are placing higher demands on the image presentation and interactive experience within the cabin environment. Current cabin imaging systems often rely on single sensors to collect data, such as using cameras to acquire visual information or millimeter-wave radar to capture distance data. These single data sources cannot comprehensively reflect the internal and external environment of the cabin and the state of the drivers and passengers, easily leading to information loss or bias during image generation, thus affecting the accuracy and usability of the images.

[0003] In the data processing stage, traditional cockpit imaging systems often employ a single mode: local processing on the device side or centralized processing in the cloud. Relying solely on device side processing, limited by the computing power and storage resources of in-vehicle equipment, makes it difficult to perform in-depth analysis and complex calculations on large-scale, multi-dimensional cockpit data. This results in insufficient image prediction accuracy and an inability to promptly identify potential risks, such as misjudgments of driver and passenger fatigue or inaccurate distance estimations of external obstacles. Conversely, relying solely on cloud-side processing is susceptible to network bandwidth and latency issues during data transmission. When network signals are unstable, data transmission efficiency drops significantly, not only delaying image generation but also potentially causing delays in real-time scenarios, such as emergency braking, where image warnings may not respond promptly, impacting driving safety.

[0004] Current cockpit imaging systems lack specific constraint definitions and adaptive adjustment mechanisms in their rendering process. Rendering parameters are mostly fixed settings, failing to dynamically optimize based on real-time data from inside and outside the cockpit, such as changes in light intensity, occupant seating posture, and equipment operating status. This results in image quality distortion and poor display effects in different scenarios. Furthermore, the linkage between user interaction and image generation / risk warning is weak. Passengers cannot promptly provide feedback to the system to adjust image quality, nor can they quickly obtain and respond to risk warnings, leading to a poor overall user experience that fails to meet the personalized and intelligent cockpit imaging needs. These problems hinder current cockpit imaging systems from achieving accurate, efficient, and real-time 4D image generation, thus limiting further improvements in the intelligent cockpit experience. Summary of the Invention

[0005] The purpose of this invention is to provide a cockpit 4D image generation system based on edge-cloud collaboration to solve the problems mentioned in the background art.

[0006] To achieve the above objectives, the present invention provides a cockpit 4D image generation system based on edge-cloud collaboration, the system comprising:

[0007] The end-side data acquisition module includes a multi-sensor fusion unit, which generates multi-source cockpit data.

[0008] The cloud-side processing module receives the multi-source cockpit data and includes an image prediction unit, an image optimization unit, and a rendering control unit. The image prediction unit processes the multi-source cockpit data and outputs an image prediction signal and a risk warning signal. The image optimization unit processes the multi-source cockpit data and the image prediction signal to obtain the optimal rendering signal. The rendering control unit defines rendering constraints based on the multi-source cockpit data and outputs rendering parameters.

[0009] The edge-side image generation module includes an adaptive rendering execution unit, a cockpit equipment control unit, and a user interaction module. The adaptive rendering execution unit adjusts rendering parameters according to the optimal rendering signal. The cockpit equipment control unit adjusts cockpit equipment parameters according to the rendering parameters and the multi-source cockpit data, and collects real-time status signals. The user interaction module processes the risk warning signal and the image prediction signal and outputs a user feedback index.

[0010] Preferably, the end-side data acquisition module includes a distributed sensor unit and an external environment detection unit. The end-side data acquisition module is deployed inside and outside the cockpit and collects sensor signals. The external environment detection unit acquires environmental data around the vehicle through satellite positioning technology and forms an environmental signal. The multi-sensor fusion unit receives the sensor signals and the environmental signal to achieve heterogeneous data fusion and form the multi-source cockpit data. The multi-source cockpit data includes predicted multi-source data, optimized multi-source data, and regulated multi-source data.

[0011] Preferably, the image prediction unit includes a time series prediction model and an evidence fusion model based on a deep neural network. The prediction multi-source data includes user behavior data, vehicle motion data, external environment change data, historical image parameters, and historical image quality data. The user behavior data, the historical image parameters, the vehicle motion data, the external environment change data, and the historical image quality data are input into the time series prediction model and output the image prediction signal. The improved evidence fusion model fuses the user behavior data, vehicle motion data, and external environment change data and outputs the risk warning signal.

[0012] Preferably, the time series prediction model includes a time series feature extraction submodule, a multimodal fusion submodule, a risk quantification submodule, and a prediction result visualization module. The time series feature extraction submodule performs wavelet transform on the vehicle motion data and extracts the energy change features of the frequency band that meets the precursor frequency band range of the motion pattern as the precursor signal of motion. The multimodal fusion submodule uses an attention-weighted recurrent neural network to process the user behavior data, the vehicle motion data, and the external environment change data and outputs the data to the risk quantification submodule. The risk quantification submodule defines a risk index algorithm. The prediction result visualization module generates a dynamic risk map of the risk index based on color coding, using different colored blocks to represent different risk levels.

[0013] Preferably, the image optimization unit processes the optimized multi-source data and the image prediction signal using a rendering-quality nonlinear relationship model, outputs a rendering signal, and processes the rendering signal using an optimization algorithm to obtain the optimal rendering signal; the optimized multi-source data includes user preference data and rendering material performance data, and the rendering-quality nonlinear relationship model is constructed using the user preference data, the image prediction signal, and the rendering material performance data.

[0014] Preferably, the image optimization unit further includes a rendering performance dynamic matching library, a scene complexity analysis submodule, and a rendering path planning submodule. The rendering performance dynamic matching library stores historical rendering performance data of historical rendering materials. The scene complexity analysis submodule calculates the scene complexity coefficient based on the rendering parameters and the rendering signal. The rendering path planning submodule obtains the optimal rendering signal through an optimization algorithm based on the complexity coefficient, the rendering-quality nonlinear relationship model, and the rendering material performance data.

[0015] Preferably, the rendering control unit defines rendering constraints based on the multi-source data, outputs rendering parameters, and dynamically adjusts the rendering parameters to ensure they are not lower than the minimum value through model prediction control. The multi-source data includes real-time user attention signals and device status signals, which define the rendering constraints.

[0016] Preferably, the user interaction module includes a 3D visualization platform and a risk warning engine. The 3D visualization platform receives the multi-source cockpit data and the real-time status signal and dynamically renders the image generation process. The risk warning engine processes the risk warning signal and the image prediction signal according to a multi-attribute decision algorithm and outputs a user feedback index. When the user feedback index exceeds the feedback index threshold, the user interaction module controls the adaptive rendering execution unit to perform emergency rendering. The 3D visualization platform supports arbitrary angle and perspective analysis, can display the spatial relationship between the image and the user's line of sight in real time, and simulates image changes under different rendering schemes based on a physics engine.

[0017] Preferably, the risk warning engine includes a multi-level warning mechanism with color states: yellow warning state, orange warning state, and red warning state. When the image quality in the image prediction signal drops to a first quality threshold, the yellow warning state is activated, reducing the rendering device's operating speed to a warning operating speed. When the image quality in the image prediction signal drops to a second quality threshold, the orange warning state is activated, suspending the rendering device's operation and controlling the adaptive rendering execution unit to perform emergency rendering. When the image quality in the image prediction signal drops to a third quality threshold, the red warning state is activated, and the risk warning engine issues an emergency fault signal and controls the end-to-cloud collaborative cockpit 4D image generation system to shut down urgently.

[0018] Preferably, the optimal rendering signal includes, but is not limited to, rendering region coordinates, rendering type priority, rendering pressure range, and rendering path planning. The adaptive rendering execution unit includes an intelligent rendering pump and an integrated online rendering quality detection device. The intelligent rendering pump can identify the optimal rendering signal and adjust the rendering flow and rendering pressure as needed. The integrated online rendering quality detection device receives the optimal rendering signal and adjusts the quality parameters of the rendered object in real time based on optical detection methods. The real-time status signal includes, but is not limited to, device temperature, device humidity, device voltage, device depth, rendering pressure, and expansion radius.

[0019] Compared with the prior art, the beneficial effects of the present invention are:

[0020] By using a multi-sensor fusion unit in the edge data acquisition module to generate multi-source cockpit data, the problem of incomplete data acquisition from traditional single sensors is effectively solved. The multi-source data covers multiple dimensions of information such as the internal and external environment of the cockpit, the status of the driver and passengers, and equipment operating parameters. This provides richer and more complete basic data for subsequent image generation, avoiding image deviations caused by missing information and making the generated 4D images more closely match the actual cockpit scene.

[0021] At the data processing level, the system adopts an edge-cloud collaborative processing model, fully leveraging the respective advantages of both the edge and cloud sides. The edge side is responsible for data acquisition and preliminary processing, reducing unnecessary data transmission and network dependence. The image prediction unit in the cloud-side processing module performs in-depth analysis of multi-source cockpit data, enabling more comprehensive mining of potential information and generating more accurate image prediction signals. Simultaneously, it promptly identifies potential risks inside and outside the cockpit and outputs risk warning signals, helping passengers perceive dangers in advance. The image optimization unit combines multi-source data and image prediction signals for comprehensive processing, optimizing image details and improving image quality to meet the image requirements of different scenarios, ensuring that the generated optimal rendering signal better matches the actual application scenario. The rendering control unit defines rendering constraints based on multi-source cockpit data, ensuring that the rendering process has clear reference points and that the output rendering parameters are more targeted, avoiding the poor rendering effects caused by traditional fixed parameters.

[0022] The addition of an edge-side image generation module further enhances the system's usability and interactivity. The adaptive rendering execution unit adjusts rendering parameters based on the optimal rendering signal, enabling real-time adaptation to changes in the cockpit's internal and external environment. For example, when the cabin's lighting intensity increases, it automatically adjusts image brightness and contrast to ensure stable image display. When complex road conditions occur externally, it optimizes image resolution and dynamic range to highlight key information. The cockpit equipment control unit adjusts cockpit equipment parameters based on rendering parameters and multi-source cockpit data, such as adjusting the display angle and seat position to match image display requirements. Simultaneously, it collects real-time equipment status signals, forming a data loop to ensure that the equipment's operating status matches the image generation requirements, avoiding usage problems caused by mismatches between equipment parameters and image quality.

[0023] The user interaction module processes risk warning signals and image prediction signals and outputs a user feedback index, strengthening the interaction between users and the system. Drivers and passengers can use this module to promptly obtain risk warning information, such as fatigue driving alerts and external obstacle warnings, quickly grasping the cabin safety status. Simultaneously, the user feedback index transmits the driver's and passengers' evaluation of the image quality to the system, providing direction for subsequent image optimization. This allows the system to gradually adjust its image generation strategy based on user needs, meeting the personalized needs of different drivers and passengers and improving the overall user experience. Furthermore, the edge-cloud collaborative mode balances the allocation of computing resources. The edge handles some data processing to reduce the pressure on the cloud, while the powerful computing capabilities of the cloud ensure complex computational needs are met. This avoids the limitations of a single processing mode, improves the overall data processing efficiency and response speed of the system, ensures the real-time and accurate generation of 4D images, and further optimizes the intelligent cabin experience. Attached Figure Description

[0024] Figure 1 This is a schematic diagram illustrating the working principle of the cockpit 4D image generation system based on edge-cloud collaboration described in this invention.

[0025] Figure 2 A flowchart illustrating the data processing of the edge data acquisition module;

[0026] Figure 3 A flowchart illustrating the internal processing of a time series forecasting model;

[0027] Figure 4 A flowchart for internal optimization within the image optimization unit;

[0028] Figure 5 A flowchart for user interaction module processing and feedback. Detailed Implementation

[0029] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0030] Please see Figure 1 The present invention provides a cockpit 4D image generation system based on edge-cloud collaboration, the system comprising: an edge data acquisition module, a cloud processing module, and an edge image generation module.

[0031] The edge data acquisition module generates multi-source cockpit data through a multi-sensor fusion unit. The cloud-side processing module receives the multi-source cockpit data and outputs image prediction signals and risk warning signals through an image prediction unit. The image optimization unit processes the multi-source cockpit data and image prediction signals to obtain the optimal rendering signal. The rendering control unit defines rendering constraints based on the multi-source cockpit data and outputs rendering parameters. The edge image generation module includes an adaptive rendering execution unit, a cockpit equipment control unit, and a user interaction module. The adaptive rendering execution unit adjusts rendering parameters based on the optimal rendering signal. The cockpit equipment control unit adjusts cockpit equipment parameters and collects real-time status signals based on the rendering parameters and multi-source cockpit data. The user interaction module processes risk warning signals and image prediction signals and outputs a user feedback index.

[0032] Example 1: See Figure 2The distributed sensor units of the end-side data acquisition module are meticulously arranged in multiple key locations both inside and outside the cabin. Inside the cabin, sensors are mainly distributed in areas frequently touched by the driver and passengers. For example, a pressure sensor is installed under the steering wheel to monitor the driver's hand movements, weight distribution sensors are embedded in the seats to detect changes in occupant posture, infrared sensors are arranged around the instrument panel to capture user gesture commands, and cameras are installed in the roof lining to record facial expressions and eye movements. Outside the cabin, the sensor system is more complex. Multiple radar probes are installed on the front and rear bumpers to detect the distance and relative speed of surrounding vehicles, ultrasonic sensors around the vehicle body are responsible for near-range obstacle recognition, the main camera inside the windshield continuously captures road image information, and the roof antenna integrated module receives satellite navigation signals. These sensors continuously generate raw data streams, forming rich sensor signals covering multiple dimensions such as user behavior, vehicle motion parameters, and equipment status.

[0033] The external environment detection unit, a crucial component of the edge data acquisition module, utilizes satellite positioning technology. This unit incorporates a high-precision GPS receiver, enhanced by the BeiDou satellite system, to acquire real-time vehicle location coordinates, altitude, and speed. Simultaneously, it connects to online map services, downloading road topology data around the vehicle, including lane numbers, road curvature, and slope changes. The environmental detection unit also integrates meteorological data reception, acquiring real-time weather conditions such as rainfall intensity, visibility, and wind speed via cellular networks. This environmental data, after initial processing, forms environmental signals, providing critical input for subsequent data fusion.

[0034] The multi-sensor fusion unit receives sensor signals from the distributed sensor unit and environmental signals from the external environment detection unit, initiating a heterogeneous data fusion process. This unit employs a hierarchical fusion architecture, first aligning the timestamps and unifying the coordinate system of various data types to eliminate data inconsistencies caused by different sampling frequencies and sensor locations. Subsequently, a Kalman filter algorithm is applied to smooth vehicle motion-related data, reducing the impact of measurement noise. For user behavior data, the fusion unit uses a feature-level fusion method to extract and integrate key feature vectors from different sensors. Environmental data is analyzed by overlaying spatial registration with road topology information. After these processing steps, the multi-sensor fusion unit ultimately outputs structured multi-source cockpit data.

[0035] Multi-source cockpit data is divided into three sub-datasets based on its purpose. Predictive multi-source data is specifically used by the image prediction unit and includes user behavior data such as head rotation angle, gesture amplitude, and gaze coordinates; vehicle motion data such as acceleration, angular velocity, and yaw angle; external environment change data such as changes in road curvature and weather trends; and historical image parameters and quality data such as resolution settings and frame rate performance over a past period. Optimization multi-source data is directed to the image optimization unit and mainly includes user preference data such as commonly used rendering modes and image quality settings, as well as rendering material performance data such as the computing power and memory usage of currently available rendering resources. Regulation multi-source data is specifically supplied to the rendering control unit and includes real-time user attention signals such as eye gaze duration and pupil diameter changes, and device status signals such as GPU temperature, memory usage, and power consumption.

[0036] After receiving and controlling multi-source data, the rendering control unit begins defining rendering constraints. This unit first analyzes real-time user attention signals, calculating the duration and movement patterns of the eye's gaze on the screen to determine the user's current level of concentration. When a significant decrease in user attention is detected, such as a scattered gaze or an increased frequency of eye closure, the system defines constraints to reduce rendering accuracy. Simultaneously, device status signals are monitored in real-time; for example, when the GPU temperature approaches a safe threshold, the system defines constraints to limit the rendering load. These constraints are quantified into specific parameter limits, such as a maximum resolution limit and a minimum frame rate limit.

[0037] The rendering constraints are defined using an integrated multi-source signal analysis algorithm deployed within the constraint decision submodule of the rendering control unit. The algorithm constructs a dynamic constraint model based on real-time user attention signals and device status signals, where the attention signals include time-series data of eye gaze dwell time. and gaze point movement trajectory divergence Device status signals include GPU temperature sampling values. Instantaneous value of video memory usage The model employs a two-level decision network: the first level is a feature extraction layer, and the sliding window method is used to calculate the temporal mean of the attention signal. and standard deviation and the differential rate of change of the device status signal and The second layer is the fusion output layer, which generates the constraint exponent through a linear combination function. The function has the following form:

[0038] ,

[0039] in , , , The weights are obtained through training on historical rendering session data. The loss function is optimized using gradient descent during the training process.

[0040] ,

[0041] in, These are manually labeled optimal constraint indices. Constraint Indices Mapped to specific parameter constraints: when Exceeding the threshold At that time, the constraint strictly limits the rendering resolution to no more than 1080p; when Below the threshold At that time, the frame rate can be increased to 120fps. Threshold and By setting system calibration values ​​to fixed values, the constraints are ensured to adaptively adapt to the real-time cockpit status. The algorithm execution cycle is 100 milliseconds, updating the constraints in real time to maintain a balance between rendering quality and device safety.

[0042] Based on defined rendering constraints, the rendering control unit outputs rendering parameters. These parameters include visual settings such as target resolution, frame rate, texture quality level, and lighting intensity. To ensure system stability, the rendering control unit uses model predictive control to dynamically adjust these parameters. This method establishes a state-space model of the rendering system, predicts system behavior over a future period, and optimizes parameter adjustment strategies based on the prediction results. Model predictive control continuously monitors the actual values ​​of the rendering parameters, ensuring they never fall below preset minimum values, such as a resolution no lower than 720p and a frame rate no lower than 30fps, thereby preventing image quality from degrading to an unacceptable level. This dynamic adjustment process is continuously performed in a rolling optimization manner, constantly updating the predictive model and parameter settings based on real-time data to achieve stable maintenance of rendering quality.

[0043] The entire data acquisition and processing flow forms a closed-loop system. Distributed sensor units and external environment detection units continuously provide raw data, multi-sensor fusion units transform this data into structured information, and the rendering control unit intelligently adjusts rendering parameters based on real-time conditions. This implementation allows the cockpit 4D imaging system to adapt to complex and ever-changing operating environments, automatically optimizing rendering effects based on user status, equipment conditions, and external factors, while maintaining system stability and reliability. All data processing and parameter adjustments are completed within an edge-cloud collaborative architecture. The cloud provides powerful computing capabilities to support the execution of complex algorithms, while the edge ensures real-time response and local data processing, jointly achieving efficient and reliable cockpit 4D image generation.

[0044] Example 2: See Figure 3 As a core component of the cloud-side processing module, the image prediction unit operates based on the collaborative work of a time-series prediction model built from a deep neural network and an improved evidence fusion model. This unit receives multi-source prediction data from the edge data acquisition module. This data contains information in five dimensions: user behavior data covering biometrics such as head rotation angle, gesture amplitude, and eye gaze coordinates; vehicle motion data including dynamic parameters such as three-axis acceleration, yaw rate, and steering wheel angle; external environment change data involving the rate of change of road curvature, visibility attenuation trend, and fluctuations in surrounding vehicle density; historical image parameters recording past time periods' rendering resolution, frame rate settings, and light and shadow intensity configurations; and historical image quality data storing historical records of quality indicators such as image sharpness, latency, and color distortion.

[0045] The time series prediction model employs a multi-level processing architecture. The time series feature extraction submodule first preprocesses the vehicle motion data, focusing on analyzing the raw signals from the triaxial accelerometer. This submodule uses wavelet transform to decompose the acceleration signal and extracts frequency band energy features related to precursors of vehicle motion patterns by setting specific frequency band filters. For example, when a vehicle is about to enter a curve, the energy of lateral acceleration in the 0.5-2Hz frequency band shows a characteristic enhancement; this energy change is quantified as a motion precursor signal. Simultaneously, the multimodal fusion submodule initiates the processing flow. This submodule uses a bidirectional long short-term memory network as its basic architecture and introduces an attention mechanism for data weighting. The network input layer receives three parallel data streams: user behavior data, vehicle motion data, and external environment change data. The attention mechanism calculates the importance weights of each data source; for example, in lane-changing scenarios, steering wheel angle data receives higher weights; under adverse weather conditions, visibility data automatically receives higher weights. The weighted data is then processed by a recurrent neural network for time-series dependency modeling, outputting a fused feature vector.

[0046] The risk quantification submodule receives the fused feature vector and executes a risk assessment algorithm. This algorithm defines multi-dimensional risk indicators, including an image quality risk index, a system latency risk index, and a user comfort risk index. Each index is calculated through a combination of linear weighting and non-linear activation functions. For example, the image quality risk index comprehensively considers factors such as motion blur probability, color distortion trend, and resolution decay rate. The calculation results are normalized to a risk value scale of 0-100. The prediction result visualization module then converts the risk values ​​into a dynamic risk map using a color coding system: the 0-30 range is displayed as a green hexagon, the 31-70 range as a yellow triangle, and the 71-100 range as a red circle. These color blocks dynamically flow across the display interface according to the prediction timeline, forming a visualized risk timeline.

[0047] The evidence fusion model runs independently as a bypass of the time series prediction model, specifically handling the fusion analysis of user behavior data, vehicle motion data, and external environmental change data. Based on an improved Dempster-Shafer theoretical framework, the model first establishes a basic probability assignment function. For user behavior data, it defines the reliability assignments for propositions such as distraction and fatigued driving; for vehicle motion data, it defines the reliability assignments for propositions such as abnormal vibration and oversteering; and for external environmental data, it defines the reliability assignments for propositions such as insufficient visibility and slippery roads. The model resolves inconsistencies among multi-source evidence through an adaptive adjustment mechanism for conflict factors. After fusion calculation, it outputs a risk warning signal, which includes three risk levels: Level 1 risk corresponds to a routine warning, Level 2 risk indicates system performance degradation, and Level 3 risk represents an emergency.

[0048] In a typical operating scenario, when a vehicle is traveling in heavy rain, the external environment detection unit reports visibility dropping below 50 meters. The time-series prediction model detects rain and fog distortion patterns previously observed in historical image quality data, and combined with the current frame rate decline trend, outputs an image quality risk value of 65. Simultaneously, the evidence fusion model calculates heavy rain evidence from the environmental data and increased braking frequency evidence from the vehicle motion data, resulting in a level 2 risk. The prediction result visualization module generates a flowing sequence of yellow triangles on the display interface, with a predicted duration of 8 minutes. The risk warning engine simultaneously receives this signal and activates a level 2 risk warning icon on the user interface.

[0049] The entire image prediction unit's processing flow forms a closed-loop feedback loop. The prediction results from the time-series prediction model update the historical image parameter database each period, while the risk level output by the evidence fusion model is used to adjust the parameter settings of the basic probability allocation function. This dynamic adjustment mechanism enables the system to continuously optimize prediction accuracy and adapt to the changing needs of different driving scenarios. All processing is completed on a cloud server cluster, achieving millisecond-level response through a distributed computing framework. The processing results are transmitted in real-time to the edge image generation module via an encrypted channel.

[0050] Example 3: See Figure 4 The image optimization unit, as a core component of the cloud-side processing module, relies on the collaborative work of multiple sub-modules to achieve its functionality. This unit receives optimized multi-source data from the edge data acquisition module and image prediction signals from the image prediction unit. The optimized multi-source data includes user preference data and rendering material performance data. User preference data is obtained by analyzing historical interaction records, including commonly used personalized parameters such as brightness settings, color saturation preferences, and dynamic effect intensity tendencies. Rendering material performance data covers hardware performance indicators such as the computing power, memory bandwidth, and video memory capacity of currently available rendering resources. The image prediction signal includes predicted image quality change trends, potential risk levels, and expected system load over a future time period.

[0051] The rendering-quality nonlinear relationship model is the core algorithmic foundation of the image optimization unit. This model establishes a mapping relationship between rendering parameters and image quality through a multinomial regression method. The main rendering parameters considered in the model include target resolution, frame rate setting, texture quality level, lighting intensity, and anti-aliasing level. Image quality is quantified through multi-dimensional indicators, including peak signal-to-noise ratio, structural similarity index, and color fidelity. During model construction, the least squares method is used to fit the parameter-quality correspondence in historical data, generating a nonlinear function of the following form:

[0052] ,

[0053] in: This represents the overall score for the predicted image quality. This represents the rendering stress parameter (taking into account factors such as resolution and frame rate). For polynomial coefficients, For the logarithmic term weighting coefficients, and These are the parameters for the exponential decay term. These parameters are obtained through training on historical data and are continuously updated as the system runs.

[0054] The dynamic rendering performance matching library serves as the supporting system, storing a large amount of historical rendering performance data for historical rendering materials. This database records the performance of different rendering materials in various scenarios, including metrics such as computation time, memory usage, and power consumption. The database organizes data using a time-series structure, supporting fast timestamp-based retrieval and performance trend analysis. Each data entry contains a triplet of material identifier, timestamp, and performance metric, and fast querying is achieved through a Bloom filter.

[0055] The scene complexity analysis submodule calculates the scene complexity coefficient based on rendering parameters and rendering signals. This module first parses the geometric information in the rendering signal, including basic parameters such as the number of polygons, vertex density, and texture size. Then, it uses a weighted calculation model based on the Analytic Hierarchy Process (AHP) to normalize various geometric parameters and sum them in a weighted manner to obtain an initial complexity estimate. To further improve accuracy, the module also considers the influence factors of dynamic elements, including the number of moving objects, the frequency of lighting changes, and the complexity of transparency effects. The final output complexity coefficient is a normalized value between 0 and 1; a higher value indicates a more complex scene.

[0056] The rendering path planning submodule receives scene complexity coefficients, the rendering-quality nonlinear relationship model output, and rendering material performance data. It then generates the optimal rendering signal through an optimization algorithm. This module employs an improved genetic algorithm as its optimization engine, decomposing the rendering task into multiple subtasks, each corresponding to a set of rendering parameter configurations. During the algorithm initialization phase, multiple candidate solutions are randomly generated, each representing a possible combination of rendering parameters. The fitness function calculates the expected quality score for each solution based on the rendering-quality nonlinear relationship model, while also considering resource constraints in the rendering material performance data.

[0057] During optimization, the algorithm performs selection, crossover, and mutation operations. The selection operation uses a roulette wheel strategy based on fitness scores, with high-quality solutions receiving a higher selection probability. The crossover operation swaps some parameter configurations among solution vectors, while the mutation operation randomly adjusts individual parameter values. The algorithm's convergence condition is set to the optimal fitness improvement being less than a threshold in consecutive iterations, ultimately outputting the best rendering signal.

[0058] The optimal rendering signal contains a detailed set of rendering instructions, including the spatial coordinate definition of the rendering region, the priority order of rendering types, the adjustable range of rendering pressure, and a detailed rendering path plan. The rendering region coordinates use three-dimensional spatial encoding to define the area that needs to be prioritized for rendering; the rendering type priority specifies the execution order of different rendering effects; the rendering pressure range gives the adjustable range of various parameters; and the rendering path plan details the resource allocation scheme and time sequence arrangement.

[0059] The entire image optimization unit operates as a closed-loop optimization system. After each rendering task, the actual rendering performance data is fed back to the dynamic matching library for rendering performance, updating historical records. Simultaneously, the comparison between the actual achieved image quality and the predicted quality is used to adjust the parameters of the rendering-quality nonlinear relationship model, continuously improving the model's prediction accuracy. This self-improving mechanism enables the system to adapt to constantly changing operating environments and user needs.

[0060] In typical operating scenarios, when the system detects a user preference for a high frame rate mode, the rendering-quality nonlinear relationship model adjusts its parameter weights accordingly, giving the frame rate parameter a greater influence. The scene complexity analysis submodule simultaneously calculates the complexity of the current driving scene; if a complex urban environment is identified, it automatically increases the priority of detail rendering. The rendering path planning submodule integrates this information and uses optimization algorithms to calculate the optimal parameter combination that maximizes visual quality while maintaining the frame rate. The resulting optimal rendering signal guides the edge rendering device to perform precise rendering operations.

[0061] All computations are performed on a high-performance computing cluster in the cloud, leveraging a parallel computing architecture to accelerate the execution of optimization algorithms. The final optimal rendering signal is transmitted to the edge-side adaptive rendering execution unit via a secure communication link, realizing a cloud-based collaborative rendering optimization mechanism. During system operation, changes in various parameters are continuously monitored, and model parameters and algorithm configurations are updated periodically to maintain the continuity and adaptability of optimization effects.

[0062] Example 4: See Figure 5 The user interaction module, as a crucial component of the edge-side image generation system, relies on the collaborative operation of the 3D visualization platform and the risk warning engine for its functionality. This module receives risk warning signals and image prediction signals from the cloud-side processing module, while simultaneously acquiring multi-source cockpit data provided by the edge-side data acquisition module and real-time status signals uploaded by the cockpit equipment control unit.

[0063] The 3D visualization platform uses physically based rendering technology to construct a virtual cockpit environment. The platform engine first establishes an accurate 3D model of the cockpit, including geometric modeling of the internal structures such as seats, dashboard, and center console. The model is represented using a polygonal mesh, with each vertex containing position coordinates, normal vectors, and texture coordinates. The platform receives multi-source cockpit data in real time, including biometric information such as user eye-tracking coordinates, head posture angles, and hand movement trajectories. The platform maps this data onto the virtual cockpit model, driving the motion trajectory and viewpoint changes of the virtual camera.

[0064] Real-time status signals include physical parameters such as device temperature, humidity, voltage, and rendering pressure. The visualization platform transforms these parameters into visual elements, such as using color gradients to represent temperature distribution and fluid animation to simulate pressure changes. The platform supports free viewing angle adjustment at any angle, allowing users to control the movement, rotation, and zoom of the virtual camera via gestures or eye movements. In viewpoint analysis mode, the platform calculates and displays the spatial relationship between the image content and the user's line of sight in real time, including parameters such as field of view coverage and focal area overlap.

[0065] The physics engine is integrated into the visualization platform to simulate image changes under different rendering schemes. The engine calculates the trajectory of virtual objects based on rigid body dynamics principles and uses ray tracing algorithms to simulate light and shadow interactions. When a new rendering scheme is received, the engine calculates the lighting distribution, shadow effects, and material properties under that scheme in real time and presents a comparison in the visualization interface. Users can observe the image changes at different times by dragging the timeline.

[0066] The risk warning engine employs a multi-attribute decision-making algorithm to process input signals. The algorithm establishes a decision matrix, where rows represent different warning scenarios and columns represent evaluation indicators, including image quality scores, system latency, and resource utilization. The weight coefficient for each indicator is determined using the entropy weighting method, reflecting the differences in importance among the indicators. The algorithm calculates a comprehensive score for each scenario and outputs a user feedback index, which ranges from 0 to 100.

[0067] The warning engine employs a multi-level color-coded status mechanism, corresponding to different risk levels. A yellow warning is activated when image quality drops to the first quality threshold, and the system automatically reduces the rendering equipment's operating speed to a preset warning speed. An orange warning is triggered when image quality drops to the second quality threshold, and the system suspends regular rendering tasks and initiates an emergency rendering process. A red warning is activated when image quality drops to the third quality threshold, and the system issues an emergency fault signal and executes a shutdown procedure.

[0068] Table 1: Risk Warning Levels and Corresponding Measures

[0069]

[0070] When the user feedback index exceeds a preset threshold, the system triggers an emergency response mechanism. The user interaction module sends control commands to the adaptive rendering execution unit to initiate the emergency rendering process. Emergency rendering uses a simplified rendering pipeline, reducing shader complexity and post-processing effects, prioritizing the operation of basic visualization functions. In a typical scenario, when a vehicle is traveling in a tunnel, the ambient light suddenly dims. The image prediction unit detects the brightness change trend and predicts a potential decrease in image quality. The risk warning engine calculates that the user feedback index has risen to 75, exceeding the threshold of 65. The system immediately activates a yellow warning state, reducing the rendering device speed to 70% of the standard value. The 3D visualization platform simultaneously adjusts the interface brightness, enhances contrast, and displays a flashing yellow warning indicator at the interface edges.

[0071] If the tunnel environment persists and the image quality further degrades to 62, the system upgrades to an orange alert status. The platform automatically switches to emergency rendering mode, disabling unnecessary visual effects while retaining core information display. Simultaneously, a backup rendering pipeline is activated, using simplified shaders to maintain basic image generation. All operations are completed without the user's awareness, ensuring a continuous driving experience. When the vehicle exits the tunnel and ambient lighting returns to normal, the system automatically detects that the image quality has rebounded to 88. The alert status is lifted, the equipment operating speed gradually returns to standard values, and the rendering mode switches back to the standard pipeline. The entire state transition is smooth, avoiding sudden performance changes that could impact the user experience.

[0072] The user interaction module also provides manual intervention functionality. Users can actively adjust the alert sensitivity and set personalized quality threshold parameters through gestures or voice commands. All user operation records are saved in a preference configuration file for optimizing subsequent risk assessment algorithms. The system's self-learning mechanism continuously analyzes historical operational data and adjusts the weight parameters in the multi-attribute decision-making algorithm. By analyzing the correspondence between user feedback indices and actual experiences in different scenarios, the system continuously optimizes alert accuracy and emergency response strategies. This continuous improvement mechanism ensures that the system can adapt to various complex operating environments and user needs.

[0073] Example 5: The optimal rendering signal, as the output of the image optimization unit, includes a multi-dimensional set of control instructions, specifically covering the spatial coordinate definition of the rendering area, the execution priority ranking of rendering types, the adjustable range setting of rendering pressure, and a detailed planning scheme for the rendering path. The rendering area coordinates are encoded using a three-dimensional Cartesian coordinate system, establishing a reference system with the center of the vehicle cabin as the origin. The X-axis points in the direction of vehicle movement, the Y-axis points to the side of the driver's seat, and the Z-axis is vertically upward. The coordinate data is accurate to the millimeter level, defining the cubic area that needs to be prioritized for rendering. For example, the windshield projection area might be defined as a cubic space of [-500, 500] × [-300, 300] × [100, 500]. The rendering type priority uses a weighted allocation mechanism, assigning execution weight coefficients to different rendering effects, such as a weight of 0.35 for lighting effects, 0.28 for texture details, and 0.22 for dynamic blur. The system determines the resource allocation order based on the weight. The rendering pressure range specifies the operable ranges for various parameters, including flexible settings such as resolution range [720p, 4K] and frame rate range [30fps, 120fps]. The rendering path planning details the spatiotemporal allocation strategy for computing resources, including specific implementation schemes such as GPU core scheduling sequence, memory allocation ratio, and shader execution order.

[0074] The core components of the adaptive rendering execution unit include an intelligent rendering pump and an integrated online rendering quality monitoring device. The intelligent rendering pump, acting as a physical actuator, has a built-in microcontroller that analyzes the instruction set in the optimal rendering signal. This device is equipped with a high-precision stepper motor drive system, controlling the flow rate of the rendering medium by adjusting the valve opening. When receiving rendering pressure range parameters, the pump automatically calculates the required pressure value and precisely outputs the corresponding pressure through the hydraulic system. The flow control system employs a closed-loop feedback mechanism, comparing the actual flow rate with the set value in real time and dynamically adjusting the motor speed using a PID algorithm. In typical operation, when the system requires increased rendering detail in a certain area, the intelligent rendering pump increases the medium flow rate in that area while simultaneously adjusting the distribution ratio of the delivery pipeline according to path planning.

[0075] The integrated online rendering quality inspection device is located at the rendering output end and consists of an optical sensor array and a real-time analysis module. The optical sensor array contains multiple sets of high-resolution CMOS sensors arranged at a specific angle around the rendering surface. The device operates based on the principle of laser scattering, emitting a laser beam of a specific wavelength to illuminate the rendering surface and evaluating rendering uniformity by analyzing the scattering pattern of the reflected light. The inspection device incorporates a feature extraction algorithm to identify quality parameters such as texture consistency and color distribution uniformity of the rendered surface. When a localized quality deviation is detected, the device generates a compensation signal and feeds it back to the intelligent rendering pump, triggering a flow fine-tuning mechanism. This process forms a closed-loop quality control circuit; for example, when insufficient color saturation is detected in a certain area, the system automatically increases the supply of coloring media to that area.

[0076] A real-time status signal monitoring system is deployed at key locations throughout the cockpit, continuously collecting six types of physical parameters. Temperature monitoring employs a distributed thermocouple network, with sensors placed in hotspot areas such as the rendering equipment's heatsinks and hydraulic lines, sampling at a frequency of 10Hz. Humidity monitoring uses capacitive sensors, primarily deployed in external interface areas susceptible to environmental influences. Voltage monitoring circuitry is connected to all electronic control units, tracking power supply fluctuations in real time. Equipment depth detection is achieved through pressure sensors, measuring the liquid level in the rendering media tanks. Rendering pressure monitoring uses piezoelectric sensors installed at key nodes in the delivery pipelines, with a measurement range of 0-10MPa. Extended radius monitoring utilizes a laser rangefinder to track displacement changes in mechanical actuators.

[0077] During system operation, real-time status signals are continuously transmitted to the cabin equipment control unit. This unit establishes a signal threshold early warning mechanism, setting safe operating ranges for each type of parameter. When the temperature signal exceeds a preset threshold, the control unit automatically activates the cooling fan speed-up program; when voltage fluctuations exceed the tolerance range, the voltage regulation circuit adjustment mechanism is triggered; when the equipment depth drops to the warning line, a media replenishment alarm is activated. All status data is synchronously uploaded to the cloud for updating the equipment health status model.

[0078] A typical application scenario demonstrates the system's collaborative workflow: When a vehicle enters a tunnel, changes in ambient light trigger the image optimization unit to adjust the optimal rendering signal, requiring an increase in the HDR effect weight to 0.4. Upon receiving the signal, the intelligent rendering pump increases the delivery pressure of the light and shadow medium to 2.5 MPa, while simultaneously adjusting the flow distribution ratio of the three areas to 3:5:2. An integrated detection device monitors the rendered surface in real time, detecting insufficient highlight detail in the left area and providing a compensation signal. Simultaneously, a temperature sensor detects that the GPU temperature has risen to 75°C, and the cockpit equipment control unit automatically increases the cooling fan speed by 20%. The entire process is completed within 200 milliseconds, achieving dynamic adjustment without manual intervention.

[0079] The system maintenance mechanism includes self-diagnostic functions and regularly executes equipment calibration procedures. The intelligent rendering pump automatically performs zero-point calibration monthly to eliminate mechanical errors. The optical inspection unit performs benchmark tests weekly, using standard color charts to verify measurement accuracy. All calibration data is recorded in the equipment log for predictive maintenance analysis. When sensor drift is detected to exceed the allowable range, the system automatically marks the faulty unit and initiates a backup sensor switching process.

[0080] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0081] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A cockpit 4D image generation system based on edge-cloud collaboration, characterized in that, include: The end-side data acquisition module includes a multi-sensor fusion unit, which generates multi-source cockpit data. The multi-sensor fusion unit receives sensor signals and environmental signals to achieve heterogeneous data fusion and generate the multi-source cockpit data, which includes predicted multi-source data, optimized multi-source data, and regulated multi-source data. The cloud-side processing module receives the multi-source cockpit data and includes an image prediction unit, an image optimization unit, and a rendering control unit. The image prediction unit processes the multi-source cockpit data and outputs an image prediction signal and a risk warning signal. The image optimization unit processes the multi-source cockpit data and the image prediction signal to obtain an optimal rendering signal. The rendering control unit defines rendering constraints based on the multi-source cockpit data and outputs rendering parameters. The image prediction unit includes a time-series prediction model based on a deep neural network and an evidence fusion model. The predicted multi-source data is input into the time-series prediction model and the image prediction signal is output. The improved evidence fusion model fuses user behavior data, vehicle motion data, and external environment change data and outputs the risk warning signal. The edge-side image generation module includes an adaptive rendering execution unit, a cockpit equipment control unit, and a user interaction module. The adaptive rendering execution unit adjusts rendering parameters according to the optimal rendering signal. The cockpit equipment control unit adjusts cockpit equipment parameters according to the rendering parameters and the multi-source cockpit data, and collects real-time status signals. The user interaction module processes the risk warning signal and the image prediction signal and outputs a user feedback index.

2. The cockpit 4D image generation system based on edge-cloud collaboration according to claim 1, characterized in that, The end-side data acquisition module includes a distributed sensor unit and an external environment detection unit. The end-side data acquisition module is deployed inside and outside the cockpit and collects sensor signals. The external environment detection unit acquires environmental data around the vehicle through satellite positioning technology and forms environmental signals.

3. The cockpit 4D image generation system based on edge-cloud collaboration according to claim 1, characterized in that, The time series prediction model includes a time series feature extraction submodule, a multimodal fusion submodule, a risk quantification submodule, and a prediction result visualization module. The time series feature extraction submodule performs wavelet transform on the vehicle motion data to extract energy change features of frequency bands that meet the precursor frequency band range of the motion pattern as motion precursor signals. The multimodal fusion submodule uses an attention-weighted recurrent neural network to process the user behavior data, the vehicle motion data, and the external environment change data and outputs the data to the risk quantification submodule. The risk quantification submodule defines a risk index algorithm. The prediction result visualization module generates a dynamic risk map of the risk index based on color coding, using different colored blocks to represent different risk levels.

4. The cockpit 4D image generation system based on edge-cloud collaboration according to claim 2, characterized in that, The image optimization unit processes the optimized multi-source data and the image prediction signal using a rendering-quality nonlinear relationship model, outputs a rendering signal, and then uses an optimization algorithm to process the rendering signal to obtain the optimal rendering signal. The optimized multi-source data includes user preference data and rendering material performance data. The rendering-quality nonlinear relationship model is constructed using the user preference data, the image prediction signal, and the rendering material performance data.

5. The cockpit 4D image generation system based on edge-cloud collaboration according to claim 4, characterized in that, The image optimization unit further includes a rendering performance dynamic matching library, a scene complexity analysis submodule, and a rendering path planning submodule. The rendering performance dynamic matching library stores historical rendering performance data of historical rendering materials. The scene complexity analysis submodule calculates the scene complexity coefficient based on the rendering parameters and the rendering signal. The rendering path planning submodule obtains the optimal rendering signal through an optimization algorithm based on the complexity coefficient, the rendering-quality nonlinear relationship model, and the rendering material performance data.

6. The cockpit 4D image generation system based on edge-cloud collaboration according to claim 2, characterized in that, The rendering control unit defines rendering constraints based on the multi-source data, outputs rendering parameters, and dynamically adjusts the rendering parameters to ensure they do not fall below the minimum value through model prediction control. The multi-source data includes real-time user attention signals and device status signals, which define the rendering constraints.

7. The cockpit 4D image generation system based on edge-cloud collaboration according to claim 1, characterized in that, The user interaction module includes a 3D visualization platform and a risk warning engine. The 3D visualization platform receives the multi-source cockpit data and the real-time status signal and dynamically renders the image generation process. The risk warning engine processes the risk warning signal and the image prediction signal according to a multi-attribute decision algorithm and outputs a user feedback index. When the user feedback index exceeds the feedback index threshold, the user interaction module controls the adaptive rendering execution unit to perform emergency rendering. The 3D visualization platform supports arbitrary angle and perspective analysis, can display the spatial relationship between the image and the user's line of sight in real time, and simulates image changes under different rendering schemes based on a physics engine.

8. The cockpit 4D image generation system based on edge-cloud collaboration according to claim 7, characterized in that, The risk warning engine includes a multi-level warning mechanism with color-coded states: yellow, orange, and red. When the image quality in the image prediction signal drops to a first quality threshold, the yellow warning state is activated, reducing the rendering device's operating speed to a warning speed. When the image quality in the image prediction signal drops to a second quality threshold, the orange warning state is activated, suspending the rendering device's operation and controlling the adaptive rendering execution unit to perform emergency rendering. When the image quality in the image prediction signal drops to a third quality threshold, the red warning state is activated, and the risk warning engine issues an emergency fault signal and controls the end-to-cloud collaborative cockpit 4D image generation system to shut down urgently.

9. The cockpit 4D image generation system based on edge-cloud collaboration according to claim 1, characterized in that, The optimal rendering signal includes, but is not limited to, rendering region coordinates, rendering type priority, rendering pressure range, and rendering path planning. The adaptive rendering execution unit includes an intelligent rendering pump and an integrated online rendering quality detection device. The intelligent rendering pump can identify the optimal rendering signal and adjust the rendering flow and rendering pressure as needed. The integrated online rendering quality detection device receives the optimal rendering signal and adjusts the quality parameters of the rendered object in real time based on optical detection methods. The real-time status signal includes, but is not limited to, device temperature, device humidity, device voltage, device depth, rendering pressure, and expansion radius.

Images