Dynamic generation test method, device and equipment of vehicle-mounted theme and medium

Abstract

The invention provides a dynamic generation test method, device and equipment for a vehicle-mounted theme and a medium, and the method comprises the steps: calling a multi-modal generation model to generate multiple groups of virtual scene parameters, inputting each virtual scene parameter into a theme generation model, generating a vehicle-mounted theme parameter corresponding to each virtual scene parameter, and carrying out the dynamic generation test on the vehicle-mounted theme according to the vehicle-mounted theme parameter. Performing display test on each vehicle-mounted theme parameter to obtain a plurality of target test results, if the target test result of one vehicle-mounted theme parameter is passed, inputting the vehicle-mounted theme parameter into a human factor scoring model for simulation evaluation to obtain a human factor simulation score, and if the human factor simulation score of one vehicle-mounted theme parameter is smaller than a preset simulation score threshold, outputting the vehicle-mounted theme parameter. If yes, determining the vehicle-mounted theme parameter as a target theme parameter, and carrying out simulated driving on the target theme parameter to obtain a human factor optimization model and a human factor optimization suggestion; according to the method, the adaptation capability of dynamic generation of the vehicle-mounted theme is verified through three models, and a human factor evaluation system is provided.

Description

Technical Field

[0001] The present application relates to the technical field of dynamic theme testing, and in particular to a method, device, equipment and medium for dynamically generating and testing vehicle-mounted themes. Background Art

[0002] With the development of smart cockpit technology, dynamic in-vehicle theme generation systems have gradually become a core function for enhancing the human-vehicle interaction experience. Current industry-wide testing and verification systems for in-vehicle themes face the following common technical bottlenecks: Most systems rely on pre-configured rule engines or limited scenario libraries, making them ineffective in handling dynamically generated scenarios driven by large AI models. This is particularly true for complex scenarios such as intelligent adaptation of night mode and coordinated needs among multiple occupants, resulting in insufficient scenario coverage and distorted boundary condition simulations. Human factors experience evaluation methods are outdated, lacking real-time quantitative analysis of user physiological characteristics. Furthermore, multimodal verification systems are fragmented: They often process single-modal data independently, making it difficult to verify the reliability of the coordinated operation of various subsystems under complex operating conditions. This is particularly true in scenarios where V2X traffic events trigger interface changes, leading to a lack of cross-modal causal reasoning.

[0003] The above-mentioned common technical bottlenecks lead to two major technical faults in the dynamic generation test of in-vehicle themes: on the one hand, it is impossible to effectively verify the dynamic adaptation ability of the artificial intelligence-generated interface; on the other hand, there is a lack of a scientific and quantitative human experience evaluation system. This has become a key bottleneck restricting the development of smart cockpit technology. Summary of the Invention

[0004] The present invention provides a method, device, equipment and medium for testing the dynamic generation of vehicle-mounted themes to solve the above-mentioned technical problems of being unable to verify the adaptability of the dynamic generation of vehicle-mounted themes and the lack of a human-powered evaluation system.

[0005] In one embodiment of the present application, the present application provides a dynamic generation test method for an in-vehicle theme, comprising: calling a multimodal generation model to generate multiple sets of virtual scene parameters; inputting each of the virtual scene parameters into the theme generation model respectively to generate in-vehicle theme parameters corresponding to each of the virtual scene parameters, and performing a display test on each of the in-vehicle theme parameters to obtain multiple target test results; if the target test result of an in-vehicle theme parameter is passed, inputting the in-vehicle theme parameter into a human factors scoring model for simulation evaluation to obtain a human factors simulation score; if the human factors simulation score of an in-vehicle theme parameter is less than a preset simulation score threshold, determining the in-vehicle theme parameter as a target theme parameter to simulate driving for the target theme parameter, and optimizing the target theme parameter based on the driver perception data and actual functional operation data in the simulated driving to obtain a human factors optimization model and human factors optimization suggestions; wherein, the multimodal generation model is obtained based on the fusion training of real scene parameters and multimodal historical monitoring data, and the human factors scoring model is obtained based on historical functional operation data, historical scene parameters and historical user task scores.

[0006] In one embodiment of the present application, the training of the multimodal generation model includes: obtaining real scene parameters and multimodal historical monitoring data, wherein the historical monitoring data includes historical visual data, historical environmental data, historical traffic data, and historical user operation preference data; pre-training the corresponding single-modal encoder for feature recognition according to the historical monitoring data of each modality to obtain multiple initial feature vectors of each modality, wherein each of the initial feature vectors includes multiple single-modal features; performing multimodal fusion on each of the initial feature vectors to obtain multiple fused feature vectors, wherein the multimodal fusion includes Each of the initial feature vectors is subjected to spatiotemporal alignment and feature alignment training to obtain a plurality of multimodal features, and inter-modal causal reasoning is performed on the multimodal features to adjust the feature weights of the corresponding unimodal features in the multimodal features; the feature alignment training is used to align the unimodal features of different modalities in the same event in the latent space according to the real multimodal features to obtain generated multimodal features, and the real multimodal features are obtained based on the initial feature vectors after spatiotemporal alignment; scene parameter generation training is performed according to the real scene parameters and the real fusion feature vector to obtain a multimodal generation model.

[0007] In one embodiment of the present application, scene parameter generation training is performed based on the real scene parameters and the real fusion feature vector to obtain a multimodal generation model, including: teacher-forced training is performed on the first generation model based on the real scene parameters and the real fusion feature vector to obtain a second generation model, and the teacher-forced training adjusts the model parameters of the first generation model through the mean square error between the first predicted scene parameters output by the first generation model and the corresponding real scene parameters, and the physical compliance of the first predicted scene parameters; free-running training is performed on the second generation model based on the real scene parameters, the real fusion feature vector and the preset noise vector to obtain a multimodal generation model, and the free-running training includes adjusting the model parameters of the second generation model through the optimized adversarial loss between the second predicted scene parameters output by the second generation model and the corresponding real scene parameters, and the physical compliance of the second predicted scene parameters; wherein the model parameters of the first generation model and the second generation model include constraint weights that dynamically adjust preset scene generation constraints according to the driver state.

[0008] In one embodiment of the present application, the target theme parameters are optimized and trained based on driver perception data and actual functional operation data during simulated driving to obtain a human factors optimization model and human factors optimization recommendations, including: obtaining driver perception data and actual functional operation data during simulated driving, the driver perception data including eye movement perception data and physiological perception data, and the actual functional operation data including actual operation steps and actual path length; determining an operation efficiency index based on a first difference coefficient between a preset theoretical operation step and the actual operation step, and a second difference coefficient between a preset theoretical path length and the actual path length; determining an actual distraction index based on the eye movement perception data and the physiological perception data; determining a target optimization strategy from multiple preset optimization strategies based on a first comparison result of the operation efficiency index with multiple preset operation efficiency intervals, and a second comparison result of the actual distraction index with multiple preset distraction index intervals; updating the target theme parameters based on the target optimization strategy, and performing simulated driving based on the updated target theme parameters to adjust each of the preset operation efficiency intervals, each of the preset distraction index intervals, and each of the preset optimization strategies to obtain the human factors optimization model and human factors optimization recommendations.

[0009] In one embodiment of the present application, an actual distraction index is determined based on the eye movement perception data and the physiological perception data, including: determining the first fixation duration and the eye movement deviation duration based on the eye movement perception data; determining the cognitive load based on a preset first coefficient, the number of skin conductance reactions, a preset second coefficient, a heart rate power ratio, a preset third coefficient, and a skin conductance level, wherein the number of skin conductance reactions and the skin conductance level are obtained based on the galvanic skin perception data, and the heart rate power ratio is obtained based on the heart rate perception data and is used to characterize the power ratio between low frequency and high frequency, and the physiological perception data includes the galvanic skin perception data and the heart rate perception data; determining the increase in heart rate variability based on the heart rate power ratio and a baseline value in the heart rate perception data; and determining the actual distraction index based on the first fixation duration, the eye movement deviation duration, the increase, and the cognitive load.

[0010] In one embodiment of the present application, a display test is performed on the in-vehicle theme parameters to obtain a target test result, including: verifying the functional correctness of the in-vehicle theme parameters according to preset single-modal rules to obtain a first test result; calling an in-vehicle rendering engine to generate a visualization interface of the in-vehicle theme parameters, and comparing the actual performance of the visualization interface with a preset performance benchmark to obtain a second test result, wherein the actual performance includes at least one of theme switching delay, image processing memory occupancy, and system crash rate corresponding to the noise stress test; performing cross-modal rule verification on the in-vehicle theme parameters according to preset multi-modal rules to obtain a third test result; and determining the target test result based on the first test result, the second test result, and the third test result.

[0011] In one embodiment of the present application, after determining the target test result based on the first test result, the second test result and the third test result, it also includes: if the target test result of a vehicle-mounted theme parameter is failed, the vehicle-mounted theme parameter is determined as an abnormal theme parameter, and the virtual scene parameter corresponding to the abnormal theme parameter is determined as an abnormal virtual scene; if the abnormal theme parameter has an interface rendering error, the allocation of rendering resources is detected to obtain first abnormal information; if the abnormal theme parameter does not have an interface rendering error, and the abnormal virtual scene has missed physical compliance, the confidence of the abnormal virtual scene is retroactively generated to obtain second abnormal information; if the abnormal theme parameter does not have an interface rendering error, and the abnormal virtual scene does not have missed physical compliance, the synchronization of multimodal historical monitoring data is verified to obtain third abnormal information; cluster analysis is performed on the abnormal virtual scene based on all target abnormal information to obtain defective scene parameters, and the target abnormal information includes at least one of the first abnormal information, the second abnormal information and the third abnormal information.

[0012] In one embodiment of the present application, the present application provides a dynamic generation test device for an in-vehicle theme, comprising: a scene parameter generation module for calling a multimodal generation model to generate multiple sets of virtual scene parameters; a theme parameter testing module for inputting each of the virtual scene parameters into the theme generation model, generating in-vehicle theme parameters corresponding to each of the virtual scene parameters, and performing a display test on each of the in-vehicle theme parameters to obtain multiple target test results; a human factor scoring module for inputting the in-vehicle theme parameter into the human factor scoring model for simulation evaluation if the target test result of a in-vehicle theme parameter is passed, to obtain a human factor simulation score ; A human factors optimization module is used to determine the on-board theme parameter as the target theme parameter if the human factors simulation score of the on-board theme parameter is less than a preset simulation score threshold, to simulate driving for the target theme parameter, and to optimize the target theme parameter according to the driver perception data and actual functional operation data in the simulated driving to obtain a human factors optimization model and human factors optimization suggestions; wherein, the multimodal generation model is obtained by fusion training based on real scene parameters and multimodal historical monitoring data, and the human factors scoring model is obtained by training based on historical functional operation data, historical scene parameters and historical user task scores.

[0013] In one embodiment of the present application, the present application provides an electronic device, which includes: one or more processors; a storage device for storing one or more programs, and when the one or more programs are executed by the one or more processors, the electronic device implements the dynamic generation test method of the vehicle theme as described in any of the above embodiments.

[0014] In one embodiment of the present application, the present application provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a computer processor, the computer executes the dynamic generation test method of the vehicle theme described in any one of the above embodiments.

[0015] Beneficial effects of embodiments of the present invention: The present application provides a dynamic generation test method, device, equipment and medium for vehicle-mounted themes. The embodiments of the present invention generate virtual scene parameters through a multimodal generation model, thereby dynamically generating vehicle-mounted theme parameters. Through display testing, human factors simulation scoring and human factors optimization suggestions, the adaptability of the dynamic generation of vehicle-mounted themes is verified, and a human factors evaluation system is provided.

[0016] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The accompanying drawings are incorporated into and constitute a part of the specification, showing embodiments consistent with the present application and, together with the specification, used to explain the principles of the present application. Obviously, the drawings described below are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without inventive effort. In the drawings:

[0018] Figure 1 A schematic diagram showing an exemplary system architecture to which the technical solutions of the embodiments of the present application can be applied;

[0019] Figure 2 A schematic diagram showing a flow chart of a method for dynamically generating a vehicle-mounted theme according to an embodiment of the present application is shown;

[0020] Figure 3 A schematic diagram of the architecture of a multimodal generation model according to an embodiment of the present application is shown;

[0021] Figure 4 FIG2 shows an analysis decision diagram of a test failure according to an embodiment of the present application;

[0022] Figure 5 A logical diagram of a dynamic generation test of an in-vehicle theme according to one embodiment of the present application is shown;

[0023] Figure 6 A block diagram of a dynamic generation test device for an in-vehicle theme according to an embodiment of the present application is shown;

[0024] Figure 7 A schematic diagram of the structure of a computer system suitable for implementing an electronic device according to an embodiment of the present application is shown. DETAILED DESCRIPTION

[0025] The following describes the embodiments of the present application through specific examples. Those skilled in the art can easily understand other advantages and effects of the present application from the content disclosed in this specification. The present application can also be implemented or applied through other different specific embodiments. The details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present application. It should be noted that the following embodiments and features in the embodiments can be combined with each other unless they conflict.

[0026] It should be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present application. Therefore, the drawings only show components related to the present application rather than being drawn according to the number, shape and size of components in actual implementation. In actual implementation, the shape, quantity and proportion of each component may be changed at will, and the component layout may also be more complicated.

[0027] In the following description, a large number of details are discussed to provide a more thorough explanation of the embodiments of the present application. However, it is obvious to those skilled in the art that the embodiments of the present application can be implemented without these specific details. In other embodiments, well-known structures and devices are shown in the form of block diagrams rather than in detail to avoid making the embodiments of the present application difficult to understand.

[0028] See also Figure 1 , Figure 1 Schematic diagram showing an exemplary system architecture to which the technical solution of the embodiment of the present application can be applied. Figure 1 As shown, the system architecture can include a scenario parameter generation module, a test execution module, a human factors scoring module, an intelligent analysis module, and a test report output module. The scenario parameter generation module relies on a multimodal generation model to generate multiple sets of virtual scenario parameters. The test execution module is used to perform vehicle theme parameter generation tests on the virtual scenario parameters. The vehicle theme parameter generation test includes display testing of the vehicle theme parameters and simulated evaluation of the vehicle theme parameters through the human factors scoring module. The intelligent analysis module is used to provide human factors optimization recommendations for the vehicle theme parameters and to mine defect scenario parameters. The human factors scoring module relies on a human factors scoring model. The test report output module is used to output information including target test results of the display test of the vehicle theme parameters, human factors simulation scores, human factors optimization recommendations, and defect scenario mining.

[0029] In related technologies, there is an inability to verify the adaptability of the dynamically generated in-vehicle themes, as well as technical issues with the lack of a human-centered evaluation system.

[0030] In order to solve the above technical problems, the present application provides a method, device, equipment and medium for dynamically generating and testing vehicle-mounted themes. The implementation details of the technical solutions of the embodiments of the present application are described in detail below.

[0031] See also Figure 2 , Figure 2 FIG. 1 shows a flow chart of a method for dynamically generating a vehicle-mounted theme according to an embodiment of the present application. Figure 2 As shown, in an exemplary embodiment, the dynamic generation test method of the vehicle theme includes at least steps S210 to S240, which are described in detail as follows:

[0032] Step S210: calling a multimodal generation model to generate multiple sets of virtual scene parameters.

[0033] Among them, the multimodal generation model is obtained by fusion training based on real scene parameters and multimodal historical monitoring data.

[0034] In one embodiment of the present application, the training of a multimodal generation model includes: obtaining real scene parameters and multimodal historical monitoring data, the historical monitoring data including historical visual data, historical environmental data, historical traffic data and historical user operation preference data; pre-training the corresponding unimodal encoder for feature recognition based on the historical monitoring data of each modality to obtain multiple initial feature vectors for each modality, each initial feature vector including multiple unimodal features; performing multimodal fusion on each initial feature vector to obtain multiple fused feature vectors, the multimodal fusion including performing spatiotemporal alignment and feature alignment training on each initial feature vector to obtain multiple multimodal features, and performing inter-modal causal reasoning on the multimodal features to adjust the feature weights of the corresponding unimodal features in the multimodal features; feature alignment training is used to align the unimodal features of different modalities in the same event in the latent space according to the real multimodal features to obtain generated multimodal features, the real multimodal features being obtained based on the initial feature vectors after spatiotemporal alignment; performing scene parameter generation training based on the real scene parameters and the real fused feature vector to obtain a multimodal generation model.

[0035] In one embodiment of the present application, see Figure 3 , Figure 3 FIG. 1 shows a schematic diagram of the architecture of a multimodal generation model according to an embodiment of the present application. Figure 3 As shown in Figure 1, the Multimodal Generation Model (MGM) uses a layered Transformer architecture consisting of an input layer, a fusion layer, a generation layer, and an alignment module. In the input layer, a multimodal encoder group processes historical monitoring data for each modality to output multiple initial feature vectors for each modality. In the fusion layer, an alignment module is used to achieve multimodal fusion. The multimodal encoder group includes four single-modal encoders: a visual encoder, an environmental encoder, a traffic encoder, and a user preference encoder.

[0036] In one embodiment of the present application, the initial feature vector is a visual feature vector, an environmental temporal feature vector, a graph embedding feature vector, or an operational semantic feature vector.

[0037] In one embodiment of the present application, the historical visual data includes image acquisition data from a driver monitoring system (DMS). The image acquisition data includes facial expressions, gaze direction, and the like. The visual encoder processes the historical visual data using ViT-Base (a visual model based on the Transformer architecture) to output a 768-dimensional visual feature vector.

[0038] In one embodiment of the present application, the historical environmental data includes light perception data, temperature and humidity perception data, etc. The environmental encoder processes the historical environmental data using a 1D-CNN (one-dimensional convolutional neural network) based on a preset sampling rate to output a 256-dimensional environmental time series feature vector.

[0039] In one embodiment of the present application, the historical traffic data includes V2X (vehicle-to-everything) traffic topology data. The traffic encoder processes the V2X traffic topology data using a GNN (graph neural network) to output a 256-dimensional graph embedding feature vector.

[0040] In one embodiment of the present application, historical user operation preference data includes historical interaction log data such as operation click sequence and dwell time. The user preference encoder parses the historical user operation preference data using the BERT model (Bidirectional Transformer Encoder Representation Model) to output a 384-dimensional operation semantic feature vector.

[0041] In one embodiment of the present application, each unimodal encoder is pre-trained for feature recognition using historical monitoring data to establish basic feature representation capabilities for each unimodal encoder, addressing the problem of insufficient feature representation capabilities and thereby outputting an initial feature vector. The multimodal historical monitoring data can be divided into two parts: one for pre-training the unimodal encoder and the other for feature recognition based on the pre-trained unimodal encoder, resulting in multiple initial feature vectors. For example, the visual encoder can be pre-trained using 500,000 frames of DMS-annotated images from historical visual data, with labels for six states such as fatigue and distraction. The environmental encoder can be pre-trained using 100,000 sets of sensor time series data from historical environmental data, covering a temperature range of -30°C to 60°C. The traffic encoder can be pre-trained using 50,000 V2X event chains from historical traffic data, with V2X event chains classified into 12 scenarios such as accidents and congestion. The user preference encoder can be pre-trained using 300,000 historical interaction logs from historical user operation preference data, covering multiple users.

[0042] In one embodiment of the present application, during the pre-training process of feature recognition, high-quality initial feature vectors are provided for multimodal fusion through contrastive learning loss (InfoNCE) within a single modality, thereby avoiding the modal interference problem caused by directly fusing low-quality features.

[0043] In one embodiment of the present application, the contrastive learning loss is as follows:

[0044]

[0045] Among them, L contrast is the contrastive learning loss, s i、s j is the feature of the positive sample pair of the same event, and τ is the temperature parameter used to control the smoothness of the distribution.

[0046] In one embodiment of the present application, in the contrastive learning loss within a single modality, s i 、s j is the positive sample pair feature of the same event in the same modality.

[0047] In one embodiment of the present application, in the alignment module, cross-modal spatiotemporal association is established through spatiotemporal alignment and feature alignment to solve the sensor asynchrony problem and data fragmentation problem, thereby reducing the distortion of the generated virtual scene parameters.

[0048] In one embodiment of the present application, spatiotemporal alignment includes time alignment and spatial mapping alignment. The initial feature vectors corresponding to historical visual data, historical environmental data, and historical traffic data are synchronized through the Precision Time Protocol (PTP). The acquisition frequencies of historical monitoring data of different modalities are 120 Hz, 10 Hz, and 1 Hz, respectively, thereby solving the problem of cross-modal alignment error greater than 200 milliseconds. In addition, the sensor coordinate system is aligned to the vehicle coordinate system through the coordinate transformation matrix, and the coordinate alignment error is less than 3 cm.

[0049] In one embodiment of the present application, unimodal features of different modalities in the same event are aligned in a latent space according to real multimodal features to obtain generated multimodal features, including: aligning unimodal features of different modalities in the same event in a latent space through cross-modal contrastive learning and real multimodal features to obtain generated multimodal features; distinguishing the generated multimodal features from the real multimodal features through a discriminator network to adjust the alignment parameters of the latent space.

[0050] In one embodiment of the present application, cross-modal contrastive learning is used to force the alignment of the unimodal features of the same event in the latent space. For example, when the DMS detects sudden braking, it associates the obstacle warning signal of the V2X.

[0051] In one embodiment of the present application, in cross-modal contrastive learning, s i 、s j are the positive sample pair features of the same event in different modalities.

[0052] In one embodiment of the present application, a discriminator network is introduced to distinguish the generated multimodal features from the real multimodal features, so that the generated fused feature vector is aligned with the real fused feature vector, thereby improving the modality consistency. The feature alignment loss is as follows:

[0053] L align =λ1L contrast_mul+λ2L adv Formula (2)

[0054] Among them, L align is the feature alignment loss, L contrast_mul is the contrastive learning loss of cross-modal contrastive learning, λ1 is the first weight coefficient corresponding to the contrastive learning loss, L adv is the adversarial loss, and λ2 is the second weight coefficient corresponding to the adversarial loss.

[0055] In one embodiment of the present application, the first weight coefficient is determined to be 0.7 and the second weight coefficient is determined to be 0.3 through grid search.

[0056] In one embodiment of this application, by introducing inter-modal causal reasoning at the fusion layer, the feature weights of corresponding unimodal features within multimodal features can be adjusted, dynamically allocating the importance of each modality. For example, when the DMS detects a distracted driver, it automatically increases the weight of V2X traffic data from a 20% feature weight to a target weight of 65%, thereby resolving the imbalance in the importance of multimodal features and reducing the error rate in emergency decision-making by 43%.

[0057] In one embodiment of the present application, inter-modal causal reasoning is as follows: when the light sensor detects the tunnel entrance, the system synchronously triggers the DMS eye tracking verification to ensure that the dark theme does not affect the driver's line of sight, and associates the V2X positioning data to verify the activation accuracy of the navigation mode in the tunnel.

[0058] In one embodiment of the present application, the true fused feature vector is obtained based on the true multimodal features and the unadjusted feature weights, and the generated fused feature vector is obtained based on the generated multimodal feature vector and the adjusted feature weights.

[0059] In one embodiment of the present application, scene parameter generation training is performed based on real scene parameters and real fusion feature vectors to obtain a multimodal generation model, including: teacher-forced training is performed on the first generation model based on the real scene parameters and the real fusion feature vector to obtain a second generation model, and the teacher-forced training adjusts the model parameters of the first generation model through the mean square error between the first predicted scene parameters output by the first generation model and the corresponding real scene parameters, as well as the physical compliance of the first predicted scene parameters; free-running training is performed on the second generation model based on the real scene parameters, the real fusion feature vector and the preset noise vector to obtain a multimodal generation model, and the free-running training includes adjusting the model parameters of the second generation model through the optimized adversarial loss between the second predicted scene parameters output by the second generation model and the corresponding real scene parameters, as well as the physical compliance of the second predicted scene parameters; wherein the model parameters of the first generation model and the second generation model include dynamically adjusting the constraint weights of the preset scene generation constraints according to the driver's state.

[0060] In one embodiment of the present application, in the generation layer, a conditional diffusion model, such as a U-Net (a convolutional neural network) architecture, is used to generate virtual scene parameters through a 50-step denoising process with the fused feature vector as the conditional input. The input is the noise vector Z~N(0,I) + the fused feature vector F; the output is the scene description matrix S∈R NxM , N is the number of scene elements, M is the attribute dimension, and a set of virtual scene parameters is represented by the scene description matrix S. The noise vector Z is a random vector sampled from a standard normal distribution with a mean of 0 and a covariance matrix equal to the identity matrix I.

[0061] In one embodiment of the present application, the first generative model is a conditional diffusion model.

[0062] In one embodiment of the present application, in teacher-forced training, the L2 reconstruction loss (mean square error) is minimized with the real scene parameters as the condition. Here, the real fused feature vector at the previous moment is input to the first generative model to obtain the first predicted scene parameters at the next moment, and the model parameters of the first generative model are updated by the mean square error between the first predicted scene parameters and the real scene parameters at the next moment, thereby obtaining the second generative model.

[0063] In one embodiment of the present application, during free-running training, an adversarial loss (Wasserstein GAN) is optimized based on the output second predicted scene parameters. The second predicted scene parameters at the next moment are obtained by inputting a noise vector Z + a true fused feature vector F' into the second generative model. The model parameters of the second generative model are updated by optimizing the adversarial loss (Wasserstein GAN) between the second predicted scene parameters and the true scene parameters at the next moment, thereby obtaining a multimodal generative model.

[0064] In one embodiment of the present application, during the updating of the model parameters, it is also necessary to consider that the first predicted scene parameters and the second predicted scene parameters must pass the physical compliance detection of the rule engine, such as the matching of light intensity and visibility, and the rule engine is such as the CARLA simulation engine.

[0065] In one embodiment of the present application, the weight of the preset scenario generation constraint condition is dynamically adjusted according to the driver's state: for example, the weight of the interface simplification rule is increased in the fatigue state.

[0066] In one embodiment of the present application, the model parameters also include the probability of performing feature sampling on the fused feature vector.

[0067] In one embodiment of the present application, the training of the multimodal generation model relies on the obtained fusion feature vector and alignment capabilities to ensure that the predicted scene parameters conform to both the inherent correlation of the multimodal data and the physical constraints of actual driving.

[0068] In one embodiment of the present application, after the multimodal generation model is trained, multiple sets of virtual scene parameters can be generated by using a fusion feature vector generated after feature recognition of another set of multimodal historical monitoring data and a preset noise vector. The number of virtual scene parameters is 100,000 sets, covering multiple combinations such as day and night, weather, and driver status.

[0069] In one embodiment of the present application, the training of the multimodal generative model is implemented through a serving cluster equipped with PyTorch (an open source machine learning library used in fields such as natural language processing and computer vision).

[0070] In one embodiment of the present application, a multimodal generation model based on the Transformer architecture realizes the construction of virtual-real fusion scenarios, which can respond in real time to the dynamic coupling relationship between the driver's status, environmental parameters and traffic events. For example: when the DMS detects that the driver is distracted (such as the line of sight is off the road for >2 seconds) and the V2X receives an emergency braking signal, the vehicle system can generate an emergency theme layout of "warning red box + Advanced Driver Assistance Systems (ADAS) prompt on top" within 150ms. Compared with the traditional predefined scenario library, this application significantly improves the test scenario coverage of the dynamic generation of in-vehicle themes, and the scene generation efficiency of extreme driving conditions is also greatly improved.

[0071] In one embodiment of the present application, the present application solves the problem of imbalance in the importance distribution of multimodal data based on the adaptive weight distribution algorithm of the driver's state, and significantly enhances the accuracy of emergency scenario decision-making; through the combination of 4 types of heterogeneous single-modal encoders, it breaks through the limitations of traditional single-modal encoding and greatly improves the accuracy of multi-source data feature extraction, which can be increased by 20%; and through the PTP time alignment and coordinate conversion algorithm, it overcomes the problem of spatiotemporal asynchrony of multimodal data.

[0072] In step S220 , each virtual scene parameter is input into the theme generation model to generate a vehicle theme parameter corresponding to each virtual scene parameter, and a display test is performed on each vehicle theme parameter to obtain a plurality of target test results.

[0073] In one embodiment of the present application, the topic generation model can be implemented through a large language model.

[0074] In one embodiment of the present application, the vehicle computing platform is activated, the language model is loaded, and vehicle theme parameters corresponding to the virtual scene parameters are generated. The vehicle theme parameters include layout scheme, color configuration and control priority.

[0075] In one embodiment of the present application, a display test is performed on the vehicle-mounted theme parameters to obtain a target test result, including: verifying the functional correctness of the vehicle-mounted theme parameters according to preset single-modal rules to obtain a first test result; calling the vehicle-mounted rendering engine to generate a visualization interface of the vehicle-mounted theme parameters, and comparing the actual performance of the visualization interface with a preset performance benchmark to obtain a second test result, wherein the actual performance includes at least one of theme switching delay, image processing memory occupancy, and system crash rate corresponding to the noise stress test; performing cross-modal rule verification on the vehicle-mounted theme parameters according to preset multi-modal rules to obtain a third test result; and determining the target test result based on the first test result, the second test result, and the third test result.

[0076] In one embodiment of the present application, functional correctness verification and performance benchmark tests corresponding to visualization can be executed in parallel, sharing the same data input source, to ensure consistency of test results.

[0077] In one embodiment of the present application, the in-vehicle theme parameters are checked one by one through preset single-modal rules, for example, entertainment controls should not be displayed in a distracted state.

[0078] In one embodiment of the present application, the first test result also includes checking whether the functional coverage meets the quantitative indicators.

[0079] In one embodiment of the present application, the theme switching delay is used to represent the time from when the theme generation model outputs the vehicle theme parameters to when the interface rendering is completed. If the theme switching delay is less than a preset delay time threshold, the theme switching delay performance test passes. The preset delay time threshold can be set to 200 milliseconds or 150 milliseconds.

[0080] In one embodiment of the present application, the image processing memory usage is the memory usage of the graphics processing unit (GPU) to verify the resource efficiency of the topic generation model reasoning. If the image processing memory usage is less than a preset memory usage threshold, the image processing memory usage performance test is passed. The preset memory usage threshold can be set to 1.5GB.

[0081] In one embodiment of the present application, a noise stress test measures the system crash rate when injected with noise data, such as sensor failures and signal delays. If the system crash rate is less than a preset crash rate threshold, the test passes. The preset crash rate threshold can be set to 0.01%.

[0082] In one embodiment of the present application, the preset performance benchmark includes a preset delay time threshold, a preset memory usage threshold, and a preset crash rate threshold.

[0083] In one embodiment of the present application, pre-defined multimodal rules are used to characterize causal rules across modalities to address the issue of misjudgment of unimodal triggering. The accuracy of rule triggering is verified using a logic inference engine (such as Drools), and simulation data shows that the accuracy of cross-modal rule verification is significantly improved.

[0084] In one embodiment of the present application, after determining the target test result based on the first test result, the second test result and the third test result, it also includes: if the target test result of a vehicle-mounted theme parameter is failed, the vehicle-mounted theme parameter is determined as an abnormal theme parameter, and the virtual scene parameter corresponding to the abnormal theme parameter is determined as an abnormal virtual scene; if there is an interface rendering error in the abnormal theme parameter, the allocation of rendering resources is detected to obtain first abnormal information; if there is no interface rendering error in the abnormal theme parameter, and there is a missed physical compliance in the abnormal virtual scene, the confidence of the abnormal virtual scene is retroactively generated to obtain second abnormal information; if there is no interface rendering error in the abnormal theme parameter, and there is no missed physical compliance in the abnormal virtual scene, the synchronization of multimodal historical monitoring data is verified to obtain third abnormal information; cluster analysis is performed on the abnormal virtual scene based on all target abnormal information to obtain defective scene parameters, and the target abnormal information includes at least one of the first abnormal information, the second abnormal information and the third abnormal information.

[0085] In one embodiment of the present application, the rendering resources include GPU resources.

[0086] In one embodiment of the present application, a method combining a decision tree and a Bayesian network is used to analyze the vehicle theme parameters and corresponding virtual scene parameters that failed the test. Figure 4 , Figure 4 FIG. 1 shows an analysis decision diagram of a test failure according to an embodiment of the present application. Figure 4 As shown, if the onboard theme parameter test fails, the system proceeds to the interface rendering error determination step. For interface rendering error determination, if an interface rendering error exists, GPU resource allocation is checked. If not, the system proceeds to the rule engine missed detection step. For rule engine missed detection, if a rule engine missed a detection, the confidence level of the virtual scene parameters generated by the multimodal generation model is tracked. If the rule engine does not miss a detection, the system verifies the synchronization of sensor data, which is the historical multimodal monitoring data used by the rule engine to determine the physical compliance of the virtual scene parameters. Through code simulation, the mean time to locate faults is reduced by at least 50% compared to manual analysis.

[0087] In one embodiment of the present application, cluster analysis is used to identify abnormal patterns other than abnormal virtual parameters, and to identify test scenarios not covered by the rule engine rules corresponding to physical compliance detection, such as the background brightness of the map card not being adapted in daylight or night mode.

[0088] In one embodiment of the present application, online adversarial tuning is used to continuously improve the diversity of generated virtual scene parameters and the coverage of extreme working conditions. The number of target test results corresponding to the same type of virtual scene parameters that fail is used to identify extreme scene parameters with low test case pass rates, such as the combination of strong backlight and the driver wearing sunglasses. The probability of feature sampling in the fused feature vector is increased (e.g., by a factor of three), and gradient penalty constraints are introduced in an improved version of the generative adversarial network (e.g., Wasserstein GAN) to stabilize the adversarial training process.

[0089] In one embodiment of the present application, in the present application, the training process of the multimodal generation model adopts a four-stage progressive training strategy, stage 1: pre-training of each single modal encoder; stage 2: multimodal alignment training; stage 3: generation training of virtual scene parameters, combining teacher forcing and free running; stage 4: online adversarial training, continuously improving the coverage of virtual scene parameters.

[0090] In one embodiment of the present application, see Figure 5 , Figure 5 FIG. 1 shows a logic diagram of a dynamic generation test of an in-vehicle theme according to an embodiment of the present application. Figure 5As shown, the test environment is initialized: the topic generation model is loaded, and the test scripts and preset verification rule libraries, such as preset unimodal rules and preset multimodal rules, are deployed to establish a standardized execution environment for subsequent tests to ensure that the test results are reproducible; virtual scene parameters are generated: the multimodal generation model is called to obtain multiple sets of virtual scene parameters; dynamic topic generation triggering: the vehicle topic parameters corresponding to the virtual scene parameters are generated through the topic generation model; parallel verification branch: parallel testing of functional correctness verification and performance benchmark testing is implemented; functional correctness verification: the functional correctness of the vehicle topic parameters is verified according to the preset unimodal rules to obtain the first test result; performance benchmark testing: the actual performance of the visualization interface of the vehicle topic parameters is compared with the preset performance benchmark to obtain the second test result; multimodal decision consistency check: the vehicle topic parameters are cross-modal rule verified according to the preset multimodal rules to obtain the third test result; abnormal root cause location: the root cause of the abnormal topic parameters is located by combining the decision tree and the Bayesian network to obtain the target abnormality information; automatic test report generation: the target test results and target abnormality information of the vehicle topic parameters are output; model continuous optimization: the multimodal generation model is continuously optimized through online adversarial training. Through the above-mentioned full-process automated test closed loop: parallel verification of functions / performance + cross-modal causal reasoning, the misjudgment rate is reduced to ≤2%, the test efficiency is improved by more than 50%, and the self-iteration of "virtual scene parameter generation - testing of vehicle-mounted theme parameters - model optimization" is achieved.

[0091] In step S230 , if the target test result of a vehicle-mounted theme parameter is passed, the vehicle-mounted theme parameter is input into the human factor scoring model for simulation evaluation to obtain a human factor simulation score.

[0092] Among them, the human factors scoring model is trained based on historical functional operation data, historical scenario parameters and historical user task scores.

[0093] In one embodiment of the present application, the historical function operation data, historical scenario parameters, and historical user task scores are obtained based on the first simulated driving experiment and / or real driving data.

[0094] In one embodiment of this application, users of varying ages and driving experience are recruited to complete pre-set tasks in a simulated cockpit. Their actions and Net Promoter Score (NPS) scores are recorded. Pre-set tasks include navigation settings and music switching; actions include touch click coordinates, gesture trajectories, and the duration between actions. NPS scores are calculated immediately after each task to avoid memory bias and are rated on a scale of 0-10.

[0095] In one embodiment of the present application, under the premise of compliance, anonymized operation logs and user-initiated ratings are collected to obtain real driving data.

[0096] In one embodiment of the present application, the training data format for the human factors scoring model consists of an operation sequence corresponding to historical functional operation data, an NPS tag corresponding to historical user task scores, and an environmental context corresponding to historical scenario parameters. The operation sequence can record 200 operation events with 0.01-second accuracy, totaling 30 seconds. The NPS tag contains the user's satisfaction rating of the interface design during that time period. The environmental context includes driving conditions (e.g., highway, congestion) and time (daytime / nighttime).

[0097] In one embodiment of the present application, 10,000 sets of valid data that meet the training data format are used as the basic training set, which can cover more than 100 interface designs. The valid data is mirrored and perturbed to generate 3 times expanded data, thereby enhancing the data.

[0098] In one embodiment of the present application, a human factors scoring model is trained using a Long Short-Term Memory Network (LSTM) model to analyze time series data (e.g., continuous operation actions) and capture the long-term dependencies of operating habits. The simplified structure of the LSTM model is as follows: the input layer receives a time-ordered sequence of operations, such as click location coordinates, gesture type, sliding speed, and operation frequency; the memory layer uses recurrent network units to identify operating patterns, such as frequent return corrections → excessive interface complexity; and the output layer predicts the NPS score, such as a continuous value from 0 to 10.

[0099] In one embodiment of the present application, the training steps of the human factor scoring model include data cleaning, standardization, batch training and anti-overfitting measures. Among them, data cleaning is used to filter invalid data, such as interference fragments where the user makes a phone call in the middle of the training. Standardization is used to convert screen coordinates into relative position ratios, such as 0% to 100%. Batch training includes dividing the data into 100 groups / batches, each group contains 50 operation sequences, and the LSTM model automatically learns the association between operation features and scores, for example: positive association: high sliding smoothness → score +1.2 points, negative association: frequent false touches → score -2.5 points. Anti-overfitting measures include random dropout and early stopping mechanism. Random dropout means randomly ignoring some non-critical operation nodes during training, such as short stops; the early stopping mechanism includes automatic termination when the accuracy rate does not improve after 10 consecutive trainings.

[0100] In one embodiment of the present application, the effectiveness of the human factors rating model is verified: the mean absolute error (MAE) is 0.6 points, that is, the average difference between the predicted value and the actual score is 0.6 points, and the consistency ratio is 89%, that is, 89% of the human factors simulation scores deviate from the user's actual feelings by ≤1 point.

[0101] In one embodiment of this application, after learning the correlation between operational characteristics and scores, the human-factor scoring model can directly simulate and evaluate the in-vehicle theme parameters, as they correlate with theoretical operational characteristics, to obtain a simulated human-factor score. Alternatively, in actual use, the in-vehicle theme parameters can be predicted and evaluated based on the user's real-time operational characteristics to obtain a predicted human-factor score. By replacing traditional questionnaires with a human-factor scoring model, user experience evaluation can be performed seamlessly and in real time, increasing evaluation efficiency by at least five times.

[0102] In one embodiment of the present application, the model is continuously upgraded by collecting 10,000 new sets of real driving data each month to incrementally train the human factors scoring model; the model version is updated every quarter to reduce the scoring error by 0.2 points per time.

[0103] In step S240, if the human factors simulation score of an in-vehicle theme parameter is less than a preset simulation score threshold, the in-vehicle theme parameter is determined as the target theme parameter, and simulated driving is performed on the target theme parameter. The target theme parameter is optimized and trained based on the driver perception data and actual functional operation data in the simulated driving to obtain a human factors optimization model and human factors optimization suggestions.

[0104] In one embodiment of the present application, all target theme parameters are used as driving theme parameters for the second simulated driving.

[0105] In one embodiment of the present application, target theme parameters are optimized and trained based on driver perception data and actual functional operation data in simulated driving to obtain a human factors optimization model and human factors optimization suggestions, including: obtaining driver perception data and actual functional operation data in simulated driving, the driver perception data including eye movement perception data and physiological perception data, and the actual functional operation data including actual operation steps and actual path length; determining an operation efficiency index based on a first gap coefficient between preset theoretical operation steps and actual operation steps, and a second gap coefficient between a preset theoretical path length and actual path length; determining an actual distraction index based on the eye movement perception data and the physiological perception data; determining a target optimization strategy from multiple preset optimization strategies based on a first comparison result of the operation efficiency index with multiple preset operation efficiency intervals, and a second comparison result of the actual distraction index with multiple preset distraction index intervals; updating the target theme parameters based on the target optimization strategy, and performing simulated driving based on the updated target theme parameters to adjust each preset operation efficiency interval, each preset distraction index interval, and each preset optimization strategy to obtain a human factors optimization model and human factors optimization suggestions.

[0106] In one embodiment of the present application, an actual distraction index is determined based on eye movement perception data and physiological perception data, including: determining the first fixation duration and the eye movement deviation duration based on the eye movement perception data; determining the cognitive load based on a preset first coefficient, the number of skin conductance reactions, a preset second coefficient, a heart rate power ratio, a preset third coefficient, and a skin conductance level, wherein the number of skin conductance reactions and the skin conductance level are obtained based on the galvanic skin perception data, the heart rate power ratio is obtained based on the heart rate perception data and is used to characterize the power ratio between low frequency and high frequency, and the physiological perception data includes the galvanic skin perception data and the heart rate perception data; determining the increase in heart rate variability based on the heart rate power ratio and the baseline value in the heart rate perception data; and determining the actual distraction index based on the first fixation duration, the eye movement deviation duration, the increase, and the cognitive load.

[0107] In one embodiment of the present application, the first gap coefficient is obtained based on the ratio of the preset theoretical operation steps to the actual operation steps, and the second gap coefficient is obtained based on the ratio of the preset theoretical path length to the actual path length. The operation efficiency index is determined as follows:

[0108]

[0109] In one embodiment of the present application, in the second simulated driving, the eye tracker is used to record the distribution of gaze focus and output the gaze point coordinate sequence (x t ,y t ), t = 1, 2, ..., T, and thus calculate the attention paid to the topic elements, such as the proportion of dwell time in the navigation area. Furthermore, physiological sensors such as heart rate sensors and galvanic skin sensors can be used to assess changes in cognitive load when switching topics.

[0110] In one embodiment of the present application, the physiological sensor is a wearable physiological signal monitoring device, and the sampling rate of the eye tracker is 120 Hz.

[0111] In one embodiment of the present application, determining the degree of attention to a theme element includes: pre-segmenting the elements of a screenshot of the in-vehicle interface corresponding to the target theme parameters, thereby calibrating the interface element area; mapping the gaze point coordinates to the interface element area; and counting the attention time of the interface element area to obtain the percentage of dwell time and the first gaze time.

[0112] In one embodiment of the present application, the gaze point coordinates are mapped to the interface element area as follows:

[0113]

[0114] Among them, (x t ,y t ) is the gaze point coordinate at time t, and Polygon(element) is the polygonal area Polygon to which the interface element element belongs.

[0115] In one embodiment of the present application, the dwell time ratio is determined as follows:

[0116]

[0117] Among them, Attention Ratio_i is the proportion of the dwell time of the i-th interface element, δ(Element i ,t) is the indicator function, δ=1 means looking at the i-th interface element at time t, T total is the total fixation duration.

[0118] In one embodiment of the present application, the first fixation duration is determined as follows:

[0119] T FirstFix_i =min{t|(x t ,y t )∈Element i Formula (6)

[0120] Among them, T FirstFix_i is the first fixation duration corresponding to the i-th interface element, (x t ,y t ) is the gaze point coordinate at time t, Element i is the i-th interface element.

[0121] In one embodiment of the present application, the first gaze duration is used to represent the duration corresponding to the first gaze at an interface element.

[0122] In one embodiment of the present application, the eye movement deviation duration is used to represent the duration of time the user's gaze is moved away from a preset interface element, which can be obtained based on the dwell time ratio. The preset interface element is, for example, a navigation element.

[0123] In one embodiment of the present application, heart rate variability (HRV) analysis is performed on heart rate sensing data to obtain a frequency domain metric: the LF / HF ratio (low-frequency / high-frequency power ratio). This is correlated with cognitive load level, e.g., a 15% increase in the LF / HF ratio results in a cognitive load level increase of 1. Electrodermal activity (EDA) analysis is performed on galvanic skin sensing data to obtain the skin conductance response (SCR) and skin conductance level (SCL). The SCR is the number of peaks within a window period, and the SCL is the baseline value after removing the trend term.

[0124] In one embodiment of the present application, the coefficients between SCR, SCL and LF / HF are determined by ridge regression, wherein the preset third coefficient is compressed and the goodness of fit R of the ridge regression is 2 is 0.89. The cognitive load is as follows:

[0125] Cognitive Load = 0.43 × SCR + 0.57 × LF / HF Formula (7)

[0126] Among them, CognitiveLoad is cognitive load, 0.43 is the value of the preset first coefficient, SCR is the number of skin conductance responses, 0.57 is the value of the preset second coefficient, and LF / HF is the heart rate power ratio.

[0127] In one embodiment of the present application, if the percentage of eye movement deviation duration exceeds a preset deviation duration, the risk is considered high. If the increase in heart rate variability exceeds a preset amplitude threshold, the workload is considered high. The preset deviation duration can be set to 15%, and the preset amplitude threshold can be set to 20%.

[0128] In one embodiment of the present application, the preset operation efficiency interval includes three level intervals, which can be set as 0-0.4, 0.4-0.6, and greater than 0.6. The preset distraction index interval includes three level intervals, which can be set as greater than 0.7, 0.4-0.7, and less than 0.4. By combining the operation efficiency index and the actual distraction index, a layout optimization solution is recommended, namely a target optimization strategy.

[0129] In one embodiment of the present application, the interface layout defects corresponding to the vehicle-mounted theme parameters can be accurately discovered through line of sight focus analysis, and the interface optimization of the vehicle-mounted theme parameters can be made more targeted through the operation efficiency index.

[0130] In one embodiment of the present application, the decision matrix corresponding to the target optimization strategy is as follows:

[0131] Table 1 Decision matrix corresponding to the target optimization strategy

[0132] Preset operating efficiency range Preset distraction index range Preset optimization strategy Low (0-0.4) High (>0.7) Restructure the functional hierarchy and increase the touch area Medium (0.4-0.6) Medium (0.4-0.7) High-frequency functions are placed on top + visual elements are simplified High (>0.6) Low (<0.4) Maintain design + monitor long-term stability

[0133] In one embodiment of the present application, simulated driving is performed based on updated target subject parameters to adjust each preset operating efficiency interval, each preset distraction index interval, and each preset optimization strategy to obtain a human factors optimization model, including: obtaining new driver perception data and new actual functional operation data in the simulated driving to obtain a new target optimization strategy; adjusting each preset operating efficiency interval, each preset distraction index interval, and each preset optimization strategy according to the degree of influence between the target optimization strategy and the new target optimization strategy on the driver to obtain the human factors optimization model.

[0134] In one embodiment of the present application, adjusting the preset optimization strategy includes adjusting the optimization value. For example, the optimization value corresponding to increasing the touch area is +20%, that is, increasing the touch area by 20% based on the current touch area. However, the increase of 20% at different sizes has different effects on the driver, so the optimization value needs to be adjusted.

[0135] In one embodiment of the present application, the human factors optimization suggestion includes a target optimization strategy in the human factors optimization process and at least one optimized subject parameter.

[0136] In one embodiment of the present application, the trained human factors simulation scoring and human factors optimization model are embedded in the vehicle computer system during actual application. The user's operation is automatically recorded and segmented in the background, such as every 30 seconds. The human factors simulation scoring instantly outputs a human factors predicted score, such as 7.8 points, and marks segments with abnormally low scores.

[0137] In one embodiment of the present application, low-scoring segments are automatically associated with the action heat map. For example, if a button area has a click failure rate exceeding 40%, eye tracking data is combined to verify whether this is due to an unreasonable layout. If the human factor prediction score is lower than the preset prediction score threshold for multiple consecutive times, the human factor optimization model is triggered to optimize the interface.

[0138] In one embodiment of the present application, the present application solves the problem of large subjective evaluation errors through a multimodal physiological-behavioral data fusion evaluation system, and realizes causal correlation analysis of eye movements, operations, and physiological data through a dynamic weighted attention mechanism, and lightweight human factors optimization model and human factors evaluation model embedded deployment and online incremental learning technology to achieve real-time prediction of user satisfaction and interface self-optimization.

[0139] In one embodiment of the present application, the scenario generation coverage corresponding to the virtual scenario parameters generated by the present application is greater than or equal to 95% of the real working condition matching degree; through human factors optimization suggestions, the distraction index is reduced by ≥35%, the operation steps are simplified by ≥25%, and the test efficiency is improved by ≥50%.

[0140] In one embodiment of this application, the virtual scene parameters generated by the MGM are "light intensity 10 Lux, driver fatigue index 65." The in-vehicle theme parameters output by the theme generation model are "dark theme + simplified navigation." Functional correctness verification during display testing includes checking, through test scripts, whether the navigation bar is pinned to the top and whether the entertainment controls are hidden. During the human factors optimization process, it was recorded that the time drivers spent looking away from the road decreased by 30% and their heart rate variability decreased by 15% in the simplified interface.

[0141] In another embodiment of the present application, the virtual scene parameter generated by MGM is "simulating the co-pilot passenger triggering the split-screen demand." The in-vehicle theme parameter output by the theme generation model is "main driver navigation + co-pilot video." The functional correctness verification in the display test includes: verifying through the test script whether the ADAS warning information is displayed first in the split-screen mode, whether the navigation information is highlighted by the main driver, whether the video interface interferes with driving, and whether the interface display is abnormal. During the human factors optimization process, the length of time the driver's line of sight deviates from the road and the length of time the driver stares at the car screen do not increase, and the false touch rate of the co-pilot user operation is reduced by 30%.

[0142] In one embodiment of the present application, the test report includes multi-dimensional evaluation data such as use case pass rate, main problem level, defect scenario parameters, human factors optimization suggestions, confidence of the multimodal generation model, target abnormality information and operation heat map, and is fed back to the multimodal generation model to continuously iterate and improve the matching degree between the generated virtual scene parameters and the actual working conditions.

[0143] In one embodiment of the present application, the dynamic generation test method of the vehicle-mounted theme provided by the present application can reduce the traditional manual test time from 3 man-days / scene to 0.5 man-days / scene, thus shortening the test cycle.

[0144] See also Figure 6 , Figure 6 A block diagram of a dynamic generation test device for vehicle-mounted themes according to an embodiment of the present application is shown. The device can be applied to Figure 1 The implementation environment shown is as follows. The apparatus may also be applicable to other exemplary implementation environments and specifically configured in other devices. This embodiment does not limit the implementation environment to which the apparatus is applicable.

[0145] like Figure 4 As shown, a dynamic generation test device 600 for an in-vehicle theme according to an embodiment of the present application includes: a scene parameter generation module 601, a theme parameter testing module 602, a human factor scoring module 603 and a human factor optimization module 604.

[0146] The scene parameter generation module 601 is used to call the multimodal generation model to generate multiple sets of virtual scene parameters;

[0147] The theme parameter testing module 602 is used to input each virtual scene parameter into the theme generation model, generate vehicle theme parameters corresponding to each virtual scene parameter, and perform display testing on each vehicle theme parameter to obtain multiple target test results;

[0148] The human factor scoring module 603 is configured to input the vehicle-mounted theme parameter into the human factor scoring model for simulation evaluation to obtain a human factor simulation score if the target test result of the vehicle-mounted theme parameter is passed;

[0149] The human factors optimization module 604 is configured to determine the vehicle-mounted theme parameter as a target theme parameter if the human factors simulation score of the vehicle-mounted theme parameter is less than a preset simulation score threshold, conduct a simulated driving for the target theme parameter, and optimize the target theme parameter based on the driver's perception data and actual functional operation data during the simulated driving to obtain a human factors optimization model and human factors optimization suggestions;

[0150] Among them, the multimodal generation model is trained based on the fusion of real scene parameters and multimodal historical monitoring data, and the human factors scoring model is trained based on historical functional operation data, historical scene parameters and historical user task scores.

[0151] It should be noted that the dynamic generation test device for vehicle-mounted themes provided in the above embodiment and the dynamic generation test method for vehicle-mounted themes provided in the above embodiment belong to the same concept, wherein the specific manner in which each module and unit performs operations has been described in detail in the method embodiment and will not be repeated here. In actual applications, the dynamic generation test device for vehicle-mounted themes provided in the above embodiment can allocate the above functions to different functional modules as needed, that is, divide the internal structure of the device into different functional modules to complete all or part of the functions described above, and this is not limited here.

[0152] An embodiment of the present application also provides an electronic device, comprising: one or more processors; a storage device for storing one or more programs, which, when executed by one or more processors, enables the electronic device to implement the dynamic generation test method of vehicle-mounted themes provided in the above-mentioned embodiments.

[0153] See also Figure 7 , Figure 7 The following is a schematic diagram showing the structure of a computer system suitable for implementing an electronic device according to an embodiment of the present application. Figure 7 The computer system 700 of the electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0154] like Figure 7As shown, the computer system 700 includes a central processing unit (CPU) 701, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 702 or the program loaded from the storage part 708 into the random access memory (RAM) 703, such as executing the method in the above embodiment. Various programs and data required for system operation are also stored in the RAM 703. The CPU 701, ROM 702 and RAM 703 are connected to each other via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0155] The following components are connected to the I / O interface 705: an input section 706 including a keyboard, a mouse, and the like; an output section 707 including devices such as a cathode ray tube (CRT), a liquid crystal display (LCD), and a speaker; a storage section 708 including a hard disk and the like; and a communication section 709 including a network interface card such as a LAN (Local Area Network) card or a modem. The communication section 709 performs communication processing via a network such as the Internet. A drive 710 is also connected to the I / O interface 705 as needed. Removable media 711, such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory, is installed in the drive 710 as needed, so that computer programs read therefrom can be installed into the storage section 708 as needed.

[0156] In particular, according to an embodiment of the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a computer program for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network via the communication section 709, and / or installed from a removable medium 711. When the computer program is executed by the central processing unit (CPU) 701, the various functions defined in the system of the present application are executed.

[0157] It should be noted that the computer-readable medium shown in the embodiments of the present application may be a computer-readable signal medium or a computer-readable storage medium or any combination of the two. The computer-readable storage medium may be, for example, an electrical, magnetic, optical, electromagnetic, infrared or semiconductor system, device or component, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries a computer-readable computer program. This propagated data signal may take a variety of forms, including but not limited to an electromagnetic signal, an optical signal or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. A computer program embodied on a computer-readable medium may be transmitted using any suitable medium, including but not limited to wireless, wired, or any suitable combination thereof.

[0158] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions and operations of the systems, methods and computer program products according to various embodiments of the present application. Each box in the flowchart or block diagram can represent a module, program segment or part of the code, and the above-mentioned module, program segment or part of the code contains one or more executable instructions for implementing the specified logical functions. It should also be noted that in some alternative implementations, the functions marked in the boxes can also occur in an order different from that marked in the accompanying drawings. For example, two boxes shown in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart and the combination of boxes in the block diagram or flowchart can be implemented using a dedicated hardware-based system that performs the specified functions or operations, or can be implemented using a combination of dedicated hardware and computer instructions.

[0159] The units involved in the embodiments described in the present application can be implemented by software or by hardware, and the units described can also be set in a processor. Among them, the names of these units do not constitute a limitation on the units themselves under certain circumstances. Therefore, the technical solution according to the embodiment of the present application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, and includes several instructions to enable a computing device (which can be a personal computer, a server, a touch terminal, or a network device, etc.) to execute the method according to the embodiment of the present application.

[0160] Another aspect of the present application provides a computer-readable storage medium having a computer program stored thereon. When executed by a computer processor, the computer program causes the computer to perform the dynamic generation and testing methods for vehicle-mounted themes provided in the aforementioned embodiments. The computer-readable storage medium may be included in the electronic device described in the aforementioned embodiments, or may exist independently and not be incorporated into the electronic device.

[0161] In the above embodiments, unless otherwise specified, the use of serial numbers such as "first" and "second" to describe common objects only indicates that they refer to different instances of the same object, rather than indicating that the objects being described must adopt a given order, whether in time, space, sorting or any other way.

[0162] The above embodiments are merely illustrative of the principles and effects of this application and are not intended to limit this application. Anyone skilled in the art may modify or alter the above embodiments without departing from the spirit and scope of this application. Therefore, any equivalent modifications or alterations accomplished by a person of ordinary skill in the art without departing from the spirit and technical concepts disclosed in this application shall be covered by the claims of this application.

Claims

1. A dynamic generation test method for vehicle-mounted themes, characterized in that: The method comprises: Calling the multimodal generation model to generate multiple sets of virtual scene parameters; Inputting each of the virtual scene parameters into a theme generation model respectively, generating vehicle theme parameters corresponding to each of the virtual scene parameters, and performing a display test on each of the vehicle theme parameters to obtain multiple target test results; If the target test result of a vehicle-mounted theme parameter is passed, the vehicle-mounted theme parameter is input into the human factor scoring model for simulation evaluation to obtain a human factor simulation score; If the human factors simulation score of a vehicle-mounted theme parameter is less than a preset simulation score threshold, the vehicle-mounted theme parameter is determined as a target theme parameter, a simulated driving is performed for the target theme parameter, and the target theme parameter is optimized and trained based on the driver's perception data and actual functional operation data during the simulated driving to obtain a human factors optimization model and human factors optimization suggestions; Among them, the multimodal generation model is obtained by training based on the fusion of real scene parameters and multimodal historical monitoring data, and the human factors scoring model is obtained by training based on historical functional operation data, historical scene parameters and historical user task scores.

2. The method for dynamically generating and testing vehicle-mounted themes according to claim 1, wherein: The training of the multimodal generative model includes: Acquiring real scene parameters and multimodal historical monitoring data, wherein the historical monitoring data includes historical visual data, historical environmental data, historical traffic data, and historical user operation preference data; Pre-training the corresponding unimodal encoder for feature recognition based on historical monitoring data of each modality to obtain multiple initial feature vectors for each modality, each of the initial feature vectors including multiple unimodal features; Performing multimodal fusion on each of the initial feature vectors to obtain multiple fused feature vectors, wherein the multimodal fusion includes performing spatiotemporal alignment and feature alignment training on each of the initial feature vectors to obtain multiple multimodal features, and performing inter-modal causal reasoning on the multimodal features to adjust the feature weights of corresponding unimodal features in the multimodal features; the feature alignment training is used to align unimodal features of different modalities in the same event in a latent space according to real multimodal features to obtain generated multimodal features, wherein the real multimodal features are obtained based on the initial feature vectors after spatiotemporal alignment; Scene parameter generation training is performed based on the real scene parameters and the real fusion feature vector to obtain a multimodal generation model.

3. The method for dynamically generating a vehicle theme according to claim 2, wherein: The method further comprises: performing scene parameter generation training according to the real scene parameters and the real fusion feature vector to obtain a multimodal generation model, including: Performing teacher forced training on the first generative model according to the real scene parameters and the real fusion feature vector to obtain a second generative model, wherein the teacher forced training adjusts model parameters of the first generative model through a mean square error between first predicted scene parameters output by the first generative model and corresponding real scene parameters, and physical compliance of the first predicted scene parameters; performing free-running training on the second generative model based on the real scene parameters, the real fused feature vector, and the preset noise vector to obtain a multimodal generative model, wherein the free-running training includes adjusting model parameters of the second generative model through an optimized adversarial loss between second predicted scene parameters output by the second generative model and corresponding real scene parameters, and physical compliance of the second predicted scene parameters; The model parameters of the first generation model and the second generation model include constraint weights for dynamically adjusting the preset scenario generation constraint conditions according to the driver's state.

4. The method for dynamically generating and testing vehicle-mounted themes according to claim 1, wherein: The target subject parameters are optimized and trained based on the driver perception data and actual functional operation data in the simulated driving to obtain a human factors optimization model and human factors optimization suggestions, including: Acquiring driver perception data and actual functional operation data of the simulated driving, wherein the driver perception data includes eye movement perception data and physiological perception data, and the actual functional operation data includes actual operation steps and actual path length; determining an operation efficiency index according to a first gap coefficient between a preset theoretical operation step and the actual operation step, and a second gap coefficient between a preset theoretical path length and the actual path length; determining an actual distraction index based on the eye movement perception data and the physiological perception data; determining a target optimization strategy from among a plurality of preset optimization strategies based on a first comparison result of the operation efficiency index and a plurality of preset operation efficiency intervals, and a second comparison result of the actual distraction index and a plurality of preset distraction index intervals; The target theme parameters are updated according to the target optimization strategy, and simulated driving is performed based on the updated target theme parameters to adjust each of the preset operating efficiency intervals, each of the preset distraction index intervals, and each of the preset optimization strategies to obtain a human factors optimization model and human factors optimization recommendations.

5. The method for dynamically generating and testing vehicle-mounted themes according to claim 4, wherein: Determining an actual distraction index based on the eye movement perception data and the physiological perception data includes: Determining the first fixation duration and the eye movement deviation duration according to the eye movement perception data; determining cognitive load based on a preset first coefficient, a number of skin conductance reactions, a preset second coefficient, a heart rate power ratio, a preset third coefficient, and a skin conductance level, wherein the number of skin conductance reactions and the skin conductance level are obtained based on galvanic skin perception data, the heart rate power ratio is obtained based on the heart rate perception data and is used to characterize the power ratio between low frequencies and high frequencies, and the physiological perception data includes the galvanic skin perception data and the heart rate perception data; determining an increase in heart rate variability based on the heart rate-to-power ratio and a baseline value in the heart rate sensing data; An actual distraction index is determined according to the first fixation duration, the eye movement deviation duration, the rising amplitude and the cognitive load.

6. The method for dynamically generating and testing an in-vehicle theme according to any one of claims 1 to 5, characterized in that: Perform a display test on the vehicle-mounted theme parameters to obtain target test results, including: Performing functional correctness verification on the vehicle-mounted theme parameters according to a preset single-modal rule to obtain a first test result; Invoking an onboard rendering engine to generate a visualization interface of the onboard theme parameters, and comparing actual performance of the visualization interface with a preset performance benchmark to obtain a second test result, the actual performance comprising at least one of theme switching latency, image processing memory usage, and system crash rate corresponding to a noise stress test; Performing cross-modal rule verification on the vehicle-mounted theme parameters according to a preset multimodal rule to obtain a third test result; A target test result is determined according to the first test result, the second test result, and the third test result.

7. The method for dynamically generating and testing vehicle-mounted themes according to claim 6, wherein: After determining the target test result according to the first test result, the second test result, and the third test result, the method further includes: If a target test result of an in-vehicle theme parameter is failed, the in-vehicle theme parameter is determined as an abnormal theme parameter, and the virtual scene parameter corresponding to the abnormal theme parameter is determined as an abnormal virtual scene; If the abnormal theme parameter has an interface rendering error, detecting the allocation of rendering resources to obtain first abnormal information; If there is no interface rendering error in the abnormal theme parameter, and there is missed physical compliance in the abnormal virtual scene, then the confidence level of the abnormal virtual scene is retroactively generated to obtain second abnormal information; If there is no interface rendering error in the abnormal theme parameter and there is no missed physical compliance in the abnormal virtual scene, then verifying the synchronization of multimodal historical monitoring data to obtain third abnormal information; Cluster analysis is performed on the abnormal virtual scene according to all target abnormal information to obtain defect scene parameters, where the target abnormal information includes at least one of the first abnormal information, the second abnormal information, and the third abnormal information.

8. A dynamic generation test device for vehicle-mounted themes, characterized in that: The device comprises: A scene parameter generation module is used to call a multimodal generation model to generate multiple sets of virtual scene parameters; a theme parameter testing module, configured to input each of the virtual scene parameters into a theme generation model, generate vehicle theme parameters corresponding to each of the virtual scene parameters, and perform a display test on each of the vehicle theme parameters to obtain a plurality of target test results; A human factors scoring module is configured to input a vehicle-mounted theme parameter into a human factors scoring model for simulation evaluation to obtain a human factors simulation score if a target test result of the vehicle-mounted theme parameter is passed; a human factors optimization module configured to, if a human factors simulation score of a vehicle-mounted theme parameter is less than a preset simulation score threshold, determine the vehicle-mounted theme parameter as a target theme parameter, conduct a simulated driving for the target theme parameter, and optimize the target theme parameter based on driver perception data and actual functional operation data during the simulated driving to obtain a human factors optimization model and human factors optimization suggestions; Among them, the multimodal generation model is obtained by training based on the fusion of real scene parameters and multimodal historical monitoring data, and the human factors scoring model is obtained by training based on historical functional operation data, historical scene parameters and historical user task scores.

9. An electronic device, characterized in that: The electronic device comprises: one or more processors; A storage device for storing one or more programs, which, when executed by the one or more processors, enables the electronic device to implement the dynamic generation test method of the vehicle-mounted theme as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that A computer program is stored thereon, and when the computer program is executed by a processor of a computer, the computer is caused to execute the dynamic generation test method of the in-vehicle theme according to any one of claims 1 to 7.

Images