HMI touch interaction evaluation method and device and electronic equipment

Abstract

The invention provides an HMI touch interaction evaluation method and device and electronic equipment. The method comprises the steps that multiple evaluation dimensions are determined; the multiple evaluation dimensions comprise the following six dimensions: interactive interface visual coordination, operation level rationality, man-machine convenience, text and icon understandability, function integrity and safety; obtaining a weight corresponding to each evaluation dimension in the plurality of evaluation dimensions; determining an evaluation object; collecting test data; the test data comprises subjective feedback data and objective test data; obtaining scores corresponding to the evaluation dimensions according to the subjective feedback data and the objective test data corresponding to the evaluation dimensions; and obtaining a comprehensive score corresponding to the evaluation object according to the weight and the score corresponding to each evaluation dimension. According to the method provided by the invention, comprehensive, systematic and quantitative evaluation of the vehicle-mounted HMI touch interaction user experience can be realized.

Description

Technical Field

[0001] This invention relates to the field of smart cockpit technology, and in particular to an HMI touch interaction evaluation method, apparatus and electronic device. Background Technology

[0002] A smart cockpit is an upgrade of the car's driver's cabin into a comprehensive digital platform integrating infotainment, intelligent control, and human-machine interaction through a combination of hardware and software. Touch interaction based on a Human-Machine Interface (HMI) has become one of the most widely used and frequently employed human-machine interaction methods due to its advantages such as intuitiveness, high accessibility, and high customizability. Evaluating and researching the performance of touch interaction is of great significance for improving user experience, guiding interaction design, and assessing the usability of smart cockpit systems. Current evaluation schemes generally focus on the overall smart cockpit and lack a comprehensive evaluation scheme for touch interaction performance. Summary of the Invention

[0003] This invention provides an HMI touch interaction evaluation method, device, and electronic device, and proposes a comprehensive HMI touch interaction performance evaluation scheme to achieve a comprehensive, systematic, and quantitative evaluation of the user experience of in-vehicle HMI touch interaction.

[0004] This invention provides an HMI touch interaction evaluation method, which includes: determining multiple evaluation dimensions; the multiple evaluation dimensions include multiple of the following six dimensions: visual coordination of the interactive interface, rationality of operation hierarchy, human-computer convenience, comprehensibility of text and icons, functional completeness, and security; obtaining the weights corresponding to each evaluation dimension; determining the evaluation object; the evaluation object includes at least one of the following objects: settings, media, communication, navigation, and air conditioning; collecting test data; the test data includes subjective feedback data and objective test data; obtaining the score corresponding to each evaluation dimension based on the subjective feedback data and objective test data corresponding to each evaluation dimension; and obtaining a comprehensive score based on the weights and scores corresponding to each evaluation dimension.

[0005] Optionally, the weights corresponding to each evaluation dimension in the multiple evaluation dimensions can be obtained, including: for the five dimensions other than security among the six dimensions, the weights of each of the five dimensions can be obtained in the following way: obtain user weights and expert weights; construct a judgment matrix based on user weights and expert weights; use the analytic hierarchy process (AHP) to calculate the eigenvectors of the judgment matrix, and normalize the eigenvectors to obtain the weights of each of the five dimensions.

[0006] Optionally, after obtaining the initial weights of each of the five dimensions, the method further includes: calculating the largest eigenvalue of the judgment matrix; calculating the consistency index based on the largest eigenvalue; querying the average random consistency index; obtaining the consistency ratio based on the consistency index and the average random consistency index; and determining that the judgment matrix passes the consistency test if the consistency ratio is less than a preset threshold.

[0007] Optionally, test data can be collected, including: in a pre-configured static test environment, test data for at least one indicator of the test object from five dimensions: visual coordination of the interactive interface, rationality of the operation hierarchy, human-computer convenience, ease of understanding of text and icons, and functional completeness; in a pre-configured dynamic test environment and a pre-configured driving state, test data for at least one indicator of the test object from the safety dimension.

[0008] Optionally, subjective feedback data is obtained by subjectively testing at least one indicator of the evaluation object; objective test data is obtained by objectively testing at least one indicator of the evaluation object; subjective testing includes using scale assessment methods and / or semi-structured data collection to collect user subjective feedback data on at least one indicator of the evaluation object; subjective feedback data collected by scale assessment methods includes data corresponding to at least one of the following indicators: test task difficulty, functional understanding, visual clarity, feedback timeliness, user enjoyment, and personalized adaptability; subjective feedback data collected by semi-structured data collection includes data corresponding to at least one of the following indicators: Effectiveness, satisfaction, understandability, accessibility, controllability / feedback, personalization adaptability, and improvement suggestions; objective test data, including data corresponding to at least one of the following indicators: HMI system functions and hierarchy, application cold start and / or warm start latency, call operation response time, map operation response time, multimedia control response time, vehicle operation response time, Bluetooth operation response time, steering wheel control response time, interface scrolling smoothness, single and total eye-off time, number of eye-offs, eye movement hotspot distribution, single operation duration, number of single operation steps, task completion time, and total distance moved by finger operation.

[0009] Optionally, after collecting test data, the method further includes: calculating the mean and standard deviation of the subjective feedback data collected by the scale assessment method; performing qualitative coding and thematic summarization on the subjective feedback data collected by the semi-structured collection to obtain the first initial score corresponding to each indicator in the subjective feedback data; and normalizing the objective test data to obtain the second initial score corresponding to each indicator in the objective test data.

[0010] Optionally, based on the subjective feedback data and objective test data corresponding to each assessment dimension, the scores corresponding to each assessment dimension are obtained, including: calculating the scores corresponding to each assessment dimension based on the first initial score and / or the second initial score corresponding to at least one indicator corresponding to each assessment dimension.

[0011] Optionally, after collecting test data, the method may also include: using the Spearman rank correlation coefficient method to verify the consistency between subjective feedback data and objective test data.

[0012] This invention also provides an HMI touch interaction evaluation device, comprising: a first determining module for determining multiple evaluation dimensions; the multiple evaluation dimensions include multiple of the following six dimensions: visual coordination of the interactive interface, rationality of operation hierarchy, human-computer convenience, ease of understanding of text and icons, functional completeness, and security; an acquisition module for acquiring the weights corresponding to each evaluation dimension; a second determining module for determining the evaluation object; the evaluation object includes at least one of the following objects: settings, media, communication, navigation, and air conditioning; a third determining module for collecting test data; the test data includes subjective feedback data and objective test data; a single-dimensional scoring module for obtaining the score corresponding to each evaluation dimension based on the subjective feedback data and objective test data corresponding to each evaluation dimension; and a comprehensive scoring module for obtaining a comprehensive score based on the weights and scores corresponding to each evaluation dimension.

[0013] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement any of the HMI touch interaction evaluation methods described above.

[0014] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the HMI touch interaction evaluation methods described above.

[0015] The present invention also provides a computer program product, including a computer program, which, when executed by a processor, implements any of the HMI touch interaction evaluation methods described above.

[0016] The HMI touch interaction evaluation method, device, and electronic device provided by this invention evaluate HMI touch interaction across six dimensions: visual consistency of the interactive interface, rationality of the operation hierarchy, human-computer interaction convenience, ease of understanding of text and icons, functional completeness, and security. Furthermore, HMI touch interaction is decomposed into five evaluation objects: settings, media, communication, navigation, and air conditioning. After testing these objects, subjective feedback data and objective test data are obtained. Subjective feedback data reflects the user's actual experience when using the HMI touch interaction system under evaluation, while objective test data reflects the objective performance indicators of the HMI touch interaction system. Based on the test data, scores corresponding to each evaluation dimension are obtained. Then, based on the weights and scores corresponding to each evaluation dimension, a comprehensive score is obtained. Thus, through this comprehensive evaluation system of the above six core evaluation dimensions, the overall performance of the HMI touch interaction system can be comprehensively assessed, and the user experience can be characterized. This contributes to the development of HMI touch interaction systems that improve user experience and enhance HMI touch interaction performance. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in this invention or related technologies, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a flowchart illustrating the HMI touch interaction evaluation method provided in this embodiment of the invention.

[0019] Figure 2 This is a schematic diagram of the six dimensions of the HMI touch interaction evaluation method provided in this embodiment of the invention.

[0020] Figure 3 This is a schematic diagram illustrating the acquisition of user weights and expert weights to determine the weights of each dimension in some embodiments of the HMI touch interaction evaluation method provided by the present invention.

[0021] Figure 4 This is a flowchart illustrating the process of determining the weights of each dimension in some embodiments of the HMI touch interaction evaluation method provided by the present invention.

[0022] Figure 5 Example diagram of the judgment matrix in some embodiments of the HMI touch interaction evaluation method provided in this invention; Figure 6 Example diagrams of normalized judgment matrices in some embodiments of the HMI touch interaction evaluation method provided in this invention; Figures 7a to 7cThis is a schematic diagram of the test environment in some embodiments of the present invention; Figure 8 This is a schematic diagram of the module structure of the HMI touch interaction evaluation device provided in an embodiment of the present invention.

[0023] Figure 9 This is a schematic diagram of the structure of the electronic device provided in an embodiment of the present invention.

[0024] Figure label: 801: First determination module; 802: Acquisition module; 803: Second determination module; 804: Third determination module; 805: Single-dimensional scoring module; 806: Comprehensive scoring module; 910: Processor; 920: Communication interface; 930: Memory; 940: Communication bus. Detailed Implementation

[0025] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0026] With the rapid development of automotive intelligence and human-machine interaction technologies, the intelligent cockpit has become the core carrier of the overall vehicle experience. Traditional automotive instrument displays and mechanical button interactions are gradually evolving into integrated platforms that combine voice, touch, and gesture controls. Under this trend, touch interaction, due to its intuitiveness, high accessibility, and high customizability, has become one of the most widely used and frequently employed human-machine interaction methods. Therefore, research and evaluation of touch interaction performance are of great significance for improving user experience, guiding interaction design, and assessing the usability of intelligent cockpit systems.

[0027] Regarding standards and specifications, a series of standards related to in-vehicle human-machine interfaces (HMIs) have been published in relevant technologies. For example, the international standard ISO 21956:2019 sets forth functional safety design requirements for the HMI specifications of keyless ignition systems. Some regulations impose stringent testing and human factor performance requirements on driver drowsiness and attention alert systems. In related technologies, standards such as GB / T 41797-2022 "Performance Requirements and Test Methods for Driver Attention Monitoring Systems" and GB / T44176-2024 "Performance Requirements and Test Methods for Automotive Panoramic Imaging Monitoring Systems" have also been published, promoting the systematic development of intelligent cockpit HMI testing. However, these standards mainly target specific functional systems (such as attention monitoring and image display) and have not yet established a systematic evaluation framework for the touch interaction experience of human-machine interfaces (HMIs).

[0028] In the field of overall performance evaluation of intelligent cockpits, there are already some representative evaluation systems, such as the intelligent cockpit evaluation rules formulated by the China Intelligent Connected Automobile Protocol (C-ICAP) and the "Test and Evaluation Methods for the Intelligence Level of Automotive Intelligent Cockpits" published by the Society of Automotive Engineers (C-SAE). C-SAE stands for China Smart Cockpit Assessment & Evaluation, and the Intelligent Vehicle Integrated Systems Test Area (I-VISTA) is another example. These systems have provided preliminary assessments of the intelligence level and interaction capabilities of cockpits, playing a positive role in industry standardization. However, these systems still have shortcomings in the detailed evaluation of touch interaction experiences and in the comprehensive evaluation combining subjective and objective data. For example, the C-SAE system evaluates touch interaction as a secondary indicator; while C-ICAP emphasizes basic performance indicators, it lacks fine-grained experience indicators; and I-VISTA focuses on application startup time and smoothness, but its quantitative research on subjective experiences such as interface comprehensibility and visual harmony is relatively limited. The three main standards only match 25% to 26% of touch interaction evaluation items. Even in items with a high degree of matching, the evaluation methods and indicators differ, leading to complexity in the selection of evaluation standards and failing to form a comprehensive system for evaluating the user experience of touch interaction. Table 1 shows a comparison of touch interaction evaluation methods under various evaluation standards. Table 1

[0029] In recent years, the demand for testing touch interaction experiences has grown rapidly. For example, the "Safety Test and Evaluation Procedure for Touch Interaction in Intelligent Vehicle Cockpits" for the first time proposed specific test requirements for the visibility, operability, and safety of in-vehicle touch interfaces, and refined specific quantitative indicators such as the number of operation steps, time spent looking away from the road, and click response time. Major intelligent vehicle manufacturers have also generally introduced multi-dimensional data evaluation methods, such as eye-tracking analysis, touch hotspot distribution, and quantification of cognitive load, when developing next-generation intelligent cockpit systems to optimize interfaces and processes.

[0030] User experience is an important performance indicator for HMI touch interaction systems.

[0031] In terms of user experience evaluation, the evaluation methods and systems for touch interaction in in-vehicle human-machine interfaces (HMIs) lack a systematic and comprehensive evaluation framework specifically for touch interaction user experience. For example, current standards (such as ISO 21956:2019, (EU) 2021 / 1341, GB / T 41797-2022, GB / T 44176-2024) mainly focus on the performance and safety requirements of specific functional systems (such as driver attention monitoring, image display, keyless start, etc.), and have not established a systematic and multi-dimensional evaluation framework for the overall user experience of HMI touch interaction. This makes it difficult to comprehensively reflect the user's overall experience across multiple dimensions, including visual perception, operational logic, ease of use, information comprehension, functional satisfaction, and interaction safety, thus failing to accurately reflect the user experience.

[0032] Furthermore, mainstream smart cockpit evaluation systems have shortcomings in assessing touch interaction experience, with incomplete evaluation dimensions and indicators. While industry evaluation systems such as C-SAE, C-ICAP, and I-VISTA evaluate the intelligence and interaction capabilities of the cockpit, they often treat touch interaction as a secondary indicator. For example, the C-SAE system evaluates touch interaction as a secondary indicator. Although C-ICAP includes basic performance indicators such as pixel density and brightness adjustment, as well as application launch time and screen smoothness, it lacks fine-grained user experience indicators. While I-VISTA focuses on application launch time and smoothness, its quantitative research on subjective experiences such as interface understandability and visual harmony is limited.

[0033] Furthermore, the evaluation methods and modules of the three standards—C-SAE, C-ICAP, and I-VISTA—are inconsistent. The matching rate of touch interaction evaluation items among these three main standards is low (only 25%~26%). Even for similar evaluation items, their evaluation methods, test tasks (e.g., the evaluation method for navigation tasks), and evaluation indicators differ. For example, some basic performance indicators and HMI application startup time included in C-ICAP are not covered in C-SAE and I-VISTA; conversely, touch interaction display, click, swipe, and continuous interaction scenarios unique to I-VISTA are not present in C-SAE and C-ICAP. In terms of module composition, C-SAE mainly covers navigation, telephone, music, and air conditioning modules; C-ICAP includes air conditioning, radio, Bluetooth, and WIFI modules; and I-VISTA is more comprehensive, including navigation, telephone, music, radio, settings, contacts, and air conditioning modules. This inconsistency leads to poor horizontal comparability of evaluation results, making it difficult to reach a consensus in this technical field.

[0034] In addition, there is a problem of insufficient integration of subjective and objective data. Existing standards are clearly insufficient in refining the specific experience evaluation of touch interaction and in combining subjective and objective data for comprehensive evaluation. C-ICAP focuses more on subjective user experience and expert evaluation scores, while I-VISTA emphasizes the quantifiable acquisition of objective behavioral data. The two have different emphases in the balance between subjective and objective data, but the integration is not deep enough.

[0035] Furthermore, the representation and quantification of user experience in related technologies are insufficient, subjective evaluation methods dominate, and objective measurement is not applied enough. For example, a survey of literature worldwide from 2019 to 2023 found that about 67% of the studies mainly used subjective methods such as questionnaires and interviews, while objective measurements such as hand trajectories, click deviation, and gaze shift were systematically used in only 26% of the literature.

[0036] Furthermore, the evaluation methods in related technologies lack research on dynamic real-vehicle scenarios, and even less research covers high-frequency interactive behaviors in dynamic real-vehicle scenarios. This makes it difficult for the evaluation results to fully reflect the user experience and potential safety risks in real driving environments.

[0037] Furthermore, the evaluation models in related technologies suffer from imperfect objective evaluation models. Objective evaluation models for interactive behavior, driving interference, and attention load (such as human interference and hand-eye coordination efficiency) are still incomplete, and comprehensive quantitative evaluation methods are lacking.

[0038] The aforementioned shortcomings make it difficult for related technologies to comprehensively and deeply evaluate the overall impact of smart cockpit HMI touch interaction on user experience, which is detrimental to guiding the design optimization of HMI systems and substantial improvements in user experience. Therefore, there is an urgent need for an HMI touch interaction evaluation method that can integrate subjective and objective data, cover multi-dimensional experience elements, and adapt to various dynamic and static testing scenarios.

[0039] In view of this, to address the problems existing in related technologies, such as incomplete and unsystematic HMI touch interaction evaluation methods, insufficient combination of subjective and objective evaluations, lack of scientific basis for weight setting, and inadequate dynamic scenario assessment, this invention proposes an HMI touch interaction evaluation method aimed at improving user experience. This method achieves a comprehensive, systematic, and quantitative evaluation of the user experience of in-vehicle HMI touch interaction by constructing a multi-dimensional evaluation model, scientifically setting weights, and combining dynamic and static testing environments with subjective and objective evaluation data.

[0040] The following is combined with Figures 1-9 This invention describes an HMI touch interaction evaluation method, apparatus, and electronic device according to embodiments of the present invention.

[0041] Figure 1 This is one of the flowcharts of the HMI touch interaction evaluation method provided in the embodiments of the present invention. The HMI touch interaction evaluation method provided in the embodiments of the present invention may specifically include the following steps: Step 100: Determine multiple assessment dimensions.

[0042] The evaluation dimensions include multiple aspects from the following six dimensions: visual consistency of the interactive interface, rationality of the operation hierarchy, human-computer interaction convenience, comprehensibility of text and icons, functional completeness, and security. Each evaluation dimension may include at least one indicator; specifically, it may include at least one subjective indicator and at least one objective indicator.

[0043] In some embodiments, the multiple evaluation dimensions include six dimensions. That is, the present invention designs a comprehensive evaluation system containing six core evaluation dimensions, which aims to more comprehensively represent user experience. The six dimensions are as follows: Visual harmony of the interactive interface: Evaluate the visual harmony and aesthetics of interface elements (such as color, font, layout), as well as the contrast and clarity of the background and text, and the compatibility of interface colors with the in-vehicle environment, with the aim of improving the user's visual comfort and enjoyment.

[0044] Specifically, the dimension of visual harmony of the interactive interface can include one or more of the following indicators: interface color adaptation, element layout rationality, text / icon contrast, and visual style consistency.

[0045] Among them, the interface color adaptation, for example, when the background color of the central control screen is dark gray, the function icons adopt a white highlight design to match the nighttime environment inside the car and avoid strong light glare.

[0046] Text / icon contrast, for example, the contrast between text and background in the navigation interface should be ≥4.5:1 (meeting visual comfort standards) to ensure clear visibility even in strong light.

[0047] The layout of elements is reasonable. For example, icons for high-frequency functions such as air conditioning adjustment and volume control are concentrated in the lower half of the screen, which is within the range of hand operation while driving, and there is no visual redundancy.

[0048] Visual style consistency refers to whether the styles of various interface elements are consistent and whether the visual experience is harmonious.

[0049] Operation hierarchy rationality: Assess whether users can quickly find the function entry point and complete the operation, the rationality of the function entry point and button hierarchy, the simplicity of the function operation process, and the degree of conformity between the function classification method and the user's thinking and operation habits, focusing on the logic of the task operation process and cognitive load.

[0050] Specifically, the dimension of operational hierarchy rationality can include one or more of the following indicators: depth of function entry, number of steps in the operation process, fit of classification logic, and clarity of return path.

[0051] Function access depth, for example, adjusting the air conditioner temperature has ≤2 operation levels (homepage → air conditioner panel for direct adjustment), without the need for multiple jumps.

[0052] The number of operation steps, such as connecting via Bluetooth, is ≤3 steps (Settings → Bluetooth → Select Device), which conforms to the user's usual operation logic.

[0053] Categorization logic, such as grouping "Bluetooth connection" and "WiFi settings" into the "Network and Connections" module, rather than scattering them across different menus, aligns with user cognitive habits.

[0054] Human-machine interface convenience: Assess whether users can operate the vehicle safely, comfortably, and efficiently; the appropriateness of icon and text size; the rationality of the placement of frequently used operation buttons and content display; whether there is obstruction of vision or hand operation interference; and whether it conforms to ergonomics. Focus on the physical operation convenience and smoothness of the user interface.

[0055] Specifically, the dimension of human-computer convenience can include one or more of the following indicators: accessibility of the operating area, icon / text size adaptation, ergonomic adaptation, and non-interference operation.

[0056] Accessibility, for example, the steering wheel control buttons cover high-frequency functions (answering calls, changing songs), which can be touched without looking down while driving.

[0057] Size adaptation, for example, the size of core function icons is ≥10mm×10mm, and the text font and size meet the preset requirements, such as whether the font size is greater than or equal to the preset standard. For example, for most users, the font size that can be clearly distinguished is generally 14 or above, so the font size is judged to be ≥14 to meet the needs of quick recognition while driving.

[0058] There is no interference, for example, the functional area at the edge of the touch screen does not overlap with the steering wheel's rotation range, avoiding interference between the hands and the steering wheel while driving.

[0059] Text and icon comprehensibility: Assess whether users can intuitively and quickly understand relevant information, the consistency between icons and user cognition, the accuracy of text descriptions, text length and expression, and whether there is a clear explanation of innovative functions. The text and icon design should be concise and clear to reduce operational misunderstandings.

[0060] Specifically, the dimension of text and icon comprehensibility can include one or more of the following indicators: icon semantic matching degree, text description conciseness, clarity of guidance for innovative functions, and language accessibility.

[0061] Icon semantics, such as using a "speaker" icon to represent volume adjustment and a "map pointer" to represent navigation, are self-explanatory without additional textual explanation.

[0062] In textual descriptions, such as when switching air conditioning modes, use "cooling" and "heating" instead of technical terms like "cooling cycle" and "heat pump mode" to reduce the learning curve.

[0063] Innovative feature guidance, such as when using the "voice wake-up + touch confirmation" combination for the first time, a short prompt appears saying "Click the icon to complete the confirmation", without complicated explanatory text.

[0064] Functional completeness: Assess the product's ability to meet user needs, whether common basic functions are missing, and whether functional details meet usage requirements, including the breadth, diversity, and usability of functions.

[0065] Specifically, the dimension of functional completeness can include one or more of the following indicators: basic function coverage, scenario-based function adaptation, custom function support, and exception handling capability.

[0066] Basic functions are covered, such as the navigation module supporting core functions like "route planning, real-time traffic conditions, and avoiding congestion," with no key functions missing.

[0067] Context-specific adaptation, for example, the media module supports playback of "local music, online radio, and mobile screen mirroring" to meet users' different audio source needs.

[0068] For handling anomalies, such as Bluetooth connection failure, provide troubleshooting options like "Check device battery" and "Research for devices" instead of simply displaying "Connection failed".

[0069] Safety: This assesses the distraction and driving safety risks caused by driver touch operations while driving. It uses eye-tracking data to obtain indicators such as time the driver's gaze leaves the road, duration of each operation, eye movement distribution range (e.g., heatmap), operation duration, and the time and frequency of glancing at the central control screen. The safety dimension is the foundation of the six dimensions and includes at least one objective indicator. The objective test data corresponding to these indicators can be collected in a dynamic testing environment.

[0070] Specifically, the safety dimension can include one or more of the following indicators: time off-road, duration of a single operation, attentional load during operation, and emergency scenario response design.

[0071] The time the driver's gaze is off the road, such as the time spent adjusting the airflow during a single instance, should be ≤1.5 seconds (meeting the driving safety threshold). 1.5 seconds is just an example; other thresholds may also be used.

[0072] The duration of a single operation, such as the touch operation time for answering an incoming call, should be ≤0.5 seconds to avoid prolonged distraction from driving. 0.5 seconds is just an example; other thresholds may also be used.

[0073] In emergency scenarios, such as accidentally pressing the "cancel navigation" button while driving, a secondary confirmation pop-up window will appear ("Cancel current navigation?") to prevent accidental operation from causing safety risks.

[0074] The dimensions, time, and other values ​​mentioned above are merely examples and not intended to limit the specific indicators. Other values ​​can be used depending on the actual application scenario. Furthermore, the above descriptions of the indicators are illustrative and not necessary limitations on the corresponding indicators. Those skilled in the art can adapt the above illustrative descriptions to derive other examples for each type of indicator.

[0075] Step 101: Obtain the weights corresponding to each evaluation dimension in the multiple evaluation dimensions.

[0076] To ensure the accuracy and objectivity of the evaluation results, this embodiment of the invention employs the Analytic Hierarchy Process (AHP) combined with user survey data to assign weights to the five dimensions other than security. The security dimension, as a fundamental safeguard, is not included in the relative weight calculation.

[0077] In some embodiments, the weights of the five dimensions can be determined as follows: First, construct the judgment matrix: Obtain user weights and expert weights. Specifically, obtaining user weights involves acquiring data from a certain number of users who have rated each of the five dimensions, thus creating a user weight matrix. For example, you could obtain scores from 50-100 users with driving experience, assigning them scores on the importance or relative importance of the five dimensions. For instance, you could assign a 7-point importance score to each evaluation dimension, with scores from 1 to 7 representing different levels of importance, thus forming a user weight matrix.

[0078] To obtain expert weights, you can obtain data on a certain number of experts scoring each of the five dimensions to create an expert weight matrix. For example, you can obtain scores from multiple experts (e.g., 12-30 experts, ultimately using 10-20 valid questionnaires) with backgrounds in human factors engineering and interaction design to compare each dimension pairwise. You can use multiple scales such as 1 / 5, 1 / 3, 1, 3, 5 to represent the relative importance of each pair of different dimensions.

[0079] Weighted fusion of user weight matrix and expert weight matrix: Set the ratio of expert weight to user weight (e.g., 1:2), use weighted average method to fuse user weight matrix and expert weight matrix to generate judgment matrix.

[0080] After obtaining the judgment matrix, the relative weights of each dimension are obtained based on the judgment matrix using the AHP algorithm. For example, the eigenvectors of the judgment matrix are calculated using the AHP algorithm, and then normalized to obtain the weights of each dimension.

[0081] Then, to ensure the reliability of the weights, a consistency check is performed on the judgment matrix: the largest eigenvalue corresponding to the judgment matrix is ​​calculated, and based on the largest eigenvalue, a consistency index is calculated, and the average random consistency index is queried; based on the consistency index and the average random consistency index, the consistency ratio is obtained; in response to the consistency ratio being less than a preset threshold, the judgment matrix is ​​determined to have passed the consistency check. For example, if the consistency ratio (CR) is less than a preset threshold (such as 0.1 or 0.12), the consistency check is confirmed to have passed, thus ensuring the validity of the calculated weight results.

[0082] For example, a set of weights obtained through this method could be: human-computer convenience 28.42%, operational hierarchy rationality 23.33%, text and icon comprehensibility 17.98%, interactive interface visual harmony 17.57%, and functional completeness 12.71%.

[0083] The process of determining the weights of each dimension based on AHP is as follows: Figure 3 As shown.

[0084] Step 102: Determine the assessment subjects.

[0085] The evaluation targets include at least one of the following: settings, media, communication, navigation, and air conditioning. In other words, the HMI touch interaction within the smart cockpit is divided into five major functional areas as five evaluation targets. Settings can include homepage settings and / or vehicle settings. Settings, media, communication, navigation, and air conditioning respectively represent the HMI touch interaction functions related to that item.

[0086] Step 103: Collect test data.

[0087] The test data includes subjective feedback data and objective test data. In other words, the embodiments of the present invention perform a test task that combines subjective and objective methods. Subjective feedback data is obtained through subjective testing, and objective test data is obtained through objective testing. Optionally, in some embodiments, a test environment combining dynamic and static methods can also be used to perform the test task that combines subjective and objective methods and obtain test data.

[0088] Optionally, in some embodiments, methods such as Spearman's rank correlation coefficient can be used to test the consistency between subjective feedback data and objective test data. For example, p < 0.05 indicates significant correlation, which can verify the effectiveness of the evaluation system.

[0089] Step 104: Based on the subjective feedback data and objective test data corresponding to each assessment dimension, obtain the scores corresponding to each assessment dimension.

[0090] The subjective feedback data and objective test data are processed separately to obtain normalized scores for the subjective and objective indicators. Then, a weighted composite score model is constructed for the subjective indicators included in the subjective feedback data and the objective indicators included in the objective test data. For example, the composite score for the objective indicators can be calculated using the formula S = ∑(wi × si), where wi is the weight of the i-th indicator and si is its normalized score. Based on the weighted composite score model, the composite score corresponding to the objective test data and the composite score of the subjective feedback data can be obtained. The composite scores of the two types of data in one dimension are then weighted and fused to obtain the composite score for that dimension. Alternatively, the same model can be used to weight and fuse the various indicators included in the objective test data and the various indicators included in the subjective feedback data to obtain a composite score for that dimension.

[0091] Step 105: Obtain the comprehensive score based on the weight and score corresponding to each evaluation dimension.

[0092] The overall score is obtained by summing the weights and scores of each assessment dimension. Alternatively, the overall score of an assessment object can be obtained by summing the overall scores of multiple assessment objects.

[0093] Specifically, based on the weights determined in step 101 (i.e., the weights of each dimension determined by the AHP algorithm) and the comprehensive scores of each dimension obtained in step 104, the comprehensive score corresponding to each evaluation object is calculated. Then, the comprehensive scores of the five evaluation objects are summed, for example, by weighted summation, to obtain the overall comprehensive score, which is the final total evaluation score for HMI touch interaction.

[0094] Figure 1 The illustrated embodiment employs a comprehensive six-dimensional evaluation framework, systematically proposing for the first time a six-dimensional comprehensive evaluation system covering visual coordination, operational rationality, human-computer interaction, information readability, functional integrity, and security. This overcomes the shortcomings of existing methods that suffer from single dimensions or incomplete coverage. The method also employs a scientific weighting mechanism, using the Analytic Hierarchy Process (AHP) to integrate expert experience with real user survey data (weighted according to a specific ratio) to determine the weights of each evaluation dimension. This makes the weighting settings more scientific and representative, improving the objectivity and fairness of the evaluation results. Furthermore, this embodiment of the invention deeply integrates and quantifies subjective and objective data. It not only simultaneously collects subjective feedback data and objective test data (such as eye movement, response time, and operational data), but more importantly, through standardized data processing and weighting models, it effectively integrates the two for comprehensive evaluation, solving the problems of previous separation or simple superposition of subjective and objective evaluations.

[0095] based on Figure 1 Based on the embodiments shown, in some embodiments, such as Figure 4 As shown, the weight of each of the five dimensions other than security can be determined as follows: Step 401: Obtain user weights and expert weights.

[0096] For example, the user weight and expert weight can be collected using Table 2 as shown below.

[0097] Table 2 Very unimportant unimportant Equally important important Very important Compared to the visual importance of the interactive interface, how important is the understandability of text and icons? How important is human-computer interaction compared to the visual importance of the user interface? Compared to the visual importance of the user interface, how important is the logical structure of the operation hierarchy? Is the completeness of functionality more important than the visual importance of the user interface? Compared to the ease of understanding of text and icons, how important is human-computer interaction? Compared to the ease of understanding of text and icons, how important is the logical structure of operational levels? Compared to the ease of understanding of text and icons, what is the importance of functional completeness? Compared to human-machine convenience, how important is the rationality of the operational hierarchy? Compared to human-computer interaction, how important is functional integrity? Compared to the rationality of operational hierarchy, what is the importance of functional completeness? For example, data from 50 users with driving experience who rated the table above can be obtained. A 7-point importance scale can be used, with 1-7 points representing different levels of importance, to form a user weight matrix.

[0098] Data on scores given by 20 experts for the above table were obtained to form an expert weight matrix. Step 402: Construct a judgment matrix based on user weights and expert weights.

[0099] The ratio of expert weights to user weights is set to 1:2. A weighted fusion is then performed, and a weighted average method is used to merge the user weight matrix and the expert weight matrix to generate the final judgment matrix used for AHP (Advanced Hierarchy Processing). For example, the generated judgment matrix is ​​as follows: Figure 5 As shown in the diagram. D1-D5 represent five dimensions. The first row of the judgment matrix indicates that D1 is 3 times more important than D2 (slightly more important), D1 is 5 times more important than D3 (significantly more important), and so on. The diagonal lines are all 1s, indicating that each dimension is equally important compared to itself. D2's importance relative to D1 is 1 / 3.

[0100] Step 403: Calculate the eigenvectors of the judgment matrix using the analytic hierarchy process (AHP), and normalize the eigenvectors to obtain the weights of each of the five dimensions.

[0101] Next, the judgment matrix is ​​normalized column-wise: the elements of each column are summed, and then each element of that column is divided by the column sum. For example, the column sums are calculated as follows: Column 1 sum = 1 + 0.333 + 0.2 + 0.5 + 0.25 = 2.283; Column 2 sum = 3 + 1 + 0.333 + 2 + 0.5 = 6.833; Column 3 sum = 5 + 3 + 1 + 3 + 2 = 14; Column 4 sum = 2 + 0.5 + 0.333 + 1 + 0.333 = 4.166; Column 5 sum = 4 + 2 + 0.5 + 3 + 1 = 10.5. The normalized matrix (each element = original element / column sum of its column) is as follows. Figure 6 As shown.

[0102] Then, the average of the rows of the normalized matrix (i.e., the weights W) is calculated by summing the values ​​of each row of the normalized matrix and then dividing by the number of dimensions (e.g., 5), resulting in: W1 (weight of D1) = 0.419, W2 (weight of D2) = 0.163. W3 (weight of D3) = 0.067, W4 (weight of D4) = 0.250. W5 (weight of D5) = 0.100.

[0103] Thus, we can obtain the initial weight vector W = [0.419, 0.163, 0.067, 0.250, 0.100] for the five dimensions.

[0104] Next, a consistency check will be performed.

[0105] Step 404: Calculate the largest eigenvalue of the judgment matrix.

[0106] For example, first calculate the weighted sum vector: multiply the original judgment matrix by the weight vector W, then divide each element of the weighted sum vector by the corresponding weight element W, and then calculate the average of these quotients to obtain the maximum eigenvalue λ_max.

[0107] Step 405: Calculate the consistency index based on the largest eigenvalue.

[0108] For example, the consistency index CI is calculated using the following formula: CI = (λ_max - n) / (n - 1). Here, n is the matrix order (number of dimensions), which is 5 in this case.

[0109] Step 406: Query the average random consistency index.

[0110] To find the average random consistency index (RI), for example, when n=5, the table shows that RI = 1.12.

[0111] Step 407: Obtain the consistency ratio based on the consistency index and the average random consistency index.

[0112] For example, the consistency ratio CR can be calculated using the formula: CR = CI / RI. For instance, assuming CI = 0.021 and RI = 1.12, then CR = 0.021 / 1.12 ≈ 0.0188.

[0113] Step 408: In response to the consistency ratio being less than a preset threshold, determine that the judgment matrix has passed the consistency test.

[0114] For example, assuming a preset threshold of 0.1, if CR < 0.1, then the consistency of the judgment matrix is ​​considered acceptable, and the calculated weights are valid. CR ≈ 0.0188 < 0.1, therefore the consistency test passes.

[0115] It should be noted that, Figure 4 The numerical values ​​in the illustrated embodiments are merely randomly selected exemplary numbers, used only to describe the process of obtaining the weights of each dimension using the AHP algorithm, and are not intended to limit the specific values ​​of the weights in actual applications.

[0116] For example, in practical application scenarios, the weights shown in Table 3 below can be used: Table 3 Sort Dimension Weight 1 Human-machine convenience 28.42% 2 Operational hierarchy rationality 23.33% 3 Text and icon readability 17.98% 4 Visual harmony of the interactive interface 17.57% 5 Functional completeness 12.71% In some embodiments, the acquisition of test data in step 103 can be implemented in the following ways: A combined dynamic and static testing environment is used to conduct subjective and objective testing tasks.

[0117] For example, combining Figure 7a , Figure 7b and Figure 7c As shown, the static testing environment was first configured in a simulated cockpit system. The specifications for the central control screen (e.g., 15.6 inches, 1920x1080 resolution), lighting conditions (e.g., 400-450 lux), and the subject's posture were defined. An eye tracker was used to record data. In this environment, visual coordination, operational hierarchy rationality, human-machine interface convenience, text and icon comprehensibility, and functional completeness were evaluated.

[0118] Dynamic real-vehicle testing environment and driving status settings: Testing is conducted in a closed test track (e.g., a 1.5 km loop), specifying road type, traffic environment, test time and weather, driving speed (e.g., constant speed of 30-50 km / h), acceleration variation range, and steering operation range. A safety officer is present to ensure safety. Vehicle driving status data is recorded in real time via OBD (On-Board Diagnostics Interface) and Inertial Measurement Unit (IMU). This environment focuses on evaluating safety dimensions and allows for dynamic verification of other dimensions.

[0119] Next, based on the above testing environment, testing tasks will be conducted on various indicators of the evaluation objects, including subjective testing and objective testing: Subjective testing: For example, a rating scale can be used, such as a Likert scale of 5 or 7, to test indicators such as task difficulty, functional understanding, visual clarity, timely feedback, user enjoyment, and personalized adaptability. A semi-structured data collection method can be employed, for example, based on the seven user experience principles of ISO 9241-210, covering indicators such as effectiveness, satisfaction, understandability, accessibility, controllability / feedback, personalized adaptability, and improvement suggestions. Subjective feedback from users on the operation of each module can be collected to obtain subjective feedback data. Taking the air conditioning module (the evaluation object) as an example, specific test tasks such as temperature adjustment in each temperature zone and mode adjustment can be designed, and the preconditions and operating steps can be clearly defined.

[0120] Objective testing: Data was collected using professional equipment, such as the Tobii Pro Fusion head-mounted eye tracker with a sampling rate of 120Hz; the Basler acA1300-60gc high-frame-rate industrial camera with a frame rate of 60fps; and the Wacom Pro Pen 2 series smart stylus.

[0121] For example, objective data includes, but is not limited to, the following indicators: HMI system functions and levels: number of functions, average level.

[0122] Average hierarchy refers to the average number of navigation steps required for all core functions in an HMI system to navigate from the "system homepage" to the "function operation page". The more navigation steps there are, the deeper the hierarchy is, and the higher the operational cost for users to access functions; conversely, the shallower the hierarchy is, the more convenient the operation is.

[0123] The average level value can be directly related to the "reasonableness of operation level" evaluation dimensions: If the average level is ≤2 levels, it means that the function entry is shallow and the operation level is reasonable; if the average level is >2 levels (such as some functions requiring 3-4 jumps), there may be redundancy in the level, and the function classification logic needs to be optimized (such as adding high-frequency functions to the homepage quick entry) to reduce the user operation cost.

[0124] Calculate the average number of levels using the formula: Average level = (L1 + L2 + … + L…) n ) / N.

[0125] Taking a certain car model's HMI system as an example, assuming 10 core functions are selected, the statistics for each function level are as follows: Serial Number Core Functions (corresponding to the assessment targets) Number of redirects from the homepage to the action page (level L) 1 Air conditioning temperature control (air conditioning) 1 (Homepage → Air Conditioner Panel) 2 Navigation destination search (navigation) 2 (Homepage → Navigation Module → Search Page) 3 Bluetooth connection (communication) 2 (Home → Settings → Bluetooth page) 4 Media song cut (media) 1 (Home → Media Panel) 5 Vehicle brightness adjustment (settings) 2 (Home → Settings → Display Settings) 6 Navigation route switching (navigation) 2 (Homepage → Navigation Module → Route Options) 7 View call logs (communications) 1 (Homepage → Communication Module → Call Log) 8 Media volume adjustment (Media) 1 (Home → Media Panel) 9 Air conditioning mode switching (air conditioning) 1 (Homepage → Air Conditioner Panel) 10 Vehicle WiFi Settings 3 (Home → Settings → Network Settings → WiFi Page) Calculate the average number of levels using the formula: Average number of levels = (1 + 2 + 2 + 1 + 2 + 2 + 1 + 1 + 1 + 3) / 10 = 1.6 This means that the average level of the core functions of the HMI system in this model is 1.6, and the overall operation depth is relatively shallow, so users can access most functions without having to jump around multiple times.

[0126] The average level value is directly related to the "reasonableness of the operation level" evaluation dimensions: If the average number of levels is ≤2, it indicates that the function entry is shallow and the operation level is reasonable. If the average number of levels is greater than 2 (e.g., some functions require 3-4 jumps), there may be redundancy in the number of levels. The function classification logic needs to be optimized (e.g., adding high-frequency functions to the homepage shortcut entry) to reduce the user's operation cost.

[0127] Functionality count refers to the number of various functions that the HMI system supports.

[0128] Response smoothness: This includes application cold / warm start latency, call operation response time, map operation response time, multimedia control response time, vehicle operation response time, Bluetooth operation response time, steering wheel control response time, and interface scrolling smoothness. For example, interface scrolling smoothness can be evaluated using two metrics: average frame rate and number of stuttering frames.

[0129] Safety-related factors: single and total eye movement time off-path, number of eye movement off-path events, distribution of eye movement thermal zones, duration of a single operation, and number of operation steps.

[0130] The number of operation steps refers to the number of independent touch actions a user needs to perform to complete a specific touch task (such as adjusting the air conditioning temperature, connecting Bluetooth, or searching for a navigation destination) starting from the "task start state" (such as the HMI home page or the current function interface). Each operation that can independently trigger a system response, such as "click, swipe, confirm", is counted as 1 step.

[0131] For example, adjusting the air conditioner temperature from 24℃ to 26℃ requires the following steps: Step (1): Click the "Air Conditioner" icon on the HMI home page (jump to the air conditioner panel, count as 1 step). Step (2): Click the "Temperature+" button twice on the air conditioner panel (clicking consecutively is considered one step, since the goal is to "adjust to 26℃", and the overall action serves the same goal); Total number of operation steps: 2 steps.

[0132] Other metrics include: task completion time and total distance moved by the fingers.

[0133] Next, we will process and comprehensively evaluate the subjective and objective test data (including subjective feedback data and objective test data).

[0134] Subjective feedback data processing: For example, calculating the mean and standard deviation of data obtained from scale assessment; performing qualitative coding and thematic summarization on semi-structured data collection, such as using NVivo to perform three-level coding of data and using Cohen's Kappa test for consistency.

[0135] For example, using NVivo for three-level encoding can include open encoding, axisymmetric encoding, and selective encoding.

[0136] Theme summarization refers to the systematic encoding, classification, and extraction of user's scattered and disordered subjective feedback text (such as "I can't find the back button when operating", "The icon is too small to see", "It is too troublesome to change the navigation route") to form a theme with clear meaning that can cover most feedback content. This realizes the transformation from scattered text to structured information and provides a basis for subsequent calculation of the first initial score.

[0137] Verifying consistency is essential to ensuring the objectivity and reliability of topic summarization results. The relationship between topic summarization and consistency verification is that of "operational process" and "quality verification." Topic summarization relies on the coder's subjective judgment (e.g., "Should a piece of feedback be classified as 'redundant operational steps' or 'deep functional entry point'?"). If completed by only one coder, personal bias or misjudgment may exist. Consistency verification verifies whether multiple coders' coding / topic classification results for the same batch of feedback texts are consistent: high consistency indicates that the topic summarization results are not influenced by individual subjectivity and are objective; low consistency requires recalibrating the coding rules (e.g., clarifying the definition of "redundant operational steps") to avoid result bias.

[0138] Objective test data processing: The collected objective data is normalized, for example, using the Min-MaxNormalization algorithm. For negative indicators, such as off-path time and number of operation steps, reverse normalization is required to ensure consistency in the scoring direction. In this way, through normalization, the scores corresponding to each indicator in the objective test data are obtained.

[0139] The normalization of objective test data (such as response time and number of operation steps) is a key step in transforming raw data into standardized values ​​that can be compared horizontally. The normalized values ​​need to be calibrated in conjunction with the indicator type (positive / negative) and the indicator weight allocation in two core operations to obtain the final score of each objective indicator.

[0140] The core purpose of normalization is to eliminate the influence of "differences in units" between different objective indicators, rather than directly generating scores. For example, the original data unit for indicator A (application cold start latency) is "seconds" (e.g., 2 seconds, 5 seconds); the original data unit for indicator B (interface scrolling smoothness) is "frame rate" (e.g., 58 frames, 30 frames). If the original data are directly compared or added, the units of "seconds" and "frame rate" cannot be unified, and the result is meaningless. Through normalization, the original data of all indicators can be mapped to the [0,1] interval, achieving unit unification.

[0141] After normalization, the indicator type needs to be determined first. Negative indicators should be reverse-calibrated to ensure that the normalized values ​​of all indicators are in the same direction as the scoring direction (i.e., the larger the value, the higher the score). Positive metrics (such as frame rate and task completion rate): Normalized values ​​do not need to be adjusted; proceed directly to the next step (e.g., normalized value 0.8 → remains 0.8 after calibration). Negative indicators (such as the number of operation steps and the time the line of sight is off the road): need to be calibrated in reverse using "1 - normalized value" (e.g., if the normalized value of the number of operation steps is 0.3, the calibrated value will be 1-0.3=0.7, ensuring that the fewer the steps, the higher the calibrated value).

[0142] Based on the above exemplary steps, the scores of the objective test indicators are obtained. Then, these scores are combined with the scores of the indicators corresponding to the subjective feedback data to obtain the comprehensive score.

[0143] Next, a comprehensive evaluation model is constructed: Weighted comprehensive scoring models are built for the scores of each indicator in the subjective feedback data and objective test data, either separately or in combination. The comprehensive score of the indicators in the objective test data can be calculated using the formula S=∑(wi×si), where wi is the weight of the i-th indicator, and si is its normalized score. If weighted comprehensive scoring models are constructed separately for the subjective feedback data and the objective test data, then the comprehensive scores of the subjective feedback data and the objective test data need to be weighted and fused to obtain a comprehensive score for an evaluation object in one dimension.

[0144] If the same weighted composite score model is built for both subjective feedback data and objective test data, then the model is configured to perform a weighted fusion operation on the composite scores of subjective feedback data and objective test data, and the model directly outputs the composite score of an evaluation object in one dimension.

[0145] Then, based on the dimension weights determined by AHP, the scores of an evaluation object in the above five dimensions are obtained. These scores are then added to the comprehensive score of the security dimension to obtain the comprehensive score of the evaluation object. The comprehensive scores of multiple evaluation objects are summed, for example, by weighted summation, to obtain the overall comprehensive score.

[0146] It should be noted that in the above embodiments, correlation analysis is performed on the collected subjective feedback data and objective test data. For example, Spearman's Rank Correlation Coefficient is used to test the consistency between subjective evaluation results and objective measurement data. A p < 0.05 is considered a significant correlation, verifying the effectiveness of the evaluation system. p represents the value of the Spearman Rank Correlation Coefficient.

[0147] Based on the above exemplary description, it can be seen that in some embodiments, the collection of test data may be carried out in a pre-configured static test environment, collecting test data of at least one indicator of the evaluation object from five dimensions: visual coordination of the interactive interface, rationality of the operation hierarchy, human-computer convenience, ease of understanding of text and icons, and functional completeness; and / or, in a pre-configured dynamic test environment and a pre-configured driving state, collecting test data of at least one indicator corresponding to the evaluation object from the safety dimension.

[0148] The test data can include subjective feedback data and objective test data. Subjective feedback data is obtained by subjectively testing at least one indicator of the test subject, while objective test data is obtained by objectively testing at least one indicator of the test subject.

[0149] Based on the above exemplary description, subjective testing can employ scale assessment and / or semi-structured data collection to gather subjective feedback data from users on at least one indicator of the assessment object. Specifically, the subjective feedback data collected through scale assessment includes data corresponding to at least one of the following indicators: test task difficulty, functional understanding, visual clarity, timely feedback, user enjoyment, and personalized adaptability. The subjective feedback data collected through semi-structured data collection includes data corresponding to at least one of the following indicators: effectiveness, satisfaction, comprehensibility, accessibility, controllability / feedback, personalized adaptability, and improvement suggestions.

[0150] Objective test data includes data corresponding to at least one of the following indicators: HMI system function and hierarchy, application cold start and / or warm start latency, call operation response time, map operation response time, multimedia control response time, vehicle operation response time, Bluetooth operation response time, steering wheel control response time, interface scrolling smoothness, single and total eye-off time, number of eye-offs, eye movement hotspot distribution, single operation duration, number of single operation steps, task completion time, and total distance moved by finger operation.

[0151] Optionally, to ensure the validity of the data, after collecting the test data, methods such as Spearman's rank correlation coefficient can be used to verify the consistency between subjective feedback data and objective test data, thereby improving the reliability of the data.

[0152] After collecting test data, the mean and standard deviation of the subjective feedback data collected by the scale assessment method can be calculated, and the subjective feedback data collected by the semi-structured collection can be qualitatively coded and thematically summarized to obtain the first initial score corresponding to each indicator in the subjective feedback data. Then, the objective test data can be normalized to obtain the second initial score corresponding to each indicator in the objective test data.

[0153] Next, based on the first initial score and / or second initial score corresponding to at least one indicator for each assessment dimension, the score corresponding to each assessment dimension is calculated, which is to obtain the score of an assessment object in one assessment dimension.

[0154] In summary, the HMI touch interaction evaluation method proposed in this invention can solve at least one of the following technical problems: How can we construct a systematic evaluation method that can comprehensively and multidimensionally assess the user experience of HMI touch interaction? Currently, there is a lack of a comprehensive evaluation system that can simultaneously cover visual consistency, operational rationality, human-computer interaction convenience, information readability, functional completeness, and security, in order to fully reflect the overall impact of touch interaction systems on user experience.

[0155] How can we effectively integrate subjective user evaluations with objective interaction performance data, and scientifically set the weights of each evaluation dimension to improve the objectivity and accuracy of the evaluation results? Current technologies often place an emphasis on either subjective evaluation or objective data collection, lacking a method to effectively combine the two and quantify the importance of each indicator.

[0156] How can we fully consider the specificities of dynamic driving scenarios during the evaluation process and quantitatively assess the impact of touch interaction on driver attention allocation and driving safety? Related technologies lack objective evaluation models for interactive behavior, driving interference, and attention load in dynamic real-world driving scenarios, especially lacking comprehensive quantitative assessments of human-caused interference and hand-eye coordination efficiency in real dynamic driving scenarios.

[0157] How can we provide a relatively unified and standardized evaluation process and indicator system to enhance the comparability of HMI touch interaction user experiences across different vehicle models or systems? Different evaluation systems in related technologies (such as C-SAE, C-ICAP, and I-VISTA) differ in evaluation items, methods, and indicators, making it difficult to compare evaluation results horizontally.

[0158] To address the shortcomings of the aforementioned related technologies, namely the lack of systematic and comprehensive evaluation methods for touch interaction of in-vehicle human-machine interfaces (HMIs) in assessing user experience, insufficient integration of subjective and objective data, inadequate consideration of user experience and safety in dynamic driving scenarios, and lack of uniformity and comparability among various evaluation systems, this invention proposes a more scientific, comprehensive, and effective evaluation method to guide the design and optimization of touch interaction in intelligent cockpit HMIs, thereby improving user experience.

[0159] Specifically, the embodiments of the present invention achieve the following technical effects in the following aspects: A comprehensive six-dimensional evaluation framework: For the first time, a comprehensive evaluation system covering six dimensions—visual coordination, operational rationality, human-computer convenience, information readability, functional integrity, and security—is proposed, overcoming the shortcomings of related methods that are either too singular or incomplete in coverage.

[0160] Scientific weighting mechanism: The AHP method is adopted, and the weights of each evaluation dimension are determined by combining expert experience and real user survey data (weighted by a specific ratio), which makes the weight setting more scientific and representative and improves the objectivity and fairness of the evaluation results.

[0161] Deep integration and quantification of subjective and objective data: Not only does it collect subjective evaluations (scales, interviews) and objective performance data (eye movement, response time, operational data, etc.) at the same time, but more importantly, it effectively integrates the two for comprehensive evaluation through standardized data processing and weighted models, solving the problem of separation or simple superposition of subjective and objective evaluations in the past.

[0162] Enhanced dynamic and safety assessment: Special emphasis is placed on testing in dynamic real-vehicle environments, and objective indicators such as eye-off time and eye-tracking hot zone are introduced to systematically evaluate the potential impact of touch interaction on driver distraction and safety, making up for the shortcomings of related solutions in dynamic scenarios and safety quantification.

[0163] Systematization and operability: It provides a complete and operable assessment process and methodology, from assessment dimension construction, weight setting, module selection, task design, data collection and processing to comprehensive evaluation, which helps to improve the standardization level of assessment and the comparability of results.

[0164] By implementing the technical solutions of this invention, the overall impact of HMI touch interaction systems on user experience can be reflected more accurately and comprehensively. This effectively identifies design flaws and user pain points, providing a scientific basis and specific guidance for optimizing intelligent cockpit interaction design and evaluating human factors engineering, ultimately achieving the goal of improving user experience and driving safety. For example, comparative tests on a specified vehicle model before and after improvements show that scores across all evaluation dimensions have increased, demonstrating the effectiveness of this evaluation method in promoting user experience optimization.

[0165] The HMI touch interaction evaluation method for improving user experience proposed in this invention can directly bring the following beneficial effects: A comprehensive and systematic evaluation of user experience: By constructing a six-dimensional comprehensive evaluation system that includes visual consistency of the interactive interface, rationality of operation hierarchy, human-computer convenience, ease of understanding of text and icons, functional completeness, and security, this method can overcome the shortcomings of existing technologies that have single or incomplete evaluation dimensions, and conduct a more comprehensive and systematic evaluation of the user experience of HMI touch interaction, thereby more accurately reflecting the user's overall perception of the interactive system.

[0166] Enhancing the objectivity and scientific rigor of evaluation results: This method employs the Analytic Hierarchy Process (AHP) and innovatively integrates expert experience with real user survey data to determine the weights of each evaluation dimension. It also combines subjective user evaluations with objective interaction performance data for comprehensive analysis. This ensures that the evaluation results do not rely solely on a single source or purely subjective judgment, significantly improving the objectivity, accuracy, and scientific rigor of the evaluation conclusions.

[0167] Enhanced assessment of interactive behavior and safety in dynamic driving scenarios: The method proposed in this invention explicitly includes a dynamic real-vehicle testing environment and introduces objective indicators such as eye-off time and eye-tracking heat map distribution to quantitatively assess safety. This compensates for the shortcomings of related methods in assessing real dynamic driving scenarios and the insufficient quantification of driving distraction and safety risks, helping to more realistically assess the impact of HMI touch operation on driving safety.

[0168] Providing precise guidance for HMI design optimization: Through detailed multi-dimensional, subjective and objective evaluation, this method can accurately pinpoint design flaws, operational difficulties, and user pain points in HMI touch interaction systems. The evaluation results can directly provide concrete and actionable basis and direction for the iteration of smart cockpit interaction design, human factors engineering evaluation, and improvements to specific functional modules (such as homepage / vehicle settings, media, communication, navigation, and air conditioning modules), thereby effectively improving the user experience. For example, in its application verification on the Volkswagen ID.7 model, the improved version achieved higher scores across all evaluation dimensions.

[0169] Enhancing the standardization and comparability of evaluation results: This invention provides a relatively complete and standardized evaluation process, encompassing evaluation dimensions, weight settings, test modules, test environment and task design, to data collection and analysis. This helps to standardize the evaluation process for HMI touch interaction user experience, improve the comparability of evaluation results between different vehicle models or systems, and promote the improvement of industry evaluation standards.

[0170] In summary, the technology of this invention, through its systematic evaluation framework, scientific weight setting, deep integration of subjective and objective data, and focus on dynamic security, can more effectively identify and solve user experience problems in HMI touch interaction, thereby providing strong support for improving the overall interaction quality and user satisfaction of smart cockpits.

[0171] The HMI touch interaction evaluation device provided in the embodiments of the present invention is described below. The HMI touch interaction evaluation device described below and the HMI touch interaction evaluation method described above can be referred to in correspondence.

[0172] like Figure 8As shown in the figure, an HMI touch interaction evaluation device is proposed in an embodiment of the present invention. This device can be applied to the cloud and includes: The first determining module 801 is used to determine multiple evaluation dimensions; the multiple evaluation dimensions include multiple of the following six dimensions: visual consistency of the interactive interface, rationality of the operation hierarchy, human-computer convenience, ease of understanding of text and icons, functional completeness, and security; Module 802 is used to obtain the weights corresponding to each evaluation dimension in multiple evaluation dimensions; The second determining module 803 is used to determine the evaluation object; the evaluation object includes at least one of the following objects: settings, media, communication, navigation, and air conditioning; The third determination module 804 is used to collect test data; the test data includes subjective feedback data and objective test data. The single-dimensional scoring module 805 is used to obtain the scores corresponding to each assessment dimension based on the subjective feedback data and objective test data corresponding to each assessment dimension. The comprehensive score module 806 is used to obtain the comprehensive score based on the weight and score corresponding to each evaluation dimension.

[0173] Figure 9 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 9 As shown, the electronic device may include a processor 910, a communications interface 920, a memory 930, and a communication bus 940, wherein the processor 910, the communications interface 920, and the memory 930 communicate with each other via the communication bus 940. In some embodiments, the processor 910 may include the processor 10 as described in any of the above embodiments. For example, the processor 910 may be the processor 910 as described in any of the above embodiments, or the processor 910 may be a processor array composed of the processors 910 of any of the above embodiments.

[0174] Furthermore, the logical instructions in the aforementioned memory 930 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to related technologies, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0175] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer is able to execute the HMI touch interaction evaluation method provided by the above methods, the method including: Multiple evaluation dimensions are determined; these dimensions include several of the following six dimensions: visual consistency of the interactive interface, rationality of the operation hierarchy, human-computer interaction convenience, comprehensibility of text and icons, functional completeness, and security. The weights corresponding to each evaluation dimension are obtained. Evaluation objects are determined; these objects include at least one of the following: settings, media, communication, navigation, and air conditioning. Test data is collected; this data includes subjective feedback data and objective test data. Based on the subjective feedback data and objective test data corresponding to each evaluation dimension, scores are obtained for each dimension. A comprehensive score is obtained based on the weights and scores of each dimension.

[0176] In another aspect, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to perform the HMI touch interaction evaluation method provided by the methods described above, the method comprising: Multiple evaluation dimensions are determined; these dimensions include several of the following six dimensions: visual consistency of the interactive interface, rationality of the operation hierarchy, human-computer interaction convenience, comprehensibility of text and icons, functional completeness, and security. The weights corresponding to each evaluation dimension are obtained. Evaluation objects are determined; these objects include at least one of the following: settings, media, communication, navigation, and air conditioning. Test data is collected; this data includes subjective feedback data and objective test data. Based on the subjective feedback data and objective test data corresponding to each evaluation dimension, scores are obtained for each dimension. A comprehensive score is obtained based on the weights and scores of each dimension.

[0177] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0178] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the parts that contribute to the related technology, can be embodied in the form of software products. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0179] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for evaluating HMI touch interaction, characterized in that, The method includes: Multiple evaluation dimensions are determined; these multiple evaluation dimensions include several of the following six dimensions: visual consistency of the interactive interface, rationality of the operation hierarchy, human-computer convenience, ease of understanding of text and icons, functional completeness, and security; Obtain the weights corresponding to each of the multiple evaluation dimensions; The evaluation objects are determined; the evaluation objects include at least one of the following: settings, media, communication, navigation, and air conditioning; Collect test data; the test data includes subjective feedback data and objective test data; Based on the subjective feedback data and objective test data corresponding to each evaluation dimension, the scores corresponding to each evaluation dimension are obtained. The overall score is obtained based on the weight and score corresponding to each evaluation dimension.

2. The method according to claim 1, characterized in that, Obtain the weights corresponding to each of the multiple evaluation dimensions, including: For the five dimensions other than security among the six dimensions, the weight of each of the five dimensions is obtained in the following manner: Obtain user weights and expert weights; Construct a judgment matrix based on the user weights and expert weights; The eigenvectors of the judgment matrix are calculated using the Analytic Hierarchy Process (AHP), and the eigenvectors are normalized to obtain the weight of each of the five dimensions.

3. The method according to claim 2, characterized in that, After obtaining the initial weights for each of the five dimensions, the method further includes: Calculate the largest eigenvalue of the judgment matrix; Calculate the consistency index based on the maximum eigenvalue; Query the average random consistency index; The consistency ratio is obtained based on the consistency index and the average random consistency index; In response to the consistency ratio being less than a preset threshold, the judgment matrix is ​​determined to have passed the consistency check.

4. The method according to any one of claims 1-3, characterized in that, Collect test data, including: In a pre-configured static testing environment, test data for at least one indicator of the evaluation object is collected from five dimensions: visual consistency of the interactive interface, rationality of the operation hierarchy, human-computer convenience, ease of understanding of text and icons, and functional completeness. In a pre-configured dynamic testing environment and under pre-configured driving conditions, test data for at least one indicator corresponding to the evaluation object are collected from a safety perspective.

5. The method according to any one of claims 1-3, characterized in that, The subjective feedback data is obtained by subjectively testing at least one indicator of the evaluation object; the objective test data is obtained by objectively testing at least one indicator of the evaluation object. The subjective testing includes using a scale assessment method and / or semi-structured data collection to gather subjective feedback data from users on at least one indicator of the assessment object; the subjective feedback data collected by the scale assessment method includes data corresponding to at least one of the following indicators: test task difficulty, functional understanding, visual clarity, feedback timeliness, user enjoyment, and personalized adaptability; the subjective feedback data collected by the semi-structured data collection includes data corresponding to at least one of the following indicators: effectiveness, satisfaction, comprehensibility, accessibility, controllability / feedback, personalized adaptability, and improvement suggestions; The objective test data includes data corresponding to at least one of the following indicators: HMI system function and hierarchy, application cold start and / or warm start latency, call operation response time, map operation response time, multimedia control response time, vehicle operation response time, Bluetooth operation response time, steering wheel control response time, interface scrolling smoothness, single and total eye-off time, number of eye-offs, eye movement hotspot distribution, single operation duration, number of single operation steps, task completion time, and total distance moved by finger operation.

6. The method according to claim 5, characterized in that, After collecting test data, the method further includes: The mean and standard deviation of the subjective feedback data collected by the scale assessment method are calculated; the subjective feedback data collected by the semi-structured acquisition is qualitatively coded and thematically summarized to obtain the first initial score corresponding to each indicator in the subjective feedback data. The objective test data is normalized to obtain the second initial score corresponding to each indicator in the objective test data.

7. The method according to claim 6, characterized in that, Based on the subjective feedback data and objective test data corresponding to each assessment dimension, scores are obtained for each assessment dimension, including: Calculate the score corresponding to each evaluation dimension based on the first initial score and / or the second initial score corresponding to at least one indicator for each evaluation dimension.

8. The method according to claim 5, characterized in that, After collecting test data, the method further includes: The Spearman rank correlation coefficient method was used to test the consistency between the subjective feedback data and the objective test data.

9. An HMI touch interaction evaluation device, characterized in that, The device includes: The first determining module is used to determine multiple evaluation dimensions; the multiple evaluation dimensions include multiple of the following six dimensions: visual coordination of the interactive interface, rationality of the operation hierarchy, human-computer convenience, ease of understanding of text and icons, functional completeness, and security; The acquisition module is used to acquire the weights corresponding to each of the multiple assessment dimensions. The second determining module is used to determine the evaluation object; the evaluation object includes at least one of the following objects: settings, media, communication, navigation, and air conditioning; The third determination module is used to collect test data; the test data includes subjective feedback data and objective test data. The single-dimensional scoring module is used to obtain the score corresponding to each assessment dimension based on the subjective feedback data and objective test data corresponding to each assessment dimension. The overall score module is used to obtain an overall score based on the weights and scores corresponding to each evaluation dimension.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the HMI touch interaction evaluation method as described in any one of claims 1 to 8.

Images