Map availability evaluation method and system based on multi-dimensional quantitative model

Abstract

The invention discloses a map availability evaluation method and system based on a multi-dimensional quantitative model, and belongs to the technical field of map making and user experience evaluation. The method comprises the following steps: firstly, constructing a VR immersion environment according to the mapping size and purpose of a map, and collecting multi-modal cognitive behavior data and subjective evaluation data of a subject; based on the multi-modal cognitive behavior data, statistical calculation is carried out on quantitative scores of three dimensions of element distinction degree, element readability and retrieval efficiency; and finally, through a hierarchical entropy weight and expert calibration fusion weighting mechanism, performing weighted fusion on the quantitative score of each dimension and the experience scale data to obtain a comprehensive availability evaluation result. According to the method, the natural reading scene of the map can be truly restored, multi-dimensional quantitative evaluation is realized, the evaluation efficiency and accuracy are improved, and a scientific basis is provided for map optimization.

Description

Technical Field

[0001] This invention relates to the field of map making and user experience evaluation technology, specifically to a map usability evaluation method and system based on a multi-dimensional quantitative model. Background Technology

[0002] As a carrier of spatial information, the usability of maps directly impacts users' efficiency in information retrieval and the accuracy of their decisions. Map usability assessment plays a crucial role in map production and application. During map compilation, factors such as text size, map capacity, print size, reading distance, the complexity of map content, and the specific needs of the target audience are closely related. In practice, designers typically refer to standards, experience, usability studies, or user testing to determine the characteristics of relevant elements. The core of map usability assessment is evaluating the impact of visual variables such as map content, graphic symbols, colors, and annotations on users' visual experience. This type of assessment can optimize map design, making it more aligned with users' visual habits and cognitive patterns, thereby improving the map's usability.

[0003] Existing eye-tracking-based map usability assessment experiments primarily employ fixed eye-tracking to collect data from participants. The participants' eye movements are captured as videos, which are then interpreted in subsequent processing. This video-based interpretation method requires extracting the fixation point and the map content being read from the recording, necessitating manual correlation between these two elements. Rapid analysis of the results faces challenges, impacting assessment efficiency. Furthermore, fixed eye-tracking limits head movement, failing to realistically simulate the free-movement, natural reading behavior of observing large wall maps or desktop maps in larger reading scenarios. This limitation on participants is a significant factor affecting map usability assessment. Simultaneously, map assessment metrics have not yet been systematically refined and quantified into core dimensions of map usability in VR immersive environments, and the assessment process is disconnected from the map design process, preventing rapid and direct feedback of assessment results to map element optimization. Therefore, a new map usability assessment method is needed that overcomes these problems and enables closed-loop evaluation. Summary of the Invention

[0004] The present invention aims to solve the above-mentioned technical problems by providing a map usability assessment method and system based on a multi-dimensional quantification model.

[0005] To solve the above-mentioned technical problems, the technical solution provided by the present invention is as follows:

[0006] A map usability assessment method based on a multi-dimensional quantization model includes the following steps:

[0007] S1. In a VR immersive environment constructed according to the map size and purpose, collect multimodal cognitive behavior data and subjective evaluation data of the subjects when conducting map cognition experiments;

[0008] The subjective evaluation data consisted of the subjects' scores on the two dimensions of element discrimination and element readability, as well as the experts' scores on the importance of the three dimensions of element discrimination, element readability, and retrieval efficiency, and the importance of each indicator.

[0009] S2. Based on multimodal cognitive behavior data, statistically calculate quantitative scores for three dimensions: element distinguishability, element readability, and retrieval efficiency.

[0010] The element distinguishability is represented by three indicators, the element readability is represented by four indicators, and the retrieval efficiency is represented by three indicators.

[0011] S3. A comprehensive usability evaluation mechanism based on hierarchical entropy weighting and expert calibration is used to weight and fuse quantitative scores and experience scale data of each dimension to obtain a comprehensive usability evaluation result.

[0012] Preferably, the multimodal cognitive behavioral data includes eye-tracking data and head movement data.

[0013] Preferably, the element distinguishability is characterized by three indicators: task area attention concentration, task area attention duration index, and attention range concentration; the element readability is characterized by four indicators: static reading effective duration, information revisit index, blink frequency, and head rotation frequency; and the retrieval efficiency is characterized by three indicators: visual allocation efficiency, visual switching efficiency, and retrieval path efficiency.

[0014] Preferably, S1 specifically includes:

[0015] S11. Construct a VR immersive environment based on the map size and purpose, select the corresponding virtual simulation map reading environment, determine the optimal reading distance, and publish the map product data through the map management module to display it in the reading environment;

[0016] S12. Conduct scenario-guided cognitive experiments to collect multimodal cognitive behavior data;

[0017] S13. Based on the cognitive experiment, subjective evaluation data are collected. After the subjects complete each task, they are asked to score the element discrimination and element readability. At the same time, the scores of experts on the dimensions and indicators are collected.

[0018] Preferably, S2 specifically includes:

[0019] S21. Statistically evaluate the values ​​of each indicator based on the collected data.

[0020] Task area attention concentration is the proportion of the number of fixations made by the participant in the task area out of the total number of fixations, as shown in the formula below: ;in, The total number of fixations by the subject in the task area. The total number of fixations during the completion of the task represents the number of times the subject fixates; a higher value indicates higher element discrimination.

[0021] The task area attention duration index is the ratio of task area attention duration to total attention duration, as shown in the formula below: ;in, The total fixation time of the participants in the task area. The total fixation time of the participants throughout the task; a larger value indicates higher element discrimination.

[0022] The focus concentration is the ratio of the area of ​​the circumscribed polygon of the fixation point to the total map area, as shown in the formula below: ;in, To determine the coordinates of the subjects' fixation points and calculate the area of ​​the circumscribed polygon of the fixation point, This represents the map area; a larger value indicates a higher degree of feature differentiation.

[0023] The effective duration of static reading is the proportion of the total attention time to task elements during the head-static state to the cumulative head-static state duration, as shown in the following formula: ;in, To determine the total gaze duration of task elements during a static head position, This represents the cumulative duration of the header in a static state throughout the entire task; a larger value indicates higher readability of the element.

[0024] The information return index is the proportion of the total effective fixation time of a subject when they return to an element after the initial removal, out of the total fixation time. The formula is as follows: ;in, The term refers to the total effective fixation time of the subject upon returning to an element after the initial departure, with a single fixation duration ≥ 20 ms and a head angular velocity ≤ 0.5° / s. The total fixation time of the participants throughout the task; a larger value indicates higher readability of the elements.

[0025] Blink frequency is calculated as the number of effective blinks divided by the total blink duration, as shown in the formula below: ;in, The total number of effective blinks by the subject during the experiment. The total time spent by the participants on the task; a higher value indicates lower readability of the elements.

[0026] The head rotation frequency is calculated as the number of effective head rotations by the subject divided by the total duration, as shown in the following formula: ;in, This refers to the number of effective head rotations by the subject during the experiment, where the effective head rotation angle is >5° and the duration is >100ms. The total time spent by the participants on the task; a higher value indicates lower readability of the elements.

[0027] Visual allocation efficiency is the proportion of the effective fixation time of the user in the target area to the total effective fixation time of the "target area + non-target area" during the subject retrieval phase (from the start of the task to the first target localization), as shown in the following formula: ;in, From the start of the task to the completion of the retrieval task at the initial location of the task target, the participants were aware of the task target's location. The cumulative effective fixation time within the period, This represents the cumulative effective fixation time of the subject in the non-task target area during this period; a larger value indicates higher retrieval efficiency.

[0028] Visual switching efficiency is the proportion of eye saccade time to the total task duration during task completion, as shown in the formula below: ;in, The time it takes for the eyes to saccade during the task. This represents the total time the subject spent on the task; a larger value indicates higher retrieval efficiency.

[0029] The retrieval path efficiency is the ratio of the length of the shortest scan path to the actual scan search path, as shown in the formula below: ;in The shortest path length for the subject to complete the task scan. To determine the actual search path length for participants to complete the task, a larger value indicates higher retrieval efficiency.

[0030] S22. For each dimension, preprocess the indicator data, and perform positive transformation, outlier handling, and subjective score standardization on negatively correlated indicators.

[0031] S23. For each dimension, the median-absolute median difference method is used to standardize the preprocessed index data of the three dimensions, statistical benchmark parameters are obtained, the standardization boundary is determined, and the standardized value is calculated to achieve scale uniformity for indicators of different dimensions.

[0032] Preferably, S3 specifically includes:

[0033] S31, Level 1 Empowerment:

[0034] Weights are assigned to the indicators within the three dimensions of element distinguishability, element readability, and retrieval efficiency, ensuring that the sum of the weights of the indicators within each dimension is 1.

[0035] Experts were invited to rate the importance of each indicator within the target dimension on a scale of 1 to 10, and the scores were normalized to obtain the expert weight vector.

[0036] Based on standardized indicator data, the indicator weight, information entropy, and entropy weight are calculated, and the indicator fusion coefficient is used. The indicator entropy weight and indicator expert weight are weighted and integrated, and normalization ensures that the sum of indicator weights in each dimension is 1.

[0037] S32, Second-level empowerment:

[0038] Weights were calculated for three dimensions: feature distinguishability, feature readability, and retrieval efficiency. Dimension scores were calculated for each participant, and dimensional entropy weights were calculated based on these scores. Experts were invited to rate the overall importance of the three dimensions, and the scores were normalized to obtain the expert weights for each dimension. Dimension fusion coefficients were then used. The dimensional entropy weight and dimensional expert weight are integrated, and normalization is performed to ensure that the sum of the dimensional weights is 1.

[0039] S33. Based on the quantitative scores of each dimension and the experience scale data, obtain the usability assessment results.

[0040] The present invention also provides a map usability evaluation system based on a multi-dimensional quantitative model, including a map management module, a map reading experiment module, and a terminal device;

[0041] The map management module is used to establish a map database based on the map to manage the compiled vector map. It is designed for virtual reality scene interaction, and enables map service publishing and map query and analysis services. Based on the geographic coordinates of the gaze point provided by the map reading experiment module, it retrieves the appearance information of the gaze element and supports dynamic updates of map data.

[0042] The map reading experiment module includes an experiment guidance module and an experiment recording module;

[0043] The experiment guidance module guides participants to conduct map cognition experiments based on the experimental scenario, determines the participants' gaze positions, sends the gaze positions to the map management module and interacts with it, and picks up map elements to obtain map element information.

[0044] The experimental recording module collects cognitive data of the participants in the map reading experiment based on eye-tracking technology, including fixation timestamp, fixation position, fixation duration, saccade timestamp, saccade duration, saccade path coordinates, blink timestamp, instantaneous head VR angular velocity, cumulative head rotation angle, fixation map element ID, element appearance information, etc.

[0045] The terminal device includes a VR headset, VR controller, eye-tracking module, and motion capture module;

[0046] VR headsets and VR controllers are used for scene roaming, information confirmation, and subjective rating data collection;

[0047] The eye-tracking module is used to acquire eye movement information;

[0048] The motion capture module is used to acquire head movement information.

[0049] Preferably, after the map reading experiment module completes the user intent judgment, it sends a buffer query request containing geographic coordinates to the map management module. After receiving the query result data, it filters the picked elements through the gaze element matching confidence judgment method and takes the closest one as the hit object.

[0050] Methods for judging the confidence of gaze element matching include task object type matching and distance matching.

[0051] The task object type matching judgment determines the scope by matching the current task object type, and filters objects in the element library that match in both type and level according to the current task objective.

[0052] Distance matching is used to check the distance between the gaze point and the target element. If multiple elements meet the conditions, the one with the closest distance is selected as the picking object.

[0053] By employing the above methods and systems, the present invention has the following advantages:

[0054] 1. Based on the map scale and purpose, this invention constructs corresponding usage scenarios using virtual reality. Based on eye-tracking technology, it evaluates the quality of map compilation by collecting data on the impact of map elements on the visual perception and cognitive process of the subjects.

[0055] 2. This invention constructs a map usability evaluation system based on a VR immersive environment, breaking through the technical limitations of traditional fixed eye-tracking. It realistically reproduces various reading scenarios, from wall charts to desktop maps, and by integrating head movement and eye-tracking data, it achieves more detailed capture of users' natural reading behavior, thus more comprehensively reflecting the user behavior characteristics of map reading in actual use environments.

[0056] 3. This invention establishes a map usability evaluation index system and scientific weighting method based on VR environment. It includes three core dimensions: feature distinguishability, readability, and retrieval efficiency, which respectively evaluate symbol recognition effect, information understanding degree, and map cognitive efficiency. It also combines a hierarchical entropy weight calculation and expert calibration fusion mechanism to ensure that the weight allocation maintains data objectivity and meets the needs of professional scenarios.

[0057] The above overview is for illustrative purposes only and is not intended to be limiting in any way. In addition to the illustrative aspects, embodiments, and features described above, further aspects, embodiments, and features of the invention will become readily apparent from the accompanying drawings and the following detailed description. Attached Figure Description

[0058] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0059] Figure 1 This is a flowchart illustrating the evaluation method of the present invention;

[0060] Figure 2 This is a schematic diagram of the system functional modules of the present invention;

[0061] Figure 3 This is the eye-tracking and map interaction data flow diagram of the present invention;

[0062] Figure 4 This is the experimental flowchart of the present invention. Detailed Implementation

[0063] Specific embodiments of the invention will now be described in detail. Although the invention is described in conjunction with these specific embodiments, it should be understood that the invention is not intended to be limited to these specific embodiments. Rather, these embodiments are intended to cover alternative, modified, or equivalent embodiments that may be included within the spirit and scope of the invention as defined by the claims. In the following description, numerous specific details are set forth in order to provide a thorough understanding of the invention. The invention may be practiced without some or all of these specific details. In other instances, well-known processes have not been described in detail so as not to unnecessarily obscure the invention.

[0064] When used in conjunction with the terms "comprising," "method comprising," or similar language in this specification and appended claims, the singular forms "a," "some," and "the" include plural references unless the context clearly indicates otherwise. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0065] The present invention will now be described in further detail with reference to the full text.

[0066] Combined with appendix Figures 1-4The purpose of this invention is to provide a VR map usability evaluation method and system that can realistically reproduce the natural reading scene of a map and quantify the evaluation results from multiple dimensions.

[0067] To achieve the above objectives, the technical solution of the present invention is: a map usability assessment method and system based on multimodal data and a multidimensional quantization model in a VR environment, comprising the following steps:

[0068] S1: In a VR immersive environment constructed according to the map size and purpose, collect multimodal cognitive behavior data and subjective evaluation data of the subjects when conducting map cognition experiments.

[0069] The multimodal cognitive behavior data includes eye-tracking data and head movement data. The subjective evaluation data consists of participants' scores (1-10 points) on two dimensions: element discrimination and element readability, as well as expert scores (1-10 points) on the importance of three dimensions: element discrimination, element readability, and retrieval efficiency, and the importance of each indicator.

[0070] S2: Based on the multimodal cognitive behavior data, calculate the quantitative scores of three dimensions: element distinguishability, element readability, and retrieval efficiency.

[0071] Among them, the element differentiation can be characterized by three indicators: task area attention concentration, task area attention duration index, and attention scope concentration; the element readability can be characterized by four indicators: static reading effective duration, information revisit index, blink frequency, and head rotation frequency; and the retrieval efficiency can be characterized by three indicators: visual allocation efficiency, visual switching efficiency, and retrieval path efficiency.

[0072] S3: A comprehensive usability evaluation mechanism based on hierarchical entropy weighting and expert calibration is used to weight and fuse quantitative scores and experience scale data of each dimension to obtain a comprehensive usability evaluation result.

[0073] Step S1 includes:

[0074] S11. Construct a VR immersive environment based on the map's size and intended use;

[0075] S12. Guide cognitive experiments based on the scenario and collect multimodal cognitive behavior data;

[0076] S13. Based on the cognitive experiment, collect subjective evaluation data;

[0077] The usability assessment dimensions include feature distinguishability, feature readability, and retrieval efficiency.

[0078] Step S2 includes:

[0079] S21. Statistically evaluate the values ​​of each indicator based on the collected data.

[0080] S22. For each dimension, the indicator data is preprocessed;

[0081] S23. For each dimension, the indicator data is standardized.

[0082] When the usability assessment dimension is element discrimination, the included indicators are: task area attention concentration, task area attention duration index, and attention scope concentration. Step S21 includes:

[0083] Task area attention concentration: The proportion of the number of fixations the subject made in the task area out of the total number of fixations;

[0084] Task Area Attention Duration Index: The ratio of task area attention duration to total attention duration;

[0085] Concentration of attention area: The proportion of the area of ​​the polygon circumscribed by the point of focus to the total map area.

[0086] In the method provided by this invention, when the usability assessment dimension is element readability, the included indicators are: effective static reading time, information revisit index, blink frequency, and head rotation frequency. Step S21 includes:

[0087] Effective static reading time: The proportion of total fixation time on task elements during head static state to the total cumulative head static state time;

[0088] Information return index: The proportion of total effective fixation time to total fixation time after a subject first leaves an element and then returns to it;

[0089] Blink frequency: Number of effective blinks / total duration;

[0090] Head rotation frequency: Number of effective head rotations by the subject / total duration;

[0091] In the method provided by this invention, when the usability evaluation dimension is retrieval efficiency, the included indicators are: visual allocation efficiency, visual switching efficiency, and retrieval path efficiency. Step S21 includes:

[0092] Visual allocation efficiency: During the subject retrieval phase (from the start of the task to the first target localization), the proportion of the user's effective fixation time in the target area to the total effective fixation time of "target area + non-target area";

[0093] Visual switching efficiency: The proportion of eye saccade time to the total task duration during task completion;

[0094] Search path efficiency: The proportion of the shortest scan path length to the actual scan search path.

[0095] In specific implementation of this invention, such as Figure 2 As shown, a map usability assessment system based on a multi-dimensional quantification model is also provided, including:

[0096] The map management module is used to manage the compiled vector maps by establishing a map database. It supports map service publishing and query / analysis services for virtual reality scene interaction. Based on the geographic coordinates of the gaze point provided by the map reading experiment module, it retrieves the appearance information of gaze features and supports dynamic updates of map data to quickly optimize map compilation results.

[0097] The map reading experiment module includes an experiment guidance module and an experiment recording module. The experiment guidance module guides participants through a map cognition experiment based on an experimental scenario. It determines the participant's gaze position, sends the gaze position to the map management module, interacts with the map management module, picks up map elements, and obtains map element information. Cognitive data from the participants in the map reading experiment is collected using eye-tracking technology, including gaze timestamps, gaze positions, gaze durations, saccade timestamps, saccade durations, saccade path coordinates, blink timestamps, instantaneous head angular velocity in VR, cumulative head rotation angle, gazed map element IDs, and element appearance information.

[0098] The terminal device consists of a VR headset, VR controllers, an eye-tracking module, and a motion capture module. The VR controllers are used for scene navigation, information confirmation, and subjective rating data collection. The eye-tracking module is used to acquire eye movement information, and the motion capture module is used to acquire head movement information.

[0099] The proposed eye-tracking-based map usability assessment method systematically constructs a mapping relationship between cognitive behavioral indicators and map usability dimensions. By quantifying the cognitive load and search strategies of subjects in map reading tasks, it deeply analyzes the cognitive mechanisms behind map design problems and provides a scientific basis for map optimization.

[0100] This invention first publishes the compiled desktop maps, wall maps, and other results as interactive electronic maps. Based on the map's purpose and size, a corresponding virtual reality reading scenario is determined, and the map and scenario are displayed on a head-mounted device worn by the subjects. Through an eye-tracking module in the head-mounted device, the subjects' cognitive behavior regarding the map is collected during the experiment. When a subject focuses on a map element, the gaze position is obtained, and the map element is retrieved using the spatial location query function on the electronic map, acquiring its graphic information. Simultaneously, the subject's gaze behavior and visual processing information at that element are recorded. This method allows for the quantitative analysis of subjects' map reading behavior, enabling mapmakers to flexibly set reading environments at different viewing distances, quickly change the appearance information of map elements and receive feedback. The data support provided by eye-tracking technology reduces the impact of subjective judgment on map compilation, improving compilation efficiency.

[0101] Example 1:

[0102] A map usability assessment method based on a multi-dimensional quantization model includes the following steps:

[0103] S1. In a VR immersive environment constructed according to the map size and purpose, collect multimodal cognitive behavior data and subjective evaluation data of the subjects when conducting map cognition experiments;

[0104] S11. Construct a VR immersive environment based on the map's size and intended use:

[0105] In its specific implementation, for map product A, a corresponding reading environment is selected based on the map size and evaluation requirements. This environment is either desktop (for short distances) or wall-mounted (for long distances), and the optimal reading distance is determined. The map product's data is published and displayed in the reading environment through the map management module of the map usability evaluation experimental system. All corresponding cognitive experimental tasks are evaluated using this reading environment.

[0106] Specifically, the system is installed on a computer equipped with 16GB of RAM, an NVIDIA GTX 1080 graphics card, and a 4.00GHz Intel i7-6700K processor to drive the HTC Vive VR device. An eye-tracking module is installed on the VR glasses and connected to the PC for eye-tracking calibration. The head-mounted VR system (HTCVIVE) integrates SteamVR Tracking 1.0 technology and the Chaperone launch system, featuring a combined resolution of 2160*1200 (1080*1200 per eye) and a 90Hz refresh rate, achieving a tracking accuracy of 0.1 degrees. The eye-tracking module can be connected to an HTC display (Aglass 2). The eye-tracking module has a refresh rate of 120~380Hz, a field of view of 110 degrees, and a tracking accuracy of 0.5 degrees.

[0107] The experiment was conducted using an experimental system developed based on Unity 2018.2.17f1, which guided the participants to complete the experiment. The participants interacted with the scene by operating the controller. The experimental system recorded the participants' head movement posture, eye movement gaze data, and gaze element information in the virtual scene in the form of logs, and integrated these data into the 3D scene for visualization analysis.

[0108] S12. Guide cognitive experiments based on the scenario and collect multimodal cognitive behavior data;

[0109] In specific implementation of this invention, such as Figure 3 and Figure 4 As shown, for map product A, an experimental checklist was compiled based on the evaluation results, and n=15 participants (including 3 domain experts) were selected to participate in the usability experiment. The experimental flowchart is shown below. Figure 4 The experiment began with an introduction, where the experimenter guided the participants through the basic information about the research process. Next, participants wore a motion-tracking eye system and underwent 3D glasses calibration. They then spent approximately five minutes training to familiarize themselves with the virtual environment. During this training, participants moved freely within the virtual scene, adapting to the environment and operating the controllers, but were placed in a location away from the experimental area. Each participant had sufficient time to familiarize themselves with the virtual reality environment and the operation of the equipment before the formal experiment.

[0110] Specifically, in the formal experiment, participants were first asked to view the experimental task, which was a location or search task. They were asked to find 10 sets of map elements, such as government locations, roads, and rivers. Specific tasks included: "Please find 'Yunlong District, Dalonghu Street' on the current map. Press the left menu button to start and the right menu button to confirm to end," "Please find 5 parks on the current map. Press the left menu button to start and the right menu button to confirm to end," and "In the city center area, please find a 'school' and name the road next to it. Press the left menu button to start and the right menu button to confirm to end," etc. Each task started with reading the instructions. After the participant clicked confirm, the system began recording behavioral information. After completing the task and pressing the confirm button, the participant was asked to rate the element discrimination and readability of the experimental object based on their own experience. After completing the evaluation, they moved on to the next element search task.

[0111] Cognitive data was collected from each participant during the experiment, including eye-tracking data, body behavior data, behavioral performance data, and subjective scoring data. Eye-tracking data included fixation data, saccade data, and blink data; body behavior data included head movement data; and behavioral performance data included task completion time.

[0112] S13. Based on the aforementioned cognitive experiment, collect subjective evaluation data:

[0113] Subjective scoring data consisted of participants' scores on two dimensions, element discrimination and element readability (1-10 points), as well as expert scores on three dimensions, element discrimination, element readability, and retrieval efficiency, and the importance of their indicators (1-10 points).

[0114] S2. Based on multimodal cognitive behavior data, statistically calculate quantitative scores for the dimensions of element distinguishability, element readability, and retrieval efficiency.

[0115] S21. Statistical evaluation of the values ​​of each indicator based on the collected data:

[0116] Based on the eye-tracking test data / behavioral performance of the i-th participant (i=1,2,...,n, n=15 in this experiment) during task completion, three indicators of their task completion performance on the element discrimination dimension were calculated:

[0117] Task area focus concentration The proportion of fixations made by the subject in the task area out of the total number of fixations is given by the following formula:

[0118] ;in, The total number of fixations by the subject in the task area. The total number of fixations made by the subject while completing the entire task;

[0119] The value ranges from [0,1], with a larger value indicating higher feature distinguishability.

[0120] Task Area Attention Duration Index The ratio of task area fixation time to total fixation time is given by the following formula:

[0121] ;in, The total fixation time of the participants in the task area. The total fixation time of the participants throughout the task;

[0122] The value ranges from [0,1], with a larger value indicating higher feature distinguishability.

[0123] Concentration of attention The formula for the proportion of the area of ​​the circumscribed polygon of the focal point to the total map area is shown below:

[0124] ;in, To determine the coordinates of the subjects' fixation points and calculate the area of ​​the circumscribed polygon of the fixation point, The area of ​​the map;

[0125] The value ranges from [0,1], with a larger value indicating higher feature distinguishability.

[0126] Based on the eye-tracking test data / behavioral performance of the i-th participant (i=1,2,...,n, n=15 in this experiment) during task completion, four indicators of their task completion performance on the element readability dimension were calculated:

[0127] Effective duration of static reading This represents the proportion of total fixation time on task elements during a static head state to the total cumulative head-static time. It is calculated as the percentage of effective fixation time (excluding saccades) based on head motion angular velocity (≤0.5° / s) during a static head state, using the formula shown below:

[0128] ;in, To determine the total gaze duration of task elements during a static head position, This represents the cumulative duration of the head in a static state throughout the entire mission.

[0129] The value ranges from [0,1], with larger values ​​indicating higher readability of the element.

[0130] Information Recall Index The formula for the proportion of the total effective fixation time to the total fixation time after the subject first leaves and then returns to the element is as follows:

[0131] ;in, The term refers to the total effective fixation time of the subject upon returning to an element after the initial departure, with a single fixation duration ≥ 20 ms and a head angular velocity ≤ 0.5° / s. The total fixation time of the participants throughout the task;

[0132] The value ranges from [0,1], with larger values ​​indicating higher readability of the element.

[0133] blink frequency The formula is: effective blink count / total blink duration, as shown below:

[0134] ;in, The total number of effective blinks by the subject during the experiment. The total time spent by the participants on the task;

[0135] Value range [10, 60], unit: times / minute, the higher the value, the lower the readability of the element;

[0136] Head rotation frequency The formula for calculating the effective number of head rotations by the subject divided by the total duration is shown below:

[0137] ;in, This refers to the number of effective head rotations by the subject during the experiment, where the effective head rotation angle is >5° and the duration is >100ms. The total time spent by the participants on the task;

[0138] Value range [0, 60], unit: times / minute, the higher the value, the lower the readability of the element;

[0139] Based on the eye-tracking test data / behavioral performance of the i-th participant (i=1,2,...,n, n=15 in this experiment) during task completion, three indicators of their task completion performance in the retrieval efficiency (EOR) dimension were calculated:

[0140] Visual allocation efficiency The effective fixation time in the target area is the proportion of the total effective fixation time in the target area plus the non-target area during the retrieval phase (from the start of the task to the initial target location). The formula is as follows:

[0141] ;in, From the start of the task to the completion of the retrieval task at the initial location of the task target, the participants were aware of the task target's location. The cumulative effective fixation time within the period, This refers to the cumulative effective fixation time of the participants in the non-task target area during this period;

[0142] The value ranges from [0,1], and the larger the value, the higher the retrieval efficiency.

[0143] Visual switching efficiency The formula for the proportion of eye saccade time to the total task duration during task completion is as follows:

[0144] ;in, The time it takes for the eyes to saccade during the task. To determine the total time spent by the participants in the task;

[0145] The value ranges from [0,1], and the larger the value, the higher the retrieval efficiency.

[0146] Search path efficiency The formula for the proportion of the shortest scan path length to the actual scan search path is shown below:

[0147] ;in The shortest path length for the subject to complete the task scan. To determine the actual search path length for participants to complete the task;

[0148] The value ranges from [0,1], and the larger the value, the higher the retrieval efficiency.

[0149] S22, Pretreatment

[0150] This includes positiveing ​​of negatively correlated indicators, outlier handling, and standardization of subjective scores, among which:

[0151] Positive correlation index: For the i-th subject, blink frequency Press RBlinkFreq norm,i =(60-RBlinkFreq i ) / (60-10) conversion, RBlinkFreq i When ≤10, take 1, RBlinkFreq i Take 0 when ≥60; Head rotation frequency RHeadFreq i Press RHeadFreq norm,i =(30-RHeadFreq i ) / 30 conversion, RHeadFreq i When the value is ≥30, the value is 0;

[0152] Outlier handling: Based on all positively calibrated indicator data, calculate the median of each indicator across the entire sample of all participants (Med). Pre ) and absolute median (MAD) Pre The normal data range is defined as [Med] Pre -3×MADPre Med Pre +3×MAD Pre Outliers outside this range are filled with the median of the corresponding metric.

[0153] Subjective score standardization: Subjective raw scores (Subj) of subject's element discrimination (DOE) and element readability (ROE). raw,i According to the formula Standardize to the [0,1] interval to obtain the subjective experience dimension score S. Subj,i .

[0154] S23. Standardization process:

[0155] To address the characteristics of eye-tracking data, such as small sample size and the tendency for extreme values, the "median-absolute median difference method" was used to standardize the preprocessed data across the three dimensions.

[0156] Statistical baseline parameter: The median of each statistical indicator in the entire sample (Med). Stan ) and absolute median (MAD) Stan );

[0157] Determine the standardization boundary: lower limit L Stan =Med Stan -3×MAD Stan , upper limit U Stan =Med Stan +3×MAD Stan ;

[0158] Standardized formula: When the index value X Stan <L Stan When X is zero, the standardized value is 0; when X is zero, the standardized value is 0. Stan >U Stan When L is at that time, the standardized value is 1; when L is at that time, the standardized value is 1. Stan ≤X Stan ≤U Stan When, the standardized value = ;

[0159] After standardization, for the i-th participant, the standardized values ​​DConcAOT for all indicators within each dimension are obtained. Std,i DDurAOT Std,i DScopeAOT Std,i ,RStaticRead Std,i RRevis Std,i RBlinkFreq Std,i RHeadFreq Std,i EAlloc Std,i ESwitch Std,i EPath Std,iThis achieves a unified scale for indicators across different dimensions.

[0160] S3: Based on the weighting mechanism that combines hierarchical entropy weighting with expert calibration, a comprehensive usability assessment result is obtained.

[0161] The weights are determined by a hierarchical entropy weighting and expert calibration fusion weighting mechanism. The weights are calculated in two levels to ensure that the sum of the weights at the same level is 1. Finally, the weights are aggregated to obtain the comprehensive score.

[0162] S31. First-level weighting: Calculation of indicator weights within each dimension:

[0163] Weights are applied to the three dimensions of DOE (Distinctiveness of Element), ROE (Readability of Element), and EOR (Efficiency of Retrieval), ensuring that the sum of the weights of the indicators within each dimension is 1.

[0164] Expert weight acquisition: Experts were invited to rate the importance of each indicator within the target dimension on a scale of 1 to 10. Taking the DOE (Discrimination Index) as an example, the expert weight vector was obtained after normalization.

[0165] ;

[0166] (satisfy ).

[0167] Entropy weight calculation: For standardized indicator data of a single dimension, using DOE... For example, the weight of indicators is calculated based on standardized data. ( For the i-th subject at the index (Standardized value), calculation index Information entropy (k=1 / lnn, n is the total number of subjects) Calculate the information entropy, and finally obtain the entropy weight. :

[0168] ;

[0169] Weighting calculation: The weight of the indicator is calculated using the indicator fusion coefficient. By weighting and fusing entropy weights and expert weights, taking DOE's DConcAOT as an example, the final weights can be obtained. The fusion formula is as follows:

[0170] ;

[0171] in For expert weighting, The weights are entropy weights, and after fusion, they are normalized to ensure that the sum of the weights is 1.

[0172] Taking the DOE (Discrimination Index of Elements) as an example, the weight vector obtained after fusion is:

[0173] ;

[0174] (satisfy );

[0175] S32, Secondary Weighting: Core Dimension Weight Calculation:

[0176] Calculate the weights for the three dimensions DOE, ROE, and EOR, ensuring that the sum of the dimension weights is 1. The three dimensions DOE, ROE, and EOR contain a total of 10 indicators.

[0177] Basic data preparation: Calculate the dimensional scores for each participant. Taking the i-th participant as an example, the formulas for each dimension score are as follows:

[0178] DOE dimension score:

[0179]

[0180] ROE score:

[0181]

[0182] EOR dimension score:

[0183]

[0184] Entropy weight and expert weight acquisition: Calculating dimensional entropy weight W based on dimensional scores as the basic data. dim_entropy_DOE W dim_entropy_ROE W dim_entropy_EOR The aforementioned experts were invited to score the overall importance of the three dimensions, and the resulting normalized scores yielded the expert weights W for each dimension. dim_expert_DOE W dim_expert_ROE W dim_expert_EOR ;

[0185] Dimensional fusion weight calculation: Dimensional fusion weights are calculated using dimensional fusion coefficients. To perform the calculation, taking the DOE (Discrimination Index of Elements) as an example, the formula is: After fusion and normalization, ensure W dim_integrat_DOE +W dim_integrat_ROE +W dim_integrat_EOR =1.

[0186] In this embodiment, the fusion weight is taken as follows: =0.4, =0.4, and the dimension weight results are shown in Table 1:

[0187] Table 1 Dimension Weight Results

[0188] S33. Based on the quantitative scores of each dimension and the experience scale data, obtain the usability assessment results.

[0189] Weights of the subjective experience dimension in the experience scale data ∈[0.1,0.3], this experiment The overall score of the i-th subject:

[0190]

[0191] Overall map usability score:

[0192] ;

[0193] A higher numerical value indicates better overall map usability. In this embodiment, the task scores for each dimension are shown in Table 2:

[0194] Table 2 Task Scores by Dimension

[0195] Substituting the data, the overall score is calculated as follows: 0.78 × 0.35 + 0.72 × 0.39 + 0.65 × 0.26 = 0.72.

[0196] According to another aspect of the present invention, an eye-tracking-based interface usability evaluation system is also provided, comprising:

[0197] (1) Map Management Module: This module establishes a map database to manage the compiled vector maps. It supports interactive virtual reality scenarios and enables map service publishing and map query analysis services. Based on the geographic coordinates of the gaze point provided by the map reading experiment module, it retrieves the appearance information of the gaze element and supports dynamic updates of map data to quickly optimize the map compilation results.

[0198] (2) Map reading experiment module, including experiment guidance module and experiment recording module. The experiment guidance module guides the participants to conduct map cognition experiments based on the experimental scenario, judges the participants' gaze positions, sends the gaze positions to the map management module, interacts with the map management module, picks up the gazed map elements, and obtains map element information. Based on eye-tracking technology, cognitive data of the participants in the map reading experiment is collected, including gaze timestamp, gaze position, gaze duration, saccade timestamp, saccade duration, saccade path coordinates, blink timestamp, head VR instantaneous angular velocity, cumulative head rotation angle, gazed map element ID, element appearance information, etc.

[0199] The information interaction process between the modules is described in [link to document]. Figure 3After the map reading experiment module completes the user intent judgment, it sends a buffer query request containing geographic coordinates to the map management module. Upon receiving the query result data, the map reading experiment module filters the picked elements using a gaze element matching confidence judgment method, selecting the closest element as the target. The method is as follows: First, task object type matching judgment. The range is narrowed down by matching the current task object type. Based on the current task objective, objects in the element library that match both type and level are filtered, excluding irrelevant elements to narrow the judgment range. Second, distance matching judgment. The distance between the gaze point and the target element is verified. If multiple elements meet the condition, the closest element is selected as the target. In this embodiment, when the task objective is "Dalonghu Street," only objects with type = "annotation" and level = "township level" are retained by task object type matching, and only elements with a point ≤ 5mm and an annotation ≤ 10mm are retained by distance matching.

[0200] (3) Terminal equipment, which consists of VR headset, VR controller, eye tracking module and motion capture module. The VR controller is used for scene roaming, information confirmation and subjective scoring data collection. The eye tracking module is used to obtain eye movement information and the motion capture module is used to obtain head movement information.

[0201] The present invention and its embodiments have been described above. This description is not restrictive, and the embodiments shown throughout are only one of the embodiments of the present invention. The actual structure is not limited to this. In conclusion, if those skilled in the art are inspired by this description and design similar structures and embodiments without departing from the spirit of the present invention, they should all fall within the protection scope of the present invention.

Claims

1. A map usability evaluation method based on a multi-dimensional quantification model, characterized by, Comprise the following steps: S1, in the VR immersive environment constructed according to the map mapping size and use, collect the multi-modal cognitive behavior data and subjective evaluation data of the subject during the map cognition experiment; The subjective evaluation data is the scoring data of the subject in the two dimensions of element distinguishability and element readability, and the experts' scoring of the importance of the three dimensions of element distinguishability, element readability and search efficiency and the importance of each index; S2, based on the multi-modal cognitive behavior data, statistical calculation is carried out on the three dimension quantitative scores of element distinguishability, element readability and search efficiency; The element distinguishability includes three index representations, the element readability includes four index representations, and the search efficiency includes three index representations; S3, based on the availability comprehensive evaluation mechanism of hierarchical entropy weight and expert calibration fusion weighting, the dimension quantitative scores and experience scale data are weighted and fused to obtain the comprehensive availability evaluation result.

2. The method of claim 1, wherein the method comprises: The multi-modal cognitive behavior data includes eye tracking data and head movement data.

3. The method of claim 2, wherein the method further comprises: The element distinguishability includes three index representations of task area attention concentration, task area attention time length index and attention range concentration;The element readability includes four index representations of static reading effective time length, information review index, blink frequency and head rotation frequency;The search efficiency includes three index representations of visual allocation efficiency, visual switching efficiency and search path efficiency.

4. The method of claim 3, wherein the method further comprises: The S1 specifically comprises: S11, constructing VR immersive environment according to map mapping size and use, selecting corresponding virtual simulation map reading environment, determining optimal reading distance, and publishing map product data in reading environment through map management module; S12, based on scene guided cognition experiment, multi-modal cognitive behavior data is collected; S13, based on the cognition experiment, subjective evaluation data is collected, and the subject is required to score element distinguishability and element readability after completing each task, and expert scoring of dimensions and indexes is collected.

5. The method of claim 3, wherein the method further comprises: The S2 specifically comprises: S21, based on the collected data, the index values of each dimension are statistically evaluated; The attention concentration degree in the task area is the proportion of the number of fixations in the task area to the total number of fixations, and is shown in the following formula: ; wherein, is the total number of fixations in the task area, is the total number of fixations when the subject completes the entire task, and the greater the value represents the higher the element differentiation degree; The task area attention duration index is the ratio of the task area fixation duration to the total fixation duration, and the formula is as follows: ; wherein, is the total fixation duration of the subject in the task area, is the total fixation duration of the subject in the entire task, and the larger the value represents the higher the element differentiation degree; The attention range concentration degree is the proportion of the area of the attention point circumscribed polygon to the area of the map, and the formula is as follows: ; wherein, is the coordinate of the attention point of the statistical subject, the area of the attention point circumscribed polygon is calculated, is the area of the map, and the larger the value represents the higher the element differentiation degree; The static reading effective duration is the proportion of the total fixation duration of the task element in the head static state to the cumulative duration in the head static state, and the formula is as follows: ; wherein, is the total fixation duration of the task element in the head static state, is the cumulative duration in the head static state during the entire task, and the larger the value is, the higher the readability of the element is. The information return index is the proportion of the total effective fixation duration of the subject to the total fixation duration of the element returning after the first departure, and the formula is as follows: ; wherein, The total effective fixation duration of the subject to the element returning after the first departure is greater than or equal to 20 ms and the head angular velocity is less than or equal to 0.5° / s, The total fixation duration of the subject in the whole task is greater, which represents that the readability of the element is higher. The blink frequency is the number of valid blinks of the subject / total duration, and the formula is as follows: ; wherein, is the total number of valid blinks of the subject in the experiment, is the total duration of the subject in the task, and the higher the value represents the lower the readability of the element; The head rotation frequency is the number of valid head rotations of the subject / total duration, and the formula is as follows: ; wherein, The number of valid head rotations of the subject in the experiment is the number of valid head rotations of the subject, the valid head rotation angle is greater than 5° and the duration is greater than 100 ms, The total duration of the subject in the task is the higher the value, the lower the readability of the element; Visual allocation efficiency is the proportion of the effective fixation time of the subject in the target area to the total effective fixation time in the target area and the non-target area during the subject retrieval phase, as shown in the following formula: ;in, From the start of the task to the completion of the retrieval task at the initial location of the task target, the participants were aware of the task target's location. The cumulative effective fixation time within the period, This represents the cumulative effective fixation time of the subject in the non-task target area during this period; a larger value indicates higher retrieval efficiency. Visual switch efficiency is the proportion of saccade time in the total time of the task, and the formula is as follows: ; wherein, saccade time in the process of completing the task, the total time of the subject in the task, the larger the value represents the higher the retrieval efficiency; The search path efficiency is the ratio of the length of the shortest path to the actual search path, and the formula is as follows: ; wherein is the shortest path length of the saccade of the subject completing the task, is the actual search path length of the subject completing the task, and the larger the value is, the higher the search efficiency is; S22, for each dimension, the index data is preprocessed, the positive conversion of negative correlation index is executed, the abnormal value is processed and the subjective score is standardized; S23, for each dimension, the median-absolute median difference method is used for processing, the index data is standardized, the statistical benchmark parameter is determined, the standardization boundary is determined, the standardized value is calculated, and the scale of different dimensions is unified.

6. The method of claim 3, wherein the method further comprises: The S3 specifically comprises: S31, first level weighting: The indexes in the three dimensions of element distinguishability, element readability and search efficiency are weighted respectively, so that the weight sum of the indexes in each dimension is 1; Experts are invited to score the importance of each index in the target dimension from 1 to 10, and the expert weight vector is obtained by normalization; Based on the standardized index data, the index proportion, information entropy and entropy weight are calculated, and the index fusion coefficient is adopted The index entropy weight and the index expert weight are weighted and fused, and normalized to ensure that the index weight in each dimension is 1. S32, second level weighting: The weight is calculated for the three dimensions of element distinguishability, element readability and search efficiency, the dimension score of each subject is calculated, the dimension entropy weight is calculated based on the dimension score, experts are invited to score the overall importance of the three dimensions and normalized to obtain the dimension expert weight, and the dimension fusion coefficient The dimension entropy weight and the dimension expert weight are fused, and normalization is performed to ensure that the dimension weight sum is 1. S33, according to the dimension quantitative scores and experience scale data, the availability evaluation result is obtained.

7. A map usability assessment system based on a multi-dimensional quantification model, characterized by: Comprise map management module, map reading experiment module and terminal equipment; The map management module is used for managing the prepared vector map according to the map database established according to the map, facing virtual reality scene interaction, realizing map service publishing, map query analysis service, retrieving appearance information of the gazed element according to the gaze point geographic coordinates provided by the map reading experiment module, and supporting dynamic update of the map data; The map reading experiment module comprises an experiment guiding module and an experiment recording module; The experiment guiding module guides the subject to perform the map cognition experiment based on the experiment scene, judges the gaze position of the subject, sends the gaze position to the map management module and interacts, picks up the map element, and acquires the map element information; The experiment recording module collects the cognition data of the subject in the map reading experiment based on the eye movement tracking technology, including the gaze time stamp, the gaze position, the gaze duration, the saccade time stamp, the saccade duration, the saccade path coordinates, the blink time stamp, the head VR instantaneous angular velocity, the head cumulative rotation angle, the gazed map element ID, and the element appearance information; The terminal device comprises a helmet, a VR handle, an eye movement tracking module, and a motion capture module; The VR helmet and the VR handle are used for scene roaming, information confirmation, and subjective score data collection; The eye movement tracking module is used for acquiring eye movement information; The motion capture module is used for acquiring head movement information.

8. The map usability assessment system based on a multi-dimensional quantization model according to claim 7, characterized in that: After the map reading experiment module completes the user intention judgment, a buffer area query request containing geographic coordinates is sent to the map management module; After receiving the query result data, the gazed element is screened by a gazed element matching confidence judgment method, and the one with the highest confidence is taken as the hit object; The gazed element matching confidence judgment method comprises task object type matching judgment and distance matching judgment; The task object type matching judgment locks the range by matching the current task object type, and filters the objects in the element library that match the type and level according to the current task target; The distance matching judgment verifies the distance between the gaze point and the target element, and if multiple elements meet the condition, the one with the shortest distance is taken as the pickup object.

Images