Augmented Reality Head-Up Display Testing Methods, Systems, Equipment and Media
By acquiring eye-tracking data and region parameters, the system accurately calculates the relationship between the gaze and the display area on the augmented reality head-up display screen, solving the problem of unclear gaze and display area attribution and improving the accuracy of test results and cross-vehicle compatibility.
Patent Information
- Application Number
- CN202611237006.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-08-14
- Publication Date
- 2026-11-10
AI Technical Summary
Existing augmented reality head-up display testing schemes struggle to clearly distinguish the relationship between the line of sight and the display area, resulting in a large number of invalid samples that affect the accuracy of the evaluation results.
By acquiring the driver's eye-tracking data and the regional parameters of the candidate display area, the coordinates of the eye-tracking point on the augmented reality head-up display virtual screen are determined. Based on the eye-tracking point coordinates and regional parameters, the attribution result and duration of the gaze are determined, the target attribution result is generated, and the test result of the augmented reality head-up display scheme is finally determined.
It improves the accuracy of evaluation results for augmented reality head-up display solutions by accurately calculating the spatial transformation between the gaze origin and the direction vector, avoiding boundary misjudgments and inter-frame jumps, and significantly improving the accuracy of test data and cross-vehicle compatibility.
Smart Images

Figure CN123065494A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of augmented reality head-up display technology, and in particular to an augmented reality head-up display testing method, system, device and medium. Background Technology
[0002] Augmented Reality Head-Up Displays (AR-HUDs) project virtual content such as navigation guidance, hazard warnings, lane guidance information, and ahead vehicle signage onto the driver's field of vision, reducing the frequency with which drivers look down at the center console or instrument panel, thus improving driving convenience and safety. During the development, testing, and optimization of AR-HUDs, it is typically necessary to analyze driver attention patterns for different displayed content and areas based on eye-tracking data.
[0003] Existing augmented reality head-up display (ARH) testing schemes mostly follow the approach of fixed two-dimensional Area of Interest (AOI), statistically analyzing gaze distribution, gaze duration, or saccade trajectory. However, this approach struggles to clearly distinguish the relationship between gaze and display area, resulting in a large number of invalid samples being used as test data, which affects the accuracy of subsequent ARH evaluation results. Summary of the Invention
[0004] The purpose of this application is to provide an augmented reality head-up display testing method, system, device, and medium that can solve the problems of traditional methods, such as difficulty in clearly distinguishing the relationship between the line of sight and the display area, the use of a large number of invalid samples for testing, and the impact on the accuracy of augmented reality head-up display scheme evaluation results.
[0005] In a first aspect, embodiments of this application provide an augmented reality head-up display testing method, the method comprising: The driver's eye-tracking data and the region parameters of the candidate display area are obtained; the candidate display area represents the region of each display information in the augmented reality head-up display scheme; the augmented reality head-up display scheme includes at least one candidate display area. The coordinates of the driver's gaze point on the virtual screen of the augmented reality head-up display are determined based on the eye-tracking data. The attribution result and duration of the gaze are determined based on the eye-tracking point coordinates and the region parameters; the attribution result represents the attribution relationship between the eye-tracking point coordinates of the gaze and the candidate display region; the duration of the frames represents the number of frames the gaze maintains the attribution result. The target attribution result of the candidate display area is generated based on the attribution result and the number of consecutive frames. The test results of the augmented reality head-up display scheme are determined based on the target attribution results.
[0006] Optionally, the eye-tracking data includes: the origin of the gaze and the gaze direction vector; Determining the coordinates of the driver's gaze point on the virtual screen of the augmented reality head-up display based on the eye-tracking data includes: The driver's target spatial position inside the vehicle is determined based on the origin of the line of sight. The driver's target gaze orientation in the three-dimensional space of the vehicle is determined based on the gaze direction vector. The intersection point between the line of sight and the virtual image screen is determined based on the target's spatial location and the target's line of sight orientation; Set the intersection point as the coordinates of the eye movement landing point.
[0007] Optionally, the region parameters include the region priority of the candidate display region; Determining the attribution result and duration of the gaze based on the eye-tracking point coordinates and the region parameters includes: The positional relationship between the line of sight and the candidate display area is determined based on the eye movement point coordinates and the region parameters. The attribution of the line of sight is determined based on the positional relationship and the regional priority; The number of frames for which the gaze remains focused on the attribution result is determined based on the duration of the attribution result.
[0008] Optionally, the region parameters further include: the region range and boundary buffer distance of the candidate display region; Determining the positional relationship between the line of sight and the candidate display area based on the eye movement point coordinates and the area parameters includes: When the coordinates of the eye movement landing point are within the area, the positional relationship between the line of sight and the candidate display area is a first positional relationship; If the eye movement point coordinates are not within the area range, the display distance between the eye movement point coordinates and the candidate display area is determined based on the eye movement point coordinates and the area range. When the display distance is less than the boundary buffer distance, the positional relationship between the line of sight and the candidate display area is a second positional relationship; When the display distance is greater than or equal to the boundary buffer distance, the positional relationship between the line of sight and the candidate display area is a third positional relationship.
[0009] Optionally, the attribution results include: a first attribution result, a second attribution result, and a third attribution result; The step of determining the attribution of the line of sight based on the positional relationship and the regional priority includes: When the positional relationship is the first positional relationship, a target candidate display area that meets the preset conditions is determined according to the area priority, and the attribution result of the line of sight to the target candidate display area is the first attribution result; the first attribution result indicates that the eye movement landing point coordinates are in the candidate display area. When the positional relationship is the second positional relationship, the attribution result of the line of sight and the candidate display area is the second attribution result; the second attribution result indicates that the coordinates of the eye movement landing point are within the boundary buffer of the target candidate display area; When the positional relationship is the third positional relationship, the attribution result of the line of sight and the candidate display area is the third attribution result; the third attribution result indicates that the eye movement landing point coordinates do not fall in either of the candidate display areas and the boundary buffer.
[0010] Optionally, generating the target attribution result of the candidate display area based on the attribution result and the number of consecutive frames includes: If the attribution result is the first attribution result and the number of continuous frames is greater than or equal to a preset first threshold, the target attribution result is set as the first target attribution result; If the attribution result is the second attribution result and the number of continuous frames is greater than or equal to a preset second threshold, the target attribution result is set as the second target attribution result; If the attribution result is the third attribution result and the number of continuous frames is greater than or equal to a preset third threshold, the target attribution result is set as the third target attribution result.
[0011] Optionally, determining the test result of the augmented reality head-up display scheme based on the target attribution result includes: The test indicators for each candidate display area are determined based on the target attribution results. The test thresholds for the test indicators are determined based on the use cases of the augmented reality head-up display solution. The test results are determined based on the test metrics and the test thresholds.
[0012] Secondly, embodiments of this application provide an augmented reality head-up display testing system, the system comprising: The data acquisition module is used to acquire the driver's eye-tracking data and the region parameters of the candidate display area; the candidate display area represents the region of each display information in the augmented reality head-up display scheme; the augmented reality head-up display scheme includes at least one candidate display area; An eye-tracking point coordinate determination module is used to determine the eye-tracking point coordinates of the driver's gaze on the virtual image screen of the augmented reality head-up display based on the eye-tracking data. The first result generation module is used to determine the attribution result and the number of frames for the gaze based on the eye movement landing point coordinates and the region parameters; the attribution result represents the attribution relationship between the eye movement landing point coordinates of the gaze and the candidate display region; the number of frames for the gaze maintains the attribution result. The target result generation module is used to generate the target assignment result of the candidate display area based on the assignment result and the number of continuous frames. The test result determination module is used to determine the test result of the augmented reality head-up display scheme based on the target attribution result.
[0013] Thirdly, embodiments of this application provide an electronic device, including: a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the method described above.
[0014] Fourthly, embodiments of this application provide a computer-readable storage medium on which a computer program is stored, which, when executed by a processor, implements the method described above.
[0015] Compared with the prior art, the embodiments of this application have the following advantages: In this embodiment, the driver's eye-tracking data and the region parameters of candidate display areas are obtained. The candidate display areas represent the regions of various display information in the augmented reality head-up display scheme. The augmented reality head-up display scheme includes at least one candidate display area. The eye-tracking data is used to determine the coordinates of the driver's gaze on the virtual screen of the augmented reality head-up display. The gaze assignment result and duration frame count are determined based on the eye-tracking coordinates and region parameters. The assignment result represents the relationship between the gaze's eye-tracking coordinates and the candidate display area. The duration frame count represents the number of frames the gaze maintains the assignment result. A target assignment result for the candidate display area is generated based on the assignment result and duration frame count. The test result of the augmented reality head-up display scheme is determined based on the target assignment result. Using the relationship between the eye-tracking coordinates and the candidate display area as a test indicator improves the accuracy of the augmented reality head-up display scheme evaluation results. Attached Figure Description
[0016] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0017] Figure 1 This is a flowchart illustrating the steps of an augmented reality head-up display testing method provided in an embodiment of this application; Figure 2 This is a schematic diagram of a candidate display area provided in an embodiment of this application; Figure 3 This is a flowchart illustrating the steps for determining target attribution results provided in an embodiment of this application; Figure 4 This is a schematic diagram of the structure of an augmented reality head-up display testing system provided in an embodiment of this application; Figure 5 This is a schematic diagram of an electronic device provided in an embodiment of this application; Figure 6 This is a schematic diagram of a computer-readable storage medium provided in an embodiment of this application. Detailed Implementation
[0018] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.
[0019] The terms "first," "second," etc., used in this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein. Furthermore, in this application, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.
[0020] The following description, in conjunction with the accompanying drawings, details an augmented reality head-up display testing method, system, electronic device, and storage medium provided in this application through specific embodiments and application scenarios.
[0021] Augmented Reality Head-Up Displays (AR-HUDs) project virtual content such as navigation guidance, hazard warnings, lane guidance information, and ahead vehicle signage onto the driver's field of vision, reducing the frequency with which drivers look down at the center console or instrument panel, thus improving driving convenience and safety. During the development, testing, and optimization of AR-HUDs, it is typically necessary to analyze driver attention patterns for different displayed content and areas based on eye-tracking data.
[0022] Existing augmented reality head-up display (ARH) testing schemes mostly follow the approach of fixed two-dimensional Area of Interest (AOI), statistically analyzing gaze distribution, gaze duration, or saccade trajectory. However, this approach struggles to clearly distinguish the relationship between gaze and display area, resulting in a large number of invalid samples being used as test data, which affects the accuracy of subsequent ARH evaluation results.
[0023] Reference Figure 1 The flowchart illustrates the steps of an augmented reality head-up display testing method provided in an embodiment of this application, which may specifically include the following steps: Step 101: Obtain the driver's eye-tracking data and the region parameters of the candidate display area; the candidate display area represents the region of each display information in the augmented reality head-up display scheme; the augmented reality head-up display scheme includes at least one candidate display area; In this embodiment, the driver's eye-tracking data can be detected using a non-contact eye tracker deployed inside the vehicle; the candidate display area is the area in the AR-HUD's virtual screen where information is displayed, such as... Figure 2 The diagram shows a schematic of a candidate display area provided in this application, which includes at least one candidate display area in the virtual image screen of the AR-HUD. This embodiment expands the AR-HUD display object from a single planar display object to multiple candidate display area objects to adapt to different layout schemes and different display tasks.
[0024] Specifically, the position and size of the candidate display areas are different in different AR-HUD display solutions. The candidate display areas and their area parameters can be extracted from the AR-HUD display solution to be tested, providing a basis for judgment in subsequent testing of the AR-HUD display solution.
[0025] Step 102: Determine the coordinates of the driver's gaze point on the virtual image screen of the augmented reality head-up display based on the eye-tracking data; In this embodiment, in order to determine the relationship between eye-tracking data and candidate display areas under a unified coordinate system, it is necessary to transform the eye-tracking data. Based on the eye-tracking data, an invisible geometric line is drawn from the driver's eyes along the line of sight in the virtual three-dimensional in-vehicle space. The intersection of this geometric line and the AR-HUD on the virtual image screen outside the vehicle is the coordinate of the eye-tracking point, which is also the eye-tracking point of the driver's line of sight on the AR-HUD.
[0026] Step 103: Determine the attribution result and duration frame number of the gaze based on the eye movement landing point coordinates and the region parameters; the attribution result represents the attribution relationship between the eye movement landing point coordinates of the gaze and the candidate display region; the duration frame number represents the number of frames the gaze maintains the attribution result. In this embodiment, the attribution result between the eye movement point and the candidate display area is used as the processing object, rather than simply performing statistics on the original eye movement coordinates.
[0027] Specifically, since the AR-HUD display scheme contains at least one candidate display area, it is necessary to determine which candidate display area the driver's gaze falls on at this time, and the number of frames the driver's gaze falls on that candidate display area, based on the eye-tracking point coordinates and area parameters. Based on this, it can be determined whether the driver's gaze falls on the candidate display area in the AR-HUD display scheme, and whether the size of the candidate display area and the spacing between adjacent candidate display areas are reasonable.
[0028] Step 104: Generate the target attribution result of the candidate display area based on the attribution result and the number of continuous frames; After determining the attribution result between eye-tracking data and candidate display areas based on eye-tracking point coordinates and region parameters, this attribution result is a temporary attribution result. Therefore, in order to reduce the impact of single-frame jitter, boundary jitter, and short-term measurement errors on the temporary attribution result, this embodiment performs continuous frame stabilization processing on the temporary attribution result to obtain the final stable attribution result.
[0029] Specifically, the stability of the temporary assignment result is determined by combining the number of frames that maintain the assignment result unchanged from eye-tracking data. The stable assignment result is then used as the final target assignment result, thereby improving the accuracy of subsequent test results.
[0030] Step 105: Determine the test results of the augmented reality head-up display scheme based on the target attribution results.
[0031] AR-HUD display solution testing refers to testing the layout of candidate display areas in the same AR-HUD display solution, or candidate display areas in different AR-HUD display solutions, under preset driving scenarios, preset driving tasks, and preset display content conditions.
[0032] Specifically, the test indicators for the AR-HUD display scheme can be determined based on the target attribution results. The test indicators are then used to measure the impact of the candidate display areas in the AR-HUD display scheme on the driver's visual attention allocation, information recognition efficiency, and road observation behavior.
[0033] In this embodiment, the driver's eye-tracking data and the region parameters of candidate display areas are obtained. The candidate display areas represent the regions of various display information in the augmented reality head-up display scheme. The augmented reality head-up display scheme includes at least one candidate display area. The eye-tracking data is used to determine the coordinates of the driver's gaze on the virtual screen of the augmented reality head-up display. The gaze assignment result and duration frame count are determined based on the eye-tracking coordinates and region parameters. The assignment result represents the relationship between the gaze's eye-tracking coordinates and the candidate display area. The duration frame count represents the number of frames the gaze maintains the assignment result. A target assignment result for the candidate display area is generated based on the assignment result and duration frame count. The test result of the augmented reality head-up display scheme is determined based on the target assignment result. Using the relationship between the eye-tracking coordinates and the candidate display area as a test indicator improves the accuracy of the augmented reality head-up display scheme evaluation results.
[0034] In one embodiment of this application, the eye-tracking data includes: a gaze origin and a gaze direction vector; Determining the coordinates of the driver's gaze point on the virtual screen of the augmented reality head-up display based on the eye-tracking data includes: The driver's target spatial position inside the vehicle is determined based on the origin of the line of sight. The driver's target gaze orientation in the three-dimensional space of the vehicle is determined based on the gaze direction vector. The intersection point between the line of sight and the virtual image screen is determined based on the target's spatial location and the target's line of sight orientation; Set the intersection point as the coordinates of the eye movement landing point.
[0035] In this embodiment of the application, eye-tracking data includes the origin of the gaze. and line-of-sight vector Then, calculate the target space position of the line of sight origin inside the vehicle according to the following formula: Formula 1:
[0036] in, The target spatial position is the absolute spatial position of the driver's eyes inside the vehicle. This is a preset installation rotation matrix; The origin of the line of sight is the position of the eye relative to the camera as measured by the non-contact eye tracker. This is the preset installation translation vector.
[0037] Then, according to the following formula two, the line-of-sight vector is... Translated into the driver's target line of sight in the three-dimensional space of the vehicle: Formula 2:
[0038] in, The target line of sight, that is, the absolute line of sight in the three-dimensional space of the vehicle body; This is a preset installation rotation matrix; The origin of the line of sight is the position of the eye relative to the camera as measured by the non-contact eye tracker. This is the vector representing the line of sight.
[0039] In actual testing, because non-contact eye trackers are fixedly installed on fixed components inside the vehicle (for example, they can be installed on the top surface of the steering column or dashboard), their device coordinate system... Mapping coordinate system with target The relative spatial geometric relationship between them is a static constant. Before testing, the installation deflection angle and physical distance of the non-contact eye tracker relative to the vehicle coordinate system were measured in one go using the in-vehicle spatial calibration method, and the static installation rotation matrix was calculated. With static installation translation vector During vehicle operation and testing, the installation rotation matrix and installation translation vector can be directly invoked to achieve real-time conversion of eye-tracking data, avoiding secondary errors caused by dynamic head posture tracking and improving the overall mapping accuracy of the system.
[0040] After calculating the target's spatial location and the target's line of sight orientation, the line of sight ray equation can be constructed, as shown in Formula 3 below: Formula 3:
[0041] in, The coordinates of the eye movement point; The target spatial location; is the scaling factor, representing the length the beam extends forward; The direction of the line of sight to the target.
[0042] The line-of-sight ray equation characterizes the spatial position of the target from the eye. Start, following the direction of your line of sight A beam of flashlight light is shone outwards, varying with the scaling factor. Starting from 0 and increasing, this equation yields every spatial point G along the path of the light beam.
[0043] Since the virtual screen of the AR-HUD is located at a preset distance outside the vehicle (for example, the preset distance could be 7 meters), the equation of this plane is defined as follows: (Where, n is the normal vector perpendicular to the virtual image screen, which determines the orientation of the virtual image screen; d is the intercept, which determines how far away the virtual image screen is).
[0044] To find the exact point on the virtual screen where the line of sight ray lies, we need to substitute the equation of the line of sight ray into the equation of the plane and solve for that specific length lambda(λ). The line of sight ray is shown in Formula 4 below: Formula 4:
[0045] in, It is the normal vector; The target spatial location; This is the scaling factor; The direction of the line of sight to the target.
[0046] Expanding and rearranging the terms, we obtain the following formula: Formula 5:
[0047] in, It is the normal vector; The target spatial location; This is the scaling factor; The direction of the line of sight to the target.
[0048] Substituting this calculated length lambda back into the line-of-sight ray equation, we obtain the sixth formula for calculating the final intersection point coordinates: Formula Six:
[0049] in, The coordinates of the eye movement point; The target spatial location; is the scaling factor, representing the length the beam extends forward; The direction of the target's line of sight; It is the normal vector; The target spatial location; The direction of the line of sight to the target.
[0050] The above formula can be used to directly calculate how far the line of sight needs to travel forward to collide with the virtual screen, that is, the intersection of the line of sight and the virtual screen. Once the distance is known, the coordinates G(x,y,z) of the eye movement point that will collide with the screen can be accurately located.
[0051] This embodiment constructs a dynamic gaze ray equation by accurately calculating the spatial transformation between the gaze origin and the direction vector, and combines this with virtual image screen parameters to accurately calculate the coordinates of the eye-tracking point. This method solves the spatial mapping error problem caused by differences in device installation in traditional solutions, avoids boundary misjudgment and inter-frame jumps, effectively distinguishes sample attribution, significantly improves cross-vehicle compatibility and data accuracy of AR-HUD testing, and provides a reliable basis for display optimization.
[0052] In one embodiment of this application, the region parameter includes the region priority of the candidate display region; Determining the attribution result and duration of the gaze based on the eye-tracking point coordinates and the region parameters includes: The positional relationship between the line of sight and the candidate display area is determined based on the eye movement point coordinates and the region parameters. The attribution of the line of sight is determined based on the positional relationship and the regional priority; The number of frames for which the gaze remains focused on the attribution result is determined based on the duration of the attribution result.
[0053] In this embodiment, the positional relationship between the driver's gaze and each candidate display area in the AR-HUD display scheme can be determined based on the calculated eye-tracking point coordinates and the area parameters of the candidate display areas. Since there are multiple candidate display areas in the AR-HUD display scheme, the area priority of each candidate display area can be used to determine which candidate display area the driver's gaze belongs to, thus avoiding assigning the gaze to other candidate display areas.
[0054] Furthermore, the duration of the attribution result is determined based on the duration for which the gaze belongs to a candidate display area.
[0055] This embodiment accurately determines the gaze assignment by analyzing the regional priority of candidate display areas and the dynamic position of eye-tracking point coordinates, avoiding misjudgments or inter-frame jumps caused by blurred boundaries. The priority mechanism effectively distinguishes between primary and secondary display areas, and combined with continuous frame count statistics to filter out brief fluctuations, it significantly improves the accuracy of sample classification, solves the problem of confusion between valid assigned samples and boundary samples in traditional methods, and provides a highly reliable basis for AR-HUD display layout optimization.
[0056] In one embodiment of this application, the region parameters further include: the region range and boundary buffer distance of the candidate display region; Determining the positional relationship between the line of sight and the candidate display area based on the eye movement point coordinates and the area parameters includes: When the coordinates of the eye movement landing point are within the area, the positional relationship between the line of sight and the candidate display area is a first positional relationship; If the eye movement point coordinates are not within the area range, the display distance between the eye movement point coordinates and the candidate display area is determined based on the eye movement point coordinates and the area range. When the display distance is less than the boundary buffer distance, the positional relationship between the line of sight and the candidate display area is a second positional relationship; When the display distance is greater than or equal to the boundary buffer distance, the positional relationship between the line of sight and the candidate display area is a third positional relationship.
[0057] In this embodiment, the region parameters include the area range of the candidate display region and the boundary buffer distance. For example... Figure 2 As shown, the specific size of the candidate display area can be determined by the area range of the candidate display area, and the boundary buffer zone around the candidate display area can be determined by the boundary buffer distance.
[0058] When the coordinates of the eye movement point are within the candidate display area, that is, when the point of gaze is within the candidate display area, the positional relationship between the gaze and the candidate display area can be set as the first positional relationship. When the eye-tracking point coordinates are not within the candidate display area, the display distance between the eye-tracking point coordinates and the boundary coordinates of the candidate display area is calculated based on the eye-tracking point coordinates and the boundary coordinates of the area. When the display distance is less than the boundary buffer distance, i.e., the eye-tracking point is within the boundary buffer zone, the positional relationship between the eye-tracking point and the candidate display area is set to the second positional relationship. When the display distance is greater than or equal to the boundary buffer distance, i.e., the eye-tracking point does not fall within the boundary buffer zone of any candidate display area, the positional relationship between the eye-tracking point and the candidate display area is set to the third positional relationship.
[0059] Furthermore, after determining the positional relationship between the line of sight and the candidate display area, the assignment result needs to be determined according to the following steps: Step S1: When the positional relationship is the first positional relationship, a target candidate display area that meets the preset conditions is determined according to the area priority. The attribution result of the gaze and the target candidate display area is the first attribution result. The first attribution result indicates that the eye movement point coordinates are in the candidate display area. Step S2: When the positional relationship is the second positional relationship, the attribution result of the line of sight and the candidate display area is the second attribution result; the second attribution result indicates that the coordinates of the eye movement landing point are within the boundary buffer of the target candidate display area; Step S3: When the positional relationship is the third positional relationship, the attribution result of the line of sight and the candidate display area is the third attribution result; the third attribution result indicates that the eye movement landing point coordinates do not land in any of the candidate display areas and the boundary buffer.
[0060] Specifically, when the positional relationship is the first positional relationship, that is, when the eye movement point coordinate G is located within a certain candidate display area, the candidate display area is directly determined as the temporary assigned area of the line of sight, which is the first assigned result. Furthermore, if the eye movement point coordinate G is located within multiple candidate display areas, the candidate display area with the highest regional priority is selected as the target candidate display area according to the regional priority of each candidate display area. When the priorities are the same, the candidate display area with the smaller distance from the eye movement point coordinate G to the regional center point of the candidate display area can be further selected as the target candidate display area, and the target candidate display area is determined as the temporary assigned area of the line of sight. If the eye-tracking point coordinate G does not fall within any candidate display area, it is determined whether it is located within the boundary buffer zone (i.e., boundary buffer zone) of a candidate display area. If it falls within one or more boundary buffer zones, the assignment result of the candidate display area and the line of sight is marked as the second assignment result, i.e., the boundary candidate sample. If the eye movement point coordinates do not fall within any of the candidate display areas and the boundary buffer zone, then the assignment result of the selected display area and the line of sight is marked as the third assignment result, that is, no assigned sample.
[0061] This embodiment accurately determines the gaze assignment by analyzing the regional priority of candidate display areas and the dynamic position of eye-tracking point coordinates, avoiding misjudgments or inter-frame jumps caused by blurred boundaries. The priority mechanism effectively distinguishes between primary and secondary display areas, significantly improving sample classification accuracy and solving the problem of confusion between valid and boundary samples in traditional methods, providing a highly reliable basis for AR-HUD display layout optimization.
[0062] In one embodiment of this application, generating the target attribution result of the candidate display area based on the attribution result and the number of consecutive frames includes: If the attribution result is the first attribution result and the number of continuous frames is greater than or equal to a preset first threshold, the target attribution result is set as the first target attribution result; If the attribution result is the second attribution result and the number of continuous frames is greater than or equal to a preset second threshold, the target attribution result is set as the second target attribution result; If the attribution result is the third attribution result and the number of continuous frames is greater than or equal to a preset third threshold, the target attribution result is set as the third target attribution result.
[0063] In this embodiment, in order to reduce the impact of single-frame jitter, boundary jitter and short-term measurement errors on the output results, it is necessary to perform continuous frame stabilization processing on the temporary assignment results to obtain stable target assignment results.
[0064] Specifically, such as Figure 3 As shown, a flowchart of the determination steps for target attribution results is shown. If the attribution result is the first attribution result, and the number of times the same candidate display area appears in the most recent N frames is greater than or equal to the preset first threshold K, then the attribution relationship between the candidate display area and the line of sight is set as the first target attribution result. The first target attribution result indicates that the current line of sight can stably fall on the candidate display area, and the candidate display area belongs to the stable attribution area. If the attribution result is the second attribution result (that is, the eye movement point coordinates fall into the boundary buffer zone), and the number of consecutive frames of the second attribution result is greater than or equal to the preset second threshold M, that is, if no clear regional advantage is formed within the subsequent M consecutive frames, the attribution relationship between the candidate display area and the line of sight is set as the second target attribution result. The second target attribution result indicates that the current line of sight only falls into the boundary buffer zone and no clear regional advantage is formed between it and the candidate display area. The current line of sight belongs to the boundary state. If the attribution result is the third attribution result (that is, the eye movement point coordinates do not fall into any boundary buffer zone), and the number of consecutive frames of the third attribution result is greater than or equal to the preset third threshold B, that is, B consecutive frames do not fall into any boundary buffer zone, the attribution relationship between the selected display area and the line of sight is set as the third target attribution result. The third target attribution result indicates that the current line of sight does not fall into any candidate display area or the boundary buffer zone of any candidate display area, and the current line of sight is in an unattributed state. Optionally, when switching from the second target assignment result to the first target assignment result, the new candidate display area is required to meet the threshold K in the most recent N frames, or to appear continuously for no less than M frames, in order to form a hysteresis switching condition. For example, N=5, K=4, M=2, B=3 can be set.
[0065] Furthermore, after determining the first target assignment result, if the line of sight leaves the candidate display area and fluctuates in the boundary buffer zone of the candidate display area, the target assignment result is updated to the second target assignment result; if consecutive B frames do not fall into any boundary buffer zone, the target assignment result is updated to the third target assignment result.
[0066] After determining the second target assignment result, if the number of times the target falls into the candidate display area in the most recent N frames meets the threshold K, the target assignment result is updated to the first target assignment result; if B consecutive frames leave the boundary buffer zone and do not fall into any boundary buffer zone, the target assignment result is updated to the third target assignment result.
[0067] After determining the third target as the target, if the line of sight and the candidate display area meet the preset stability conditions (for example, it can be set to the number of frames in the most recent N frames where the line of sight falls in the same candidate display area is not less than M frames), then the target as the target is updated to the first target as the target.
[0068] By setting dynamic thresholds and a hysteresis switching mechanism, this approach effectively addresses the issues of inter-frame jumps and unstable attribution results in AR-HUD testing. A continuous frame rate threshold filters out brief fluctuations, distinguishing the relationship between gaze and candidate display areas. Hysteresis conditions prevent frequent attribution switching caused by minor jitter, ensuring classification stability. Dynamically updated rules respond in real-time to gaze changes, balancing sensitivity and reliability. This solution significantly improves the noise resistance and accuracy of sample classification, providing a highly reliable data foundation for quantitatively evaluating driver attention to each display area and optimizing AR-HUD layout. In particular, it solves the misjudgment problem caused by brief gazes or boundary jitter in traditional methods, enhancing the engineering practical value of the test results.
[0069] In one embodiment of this application, determining the test result of the augmented reality head-up display scheme based on the target attribution result includes: The test indicators for each candidate display area are determined based on the target attribution results. The test thresholds for the test indicators are determined based on the use cases of the augmented reality head-up display solution. The test results are determined based on the test metrics and the test thresholds.
[0070] After obtaining the target attribution results of different eye-tracking point coordinates and candidate display areas, the test indicators of each candidate display area can be analyzed. Then, the corresponding test thresholds are determined according to the specific use case of the AR-HUD display solution, and the final test results are determined based on the relationship between the test indicators and the test thresholds.
[0071] Specifically, during the test, the AR-HUD display content is arranged in one or more candidate display areas, eye movement data of the driver is collected during the driving task, and the coordinates of the eye movement landing point are calculated according to the formula. The eye movement landing point in the eye movement data is mapped to each candidate display area to obtain the target attribution relationship between the eye movement landing point and the candidate display area.
[0072] In practical implementation, the target attribution relationship can determine the corresponding test evaluation indicators for each candidate display area. These indicators can include: attention occupancy index, information discovery efficiency index, visual interference index, and road observation impact index. Specifically, the attention occupancy index characterizes the degree to which a driver's visual attention is occupied by a particular candidate display area; the information discovery efficiency index characterizes the speed at which the driver discovers the information displayed within that candidate display area; the visual interference index characterizes whether the candidate display area causes the driver to frequently shift their gaze or linger for extended periods; and the road observation impact index characterizes the time and ease with which the driver returns to the target area on the road ahead from the candidate display area.
[0073] In one example, the attention occupancy index can be determined based on the ratio between the cumulative gaze duration of the driver in a candidate display area belonging to the first target attribution result and the test time window; the information discovery efficiency index can be determined based on the time interval between the appearance of the displayed content and the driver's first gaze at the candidate display area (i.e., the appearance of the user's line of sight and the candidate display area belonging to the first target attribution result or the second target attribution result); the visual interference index can be determined based on the number of times the driver's gaze enters the candidate display area, the number of times it leaves the candidate display area, and the average single gaze duration, wherein the number of times the driver's gaze enters the candidate display area and the number of times it leaves the candidate display area can be determined based on the number of times the first target attribution result and the second target attribution result switch to the third target attribution result; and the road observation impact index can be determined based on the time it takes for the driver's gaze to return to the target area on the road ahead after leaving the candidate display area.
[0074] Furthermore, different AR-HUD display schemes can be compared based on the aforementioned test evaluation indicators. If the initial gaze time for a candidate display area is less than the gaze threshold, the cumulative gaze duration falls within a preset range, and the time for the gaze to return to the target road area ahead is less than the regression threshold, then the candidate display area can be determined to have good information discoverability and low road observation interference. If the cumulative gaze duration for a candidate display area is greater than the gaze threshold, the number of gaze entries is greater than the switching threshold, and the road gaze return time is greater than the regression threshold, then the candidate display area can be determined to occupy a high amount of the driver's visual attention and is not suitable for carrying non-safety-critical information.
[0075] Therefore, this implementation can transform raw eye-tracking data into regional evaluation results for AR-HUD display solution testing, enabling testers to judge the display adaptability of different candidate display areas in different driving scenarios based on the driver's real visual behavior, thereby providing a test basis for AR-HUD layout design, display area selection and display information allocation.
[0076] This application embodiment acquires the driver's eye-tracking data and the region parameters of candidate display areas. The candidate display areas represent the regions of various display information in the augmented reality head-up display (ARH) scheme. The ARH scheme includes at least one candidate display area. Based on the eye-tracking data, the coordinates of the driver's gaze on the virtual screen of the ARH display are determined. Based on the eye-tracking coordinates and region parameters, the gaze assignment result and duration frame count are determined. The assignment result represents the relationship between the gaze's eye-tracking coordinates and the candidate display area; the duration frame count represents the number of frames the gaze maintains the assignment result. Based on the assignment result and duration frame count, a target assignment result for the candidate display area is generated. Based on the target assignment result, the test result of the ARH scheme is determined. Using the relationship between the eye-tracking coordinates and the candidate display area as a test indicator improves the accuracy of the ARH scheme evaluation results.
[0077] It should be noted that, for the sake of simplicity, the method embodiments are all described as a series of actions. However, those skilled in the art should understand that the embodiments of this application are not limited to the described order of actions, because according to the embodiments of this application, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also understand that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of this application.
[0078] Reference Figure 2 The diagram illustrates the structure of an augmented reality head-up display testing system provided in an embodiment of this application, which may specifically include the following modules: The data acquisition module 201 is used to acquire the driver's eye movement data and the region parameters of the candidate display area; the candidate display area represents the region of each display information in the augmented reality head-up display scheme; the augmented reality head-up display scheme includes at least one candidate display area; The eye-tracking point coordinate determination module 202 is used to determine the eye-tracking point coordinates of the driver's gaze on the virtual image screen of the augmented reality head-up display based on the eye-tracking data. The first result generation module 203 is used to determine the attribution result and the number of sustained frames of the gaze based on the eye movement landing point coordinates and the region parameters; the attribution result represents the attribution relationship between the eye movement landing point coordinates of the gaze and the candidate display region; the number of sustained frames represents the number of frames the gaze maintains the attribution result. Target result generation module 204 is used to generate target assignment results for the candidate display area based on the assignment results and the number of continuous frames; The test result determination module 205 is used to determine the test result of the augmented reality head-up display scheme based on the target attribution result.
[0079] In one embodiment of this application, the eye-tracking data includes: a gaze origin and a gaze direction vector; the eye-tracking point coordinate determination module 202 includes: The target spatial location determination submodule is used to determine the driver's target spatial location inside the vehicle based on the line of sight origin. The target gaze orientation determination submodule is used to determine the driver's target gaze orientation in the three-dimensional space of the vehicle body based on the gaze direction vector. The intersection point determination submodule determines the intersection point between the line of sight and the virtual image screen based on the target's spatial location and the target's line of sight orientation. The eye-tracking point coordinate generation submodule is used to set the intersection point as the eye-tracking point coordinate.
[0080] In one embodiment of this application, the region parameter includes the region priority of the candidate display region; the first result generation module 203 includes: The positional relationship determination submodule is used to determine the positional relationship between the line of sight and the candidate display area based on the eye movement landing point coordinates and the area parameters; The attribution result determination submodule is used to determine the attribution result of the line of sight based on the positional relationship and the area priority; The continuous frame count determination submodule is used to determine the continuous frame count for which the line of sight maintains the attribution result based on the duration of the attribution result.
[0081] In one embodiment of this application, the region parameters further include: the region range and boundary buffer distance of the candidate display region; the positional relationship determination submodule includes: The first positional relationship determination unit is used to determine the positional relationship between the line of sight and the candidate display area as a first positional relationship when the eye movement landing point coordinates are within the area range; The display distance determination unit is used to determine the display distance between the eye movement landing point coordinates and the candidate display area based on the eye movement landing point coordinates and the area range when the eye movement landing point coordinates are not within the area range. The second positional relationship determination unit is used to determine the positional relationship between the line of sight and the candidate display area as a second positional relationship when the display distance is less than the boundary buffer distance. The third positional relationship determination unit is used to determine the positional relationship between the line of sight and the candidate display area as a third positional relationship when the display distance is greater than or equal to the boundary buffer distance.
[0082] In one embodiment of this application, the attribution result includes: a first attribution result, a second attribution result, and a third attribution result; the attribution result determination submodule includes: The first attribution result determination unit is used to determine a target candidate display area that meets preset conditions based on the region priority when the positional relationship is the first positional relationship. The attribution result of the line of sight and the target candidate display area is the first attribution result. The first attribution result indicates that the eye movement landing point coordinates are in the candidate display area. The second attribution result determination unit is configured to determine the attribution result of the line of sight and the candidate display area as the second attribution result when the positional relationship is the second positional relationship; the second attribution result indicates that the eye movement landing point coordinates are in the boundary buffer of the target candidate display area. The third attribution result determination unit is used to determine the attribution result of the line of sight and the candidate display area as the third attribution result when the positional relationship is the third positional relationship; the third attribution result indicates that the eye movement landing point coordinates do not land in either of the candidate display areas and the boundary buffer.
[0083] In one embodiment of this application, the target result generation module 204 includes: The first target attribution result determination submodule is configured to set the target attribution result as the first target attribution result when the attribution result is the first attribution result and the number of consecutive frames is greater than or equal to a preset first threshold. The second target attribution result determination submodule is used to set the target attribution result as the second target attribution result when the attribution result is the second attribution result and the number of continuous frames is greater than or equal to a preset second threshold. The third target attribution result determination submodule is used to set the target attribution result as the third target attribution result when the attribution result is the third attribution result and the number of continuous frames is greater than or equal to a preset third threshold.
[0084] In one embodiment of this application, the test result determination module 205 includes: The test index determination submodule is used to determine the test indexes for each of the candidate display areas based on the target attribution results. The test threshold determination submodule is used to determine the test threshold of the test index based on the use scenario of the augmented reality head-up display solution; The test result generation submodule is used to determine the test result based on the test indicators and the test threshold.
[0085] As the system implementation is basically similar to the method implementation, it is described in a relatively simple way. For relevant details, please refer to the description of the method implementation.
[0086] like Figure 5 As shown, in another embodiment provided in this application, an electronic device 500 is also provided, including a memory 510 and a processor 520. The memory 510 and the processor 520 are connected via a bus for communication. The memory 510 stores a computer program, which can run on the processor 520 to implement the above steps.
[0087] like Figure 6 As shown, in another embodiment provided in this application, a computer-readable storage medium 601 is also provided, which stores a computer program that implements the methods described in the above embodiments when executed by a processor.
[0088] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, embodiments of this application can take the form of entirely hardware embodiments, entirely software embodiments, or embodiments combining software and hardware aspects. Furthermore, embodiments of this application can take the form of computer program products implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0089] Although preferred embodiments of the present invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of the embodiments of the present invention.
[0090] Those skilled in the art will understand that embodiments of this application can be provided as methods, apparatus, or computer program products. Therefore, embodiments of this application can take the form of entirely hardware embodiments, entirely software embodiments, or embodiments combining software and hardware aspects. Furthermore, embodiments of this application can take the form of computer program products implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0091] This application describes embodiments with reference to flowchart illustrations and / or block diagrams of methods, terminal devices (systems), and computer program products according to embodiments of this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing terminal device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal device, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.
[0092] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing terminal device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.
[0093] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal equipment, causing a series of operational steps to be performed on the computer or other programmable terminal equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable terminal equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.
[0094] Although preferred embodiments of the present application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of the embodiments of the present application.
[0095] It should also be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal device. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or terminal device that includes said element.
[0096] The above provides a detailed description of an augmented reality head-up display testing method provided by the present invention. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.
Claims
1. A method for testing augmented reality head-up displays, characterized in that, The method includes: The driver's eye-tracking data and the region parameters of the candidate display area are obtained; the candidate display area represents the region of each display information in the augmented reality head-up display scheme; the augmented reality head-up display scheme includes at least one candidate display area. The coordinates of the driver's gaze point on the virtual screen of the augmented reality head-up display are determined based on the eye-tracking data. The attribution result and duration of the gaze are determined based on the eye-tracking point coordinates and the region parameters; the attribution result represents the attribution relationship between the eye-tracking point coordinates of the gaze and the candidate display region; the duration of the frames represents the number of frames the gaze maintains the attribution result. The target attribution result of the candidate display area is generated based on the attribution result and the number of consecutive frames. The test results of the augmented reality head-up display scheme are determined based on the target attribution results.
2. The method according to claim 1, characterized in that, The eye-tracking data includes: the origin of the gaze and the gaze direction vector; Determining the coordinates of the driver's gaze point on the virtual screen of the augmented reality head-up display based on the eye-tracking data includes: The driver's target spatial position inside the vehicle is determined based on the origin of the line of sight. The driver's target gaze orientation in the three-dimensional space of the vehicle is determined based on the gaze direction vector. The intersection point between the line of sight and the virtual image screen is determined based on the target's spatial location and the target's line of sight orientation; Set the intersection point as the coordinates of the eye movement landing point.
3. The method according to claim 1, characterized in that, The region parameters include the region priority of the candidate display region; The step of determining the attribution result and duration of the gaze based on the eye-tracking point coordinates and the region parameters includes: The positional relationship between the line of sight and the candidate display area is determined based on the eye movement point coordinates and the region parameters. The attribution of the line of sight is determined based on the positional relationship and the regional priority; The number of frames for which the gaze remains focused on the attribution result is determined based on the duration of the attribution result.
4. The method according to claim 3, characterized in that, The region parameters also include: the region range and boundary buffer distance of the candidate display region; Determining the positional relationship between the line of sight and the candidate display area based on the eye movement point coordinates and the area parameters includes: When the coordinates of the eye movement landing point are within the area, the positional relationship between the line of sight and the candidate display area is a first positional relationship; If the eye movement point coordinates are not within the area range, the display distance between the eye movement point coordinates and the candidate display area is determined based on the eye movement point coordinates and the area range. When the display distance is less than the boundary buffer distance, the positional relationship between the line of sight and the candidate display area is a second positional relationship; When the display distance is greater than or equal to the boundary buffer distance, the positional relationship between the line of sight and the candidate display area is a third positional relationship.
5. The method according to claim 4, characterized in that, The attribution results include: a first attribution result, a second attribution result, and a third attribution result; The step of determining the attribution of the line of sight based on the positional relationship and the regional priority includes: When the positional relationship is the first positional relationship, a target candidate display area that meets the preset conditions is determined according to the area priority, and the attribution result of the gaze and the target candidate display area is the first attribution result; the first attribution result indicates that the eye movement landing point coordinates are in the target candidate display area. When the positional relationship is the second positional relationship, the attribution result of the line of sight and the candidate display area is the second attribution result; the second attribution result indicates that the coordinates of the eye movement landing point are within the boundary buffer of the candidate display area; When the positional relationship is the third positional relationship, the attribution result of the line of sight and the candidate display area is the third attribution result; the third attribution result indicates that the eye movement landing point coordinates do not fall in either of the candidate display areas and the boundary buffer.
6. The method according to claim 5, characterized in that, The step of generating the target attribution result of the candidate display area based on the attribution result and the number of consecutive frames includes: If the attribution result is the first attribution result and the number of continuous frames is greater than or equal to a preset first threshold, the target attribution result is set as the first target attribution result; If the attribution result is the second attribution result and the number of continuous frames is greater than or equal to a preset second threshold, the target attribution result is set as the second target attribution result; If the attribution result is the third attribution result and the number of continuous frames is greater than or equal to a preset third threshold, the target attribution result is set as the third target attribution result.
7. The method according to claim 1, characterized in that, The step of determining the test results of the augmented reality head-up display scheme based on the target attribution result includes: The test indicators for each candidate display area are determined based on the target attribution results. The test thresholds for the test indicators are determined based on the use cases of the augmented reality head-up display solution. The test results are determined based on the test metrics and the test thresholds.
8. An augmented reality head-up display testing system, characterized in that, The system includes: The data acquisition module is used to acquire the driver's eye-tracking data and the region parameters of the candidate display area; the candidate display area represents the region of each display information in the augmented reality head-up display scheme; the augmented reality head-up display scheme includes at least one candidate display area; An eye-tracking point coordinate determination module is used to determine the eye-tracking point coordinates of the driver's gaze on the virtual image screen of the augmented reality head-up display based on the eye-tracking data. The first result generation module is used to determine the attribution result and the number of frames for the gaze based on the eye movement landing point coordinates and the region parameters; the attribution result represents the attribution relationship between the eye movement landing point coordinates of the gaze and the candidate display region; the number of frames for the gaze maintains the attribution result. The target result generation module is used to generate the target assignment result of the candidate display area based on the assignment result and the number of continuous frames. The test result determination module is used to determine the test result of the augmented reality head-up display scheme based on the target attribution result.
9. An electronic device, characterized in that, include: A memory, a processor, and a computer program stored on the memory, wherein the processor executes the computer program to implement the method of any one of claims 1-7.
10. A computer-readable storage medium, characterized in that, A computer program is stored on the computer-readable storage medium, which, when executed by a processor, implements the method as described in any one of claims 1-7.
