Vehicle instrument information prioritization method and apparatus based on eye movement trajectories

Abstract

The application discloses a vehicle instrument information priority sequencing method and device based on eye movement track, comprising: acquiring vehicle state data, external environment data and eye movement track data of a target driver of a target vehicle; determining a safety requirement level of each to-be-displayed instrument information according to the vehicle state data, determining a scene requirement level of each to-be-displayed instrument information according to the external environment data, and determining an intention requirement level of each to-be-displayed instrument information according to the eye movement track data; determining an information display priority of each to-be-displayed instrument information according to the safety requirement level, the scene requirement level and the intention requirement level; and dividing the to-be-displayed instrument information into core layer information, auxiliary layer information and redundant layer information according to the information display priority, and performing hierarchical display. The application realizes priority display of instrument information with high importance and high requirement, improves the efficiency and timeliness of information acquisition of a driver, and can be widely applied to the technical field of intelligent cockpits.

Description

Technical Field

[0001] This invention relates to the field of smart cockpit technology, and in particular to a method and apparatus for prioritizing vehicle instrument information based on eye-tracking trajectories. Background Technology

[0002] Current vehicle instrument panel information sorting technologies can be mainly divided into three categories: The first category is a fixed priority scheme, such as the instrument panel system of traditional fuel vehicles, which displays information in a fixed order of "vehicle speed-RPM-fuel level" without considering driver needs and scenario differences; the second category is a simple scenario adaptation scheme, such as the Tesla Model 3 instrument panel system, which only switches the information display combination according to highway / city scenarios, but does not take into account driver behavior characteristics; the third category is the latest publicly available technology, such as the smart instrument panel system of NIO ET9, which introduces statistics on the duration of driver gaze, but uses a traditional statistical model, does not capture the temporal characteristics of eye movement trajectory and its correlation with intent, and does not integrate dynamic data of the external environment.

[0003] In summary, the existing technology has the following drawbacks: 1) Ignoring the intentional correlation of eye movement trajectory: Relying solely on single indicators such as the duration of gaze fixation without analyzing the temporal characteristics of eye movement trajectory such as movement path and focusing sequence, it is impossible to accurately determine the driver's information needs; 2) Lack of multi-dimensional data fusion: The ranking results are not in line with driving safety requirements because they do not combine real-time vehicle status (such as abnormal tire pressure, insufficient range) with external environmental data (such as sudden road conditions, weather changes). 3) Coarse granularity of scene adaptation: It only simply divides the scene into highway / city scenes, without covering subdivided scenes such as congestion, emergency, and rural roads, and the sorting logic has poor universality; 4) Lagging priority adjustment: Traditional models cannot respond to changes in eye movement trajectory and sudden environmental events in real time, and the information update delay is long, which can easily lead to the loss of key information.

[0004] The aforementioned problems affect the efficiency and timeliness of obtaining driver information and urgently need to be addressed. Summary of the Invention

[0005] The purpose of this invention is to at least partially solve one of the technical problems existing in the prior art.

[0006] Therefore, one objective of this invention is to provide a method for prioritizing vehicle instrument information based on eye-tracking trajectories, which improves the efficiency and timeliness of driver information acquisition.

[0007] Another objective of this invention is to provide a vehicle instrument information priority sorting device based on eye-tracking trajectories.

[0008] To achieve the above-mentioned technical objectives, the technical solutions adopted in the embodiments of the present invention include: On one hand, embodiments of the present invention provide a method for prioritizing vehicle instrument information based on eye-tracking trajectories, comprising the following steps: Acquire vehicle status data, external environment data, and eye movement trajectory data of the target driver; The safety requirement level of each instrument information to be displayed is determined based on the vehicle status data, the scene requirement level of each instrument information to be displayed is determined based on the external environment data, and the intent requirement level of each instrument information to be displayed is determined based on the eye movement trajectory data. The information display priority of each instrument information to be displayed is determined based on the security requirement level, the scenario requirement level, and the intent requirement level. Based on the information display priority, the instrument information to be displayed is divided into core layer information, auxiliary layer information, and redundant layer information, and the core layer information, the auxiliary layer information, and the redundant layer information are displayed in a hierarchical manner.

[0009] Furthermore, in one embodiment of the present invention, the vehicle status data includes vehicle speed, tire pressure, remaining range, and driving assistance information. The step of determining the safety requirement level of each instrument information to be displayed based on the vehicle status data specifically includes: The vehicle status data is input into a pre-trained vehicle fault prediction model to obtain the vehicle fault prediction result. Based on the vehicle fault prediction results, the corresponding information display safety classification rules are matched in the preset personalized configuration library; The corresponding security requirement level is obtained by matching the information type of each instrument to be displayed with the information display security classification rules.

[0010] Furthermore, in one embodiment of the present invention, the external environment data includes road image data, radar detection data, and environmental weather conditions. The step of determining the scene requirement level for each of the instrument information to be displayed based on the external environment data specifically includes: The current road type is identified based on the road image data, the distance to obstacles is identified based on the radar detection data, and the current visibility is determined based on the environmental weather conditions. The current driving scenario is determined based on the current road type, the distance to the obstacle, and the current visibility. Based on the current driving scenario, the corresponding information display scenario classification rules are obtained by matching them in the preset personalized configuration library; Based on the information type of each instrument information to be displayed, the corresponding scenario requirement level is obtained by matching it with the information display scenario classification rules.

[0011] Furthermore, in one embodiment of the present invention, determining the intent demand level of each of the instrument information to be displayed based on the eye-tracking trajectory data specifically includes: Based on the eye movement trajectory data, the target driver's gaze point distribution, gaze duration, saccade path, saccade speed, and focusing frequency are extracted to obtain eye movement spatiotemporal feature data; The eye-tracking spatiotemporal feature data is input into a pre-trained driver intention prediction model to obtain the target driver's information acquisition intention. Based on the information acquisition intent, the corresponding information display intent classification rules are obtained by matching in the preset personalized configuration library; Based on the information type of each instrument information to be displayed, the corresponding intent requirement level is obtained by matching it with the information display intent classification rule.

[0012] Furthermore, in one embodiment of the present invention, the driver intention prediction model is trained through the following steps: Obtain eye movement trajectory samples of the test driver, and extract the gaze point distribution, gaze duration, saccade path, saccade speed and focusing frequency of the test driver based on the eye movement trajectory samples to obtain eye movement spatiotemporal feature samples; The information acquisition intent labels corresponding to the eye movement spatiotemporal feature samples are determined by manual annotation. The eye-tracking spatiotemporal feature samples are input into a pre-constructed deep neural network to obtain the intention to acquire predictive information; The loss value is determined based on the predicted information to obtain the intent and the information acquisition intent label; The parameters of the deep neural network are updated using the backpropagation algorithm based on the loss value to obtain the trained driver intention prediction model.

[0013] Furthermore, in one embodiment of the present invention, determining the information display priority of each of the instrument information to be displayed based on the security requirement level, the scenario requirement level, and the intent requirement level specifically involves: The security requirement level, the scenario requirement level, and the intent requirement level are weighted and summed according to preset weight coefficients to obtain the information display priority of the instrument information to be displayed.

[0014] Furthermore, in one embodiment of the present invention, the step of dividing the instrument information to be displayed into core layer information, auxiliary layer information, and redundant layer information according to the information display priority, and displaying the core layer information, the auxiliary layer information, and the redundant layer information in a hierarchical manner, specifically includes: When the information display priority is greater than or equal to a preset first threshold, the corresponding instrument information to be displayed is determined to be the core layer information. When the information display priority is less than or equal to a preset second threshold and less than the first threshold, the corresponding instrument information to be displayed is determined to be the auxiliary layer information. When the information display priority is less than the second threshold, the corresponding instrument information to be displayed is determined to be the redundant layer information. The core layer information is displayed through the HUD head-up display area, the auxiliary layer information is displayed through the instrument panel, and the redundant layer information is displayed through the HUD head-up display area or the instrument panel after the target driver issues a wake-up command.

[0015] On the other hand, embodiments of the present invention provide a vehicle instrument information priority sorting device based on eye-tracking trajectories, comprising: The data acquisition module is used to acquire vehicle status data, external environment data, and eye movement trajectory data of the target driver. The demand level classification module is used to determine the safety demand level of each instrument information to be displayed based on the vehicle status data, determine the scene demand level of each instrument information to be displayed based on the external environment data, and determine the intent demand level of each instrument information to be displayed based on the eye tracking data. The priority determination module is used to determine the information display priority of each of the instrument information to be displayed based on the security requirement level, the scenario requirement level, and the intent requirement level. The information layering module is used to divide the instrument information to be displayed into core layer information, auxiliary layer information and redundant layer information according to the information display priority, and to display the core layer information, the auxiliary layer information and the redundant layer information in a hierarchical manner.

[0016] On the other hand, embodiments of the present invention provide an electronic device, including: At least one processor; At least one memory for storing at least one program; When the at least one program is executed by the at least one processor, the at least one processor implements the above-described method for prioritizing vehicle instrument information based on eye-tracking trajectories.

[0017] On the other hand, embodiments of the present invention also provide a computer-readable storage medium storing a processor-executable computer program that, when executed by a processor, implements the above-described method for prioritizing vehicle instrument information based on eye-tracking trajectories.

[0018] On the other hand, embodiments of the present invention also provide a computer program product, including a computer program that, when executed by a processor, implements the above-described method for prioritizing vehicle instrument information based on eye-tracking trajectories.

[0019] The advantages and beneficial effects of the present invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention: This invention acquires vehicle status data, external environment data, and eye-tracking trajectory data of the target vehicle. Based on the vehicle status data, it determines the safety requirement level of each instrument information to be displayed; based on the external environment data, it determines the scenario requirement level of each instrument information to be displayed; and based on the eye-tracking trajectory data, it determines the intent requirement level of each instrument information to be displayed. Based on the safety requirement level, scenario requirement level, and intent requirement level, it determines the information display priority of each instrument information to be displayed. According to the information display priority, the instrument information to be displayed is divided into core layer information, auxiliary layer information, and redundant layer information, and then displayed hierarchically. This invention, based on vehicle status data, external environment data, and the driver's eye-tracking trajectory data, determines the safety requirement level, scenario requirement level, and intent requirement level of each instrument information to be displayed, then calculates the information display priority of each instrument information to be displayed, and divides the instrument information to be displayed into core layer information, auxiliary layer information, and redundant layer information according to the information display priority. This achieves priority display of instrument information with high importance and high demand, improving the efficiency and timeliness of driver information acquisition. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the embodiments of the present invention are described below. It should be understood that the drawings described below are only for the convenience of clearly describing some embodiments of the technical solutions of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 A flowchart illustrating the steps of a method for prioritizing vehicle instrument information based on eye-tracking trajectories, provided in an embodiment of the present invention; Figure 2 A structural block diagram of a vehicle instrument information priority sorting device based on eye-tracking trajectory provided in an embodiment of the present invention; Figure 3 This is a structural block diagram of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0022] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention. In the following description, when referring to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the embodiments of this invention; they are merely examples of apparatuses and methods consistent with some aspects of the embodiments of this invention as detailed in the appended claims.

[0023] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein is for the purpose of describing embodiments of the invention only and is not intended to limit the invention.

[0024] The vehicle instrument information priority ranking method based on eye-tracking trajectory provided in this invention can be applied to a terminal, a server, or software running on either a terminal or a server. In some embodiments, the terminal can be a smartphone, tablet, laptop, desktop computer, smart speaker, smartwatch, or in-vehicle terminal, but is not limited to these. The server can be configured as an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The server can also be a node server in a blockchain network. The software can be an application implementing the vehicle instrument information priority ranking method based on eye-tracking trajectory, but is not limited to the above forms.

[0025] This invention can be used in a wide variety of general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices. This invention can be described in the general context of computer-executable instructions, such as program modules, that are executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform specific tasks or implement specific abstract data types. This invention can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0026] It should be noted that in various specific embodiments of the present invention, when processing data related to user identity or characteristics, such as user information, user behavior data, user historical data, and user location information, user permission or consent is obtained first. Furthermore, the collection, use, and processing of this data comply with relevant laws, regulations, and standards. In addition, when embodiments of the present invention require access to sensitive personal information of users, separate permission or consent from the user is obtained through pop-ups or redirection to a confirmation page. Only after obtaining the user's separate permission or consent is the necessary user-related data for the normal operation of the embodiments of the present invention acquired.

[0027] Reference Figure 1 This invention provides a method for prioritizing vehicle instrument information based on eye-tracking trajectories, specifically including the following steps: S101. Obtain vehicle status data, external environment data, and eye movement trajectory data of the target driver; S102. Determine the safety requirement level of each instrument information to be displayed based on vehicle status data, determine the scenario requirement level of each instrument information to be displayed based on external environment data, and determine the intent requirement level of each instrument information to be displayed based on eye-tracking data. S103. Determine the information display priority of each instrument information to be displayed based on the security requirement level, scenario requirement level, and intent requirement level. S104. Based on the information display priority, divide the instrument information to be displayed into core layer information, auxiliary layer information and redundant layer information, and display the core layer information, auxiliary layer information and redundant layer information in a hierarchical manner.

[0028] This invention determines the safety requirement level, scenario requirement level, and intent requirement level of each instrument information to be displayed based on vehicle status data, external environment data, and driver's eye movement trajectory data. Then, it calculates the information display priority of each instrument information to be displayed and divides the instrument information to be displayed into core layer information, auxiliary layer information, and redundant layer information according to the information display priority. This achieves priority display of instrument information with high importance and high demand, improving the efficiency and timeliness of driver information acquisition.

[0029] As an optional implementation, vehicle status data includes vehicle speed, tire pressure, range, and driver assistance information. The safety requirement level for each instrument displaying information is determined based on the vehicle status data, specifically including: The vehicle status data is input into a pre-trained vehicle fault prediction model to obtain the vehicle fault prediction result. Based on the vehicle fault prediction results, the corresponding information display safety classification rules are obtained by matching them in the preset personalized configuration library; Based on the information type of each instrument to be displayed, the corresponding security requirement level is obtained by matching it in the information display security classification rules.

[0030] Specifically, information such as vehicle speed, tire pressure, range, and driver assistance features (e.g., lane departure warning) is acquired via the CAN bus and then input into a pre-trained vehicle fault prediction model to obtain a vehicle fault prediction result, i.e., the type of potential fault the vehicle may have. Then, based on the type of potential fault, the model matches the pre-configured information display safety classification rules corresponding to that type of potential fault in the personalized configuration library. Based on the information type of each instrument information to be displayed, the model matches the corresponding safety requirement level in the information display safety classification rules. This safety requirement level represents the importance of various types of instrument information to be displayed under that type of potential fault. For example, when the vehicle has a potential fault of insufficient battery range, the safety requirement level of information such as energy consumption and battery charge is higher.

[0031] It should be noted that the vehicle fault prediction model can use existing publicly available model training methods, which will not be elaborated upon in this embodiment of the invention.

[0032] As an optional implementation, external environmental data includes road image data, radar detection data, and environmental weather conditions. The scene requirement level for each instrument's information to be displayed is determined based on this external environmental data, specifically including: The current road type is identified based on road image data, the distance to obstacles is identified based on radar detection data, and the current visibility is determined based on environmental weather conditions. The current driving scenario is determined based on the current road type, distance to obstacles, and current visibility. Based on the current driving scenario, the system matches the corresponding information display scenario classification rules from the preset personalized configuration library. Based on the information type of each instrument to be displayed, the corresponding scene requirement level is obtained by matching it in the information display scene classification rules.

[0033] Specifically, the system uses road condition cameras to collect real-time road images to identify the current road type, millimeter-wave radar to detect the distance to obstacles ahead, and weather sensors to identify the rain / fog level and current visibility. Then, based on the current road type, obstacle distance, and current visibility, the current driving scenario is determined. The specific method can be rule mapping or decision tree model, which is not limited here. Finally, based on the current driving scenario, the system matches the corresponding information display scenario classification rules from the preset personalized configuration library. Based on the information type of each instrument information to be displayed, the system matches the corresponding scenario demand level from the information display scenario classification rules. This scenario demand level represents the importance of various instrument information to be displayed in the current driving scenario. For example, when the vehicle is in a congested urban scenario, the scenario demand level of information such as traffic light countdown and surrounding vehicle status is higher.

[0034] As a further optional implementation, the intent demand level for each instrument information to be displayed is determined based on eye-tracking trajectory data, specifically including: Based on eye movement trajectory data, the target driver's gaze point distribution, gaze duration, saccade path, saccade speed, and focusing frequency are extracted to obtain spatiotemporal eye movement feature data; Eye-tracking spatiotemporal feature data is input into a pre-trained driver intent prediction model to obtain the target driver's information acquisition intent. Based on the information acquisition intent, the system matches the corresponding information display intent classification rules from the preset personalized configuration library. Based on the information type of each instrument to be displayed, the corresponding intent requirement level is obtained by matching it in the information display intent classification rules.

[0035] Specifically, a high-frame-rate eye-tracking camera inside the vehicle is used to capture the driver's eye movement trajectory, extracting features such as gaze point distribution, gaze duration, saccade path, saccade speed, and focusing frequency to obtain spatiotemporal eye movement feature data. Then, the spatiotemporal eye movement feature data is input into a pre-trained driver intention prediction model to obtain the target driver's information acquisition intention. Finally, based on the information acquisition intention, the corresponding information display intention classification rule is matched in a preset personalized configuration library. Based on the information type of each instrument information to be displayed, the corresponding intention demand level is matched in the information display intention classification rule to obtain the corresponding intention demand level. This intention demand level represents the degree of demand for various types of instrument information to be displayed under the information acquisition intention. For example, when the driver has a need for navigation guidance, the scenario demand level of information such as navigation route and road conditions is relatively high.

[0036] As an optional implementation, the driver intent prediction model is trained through the following steps: Obtain eye movement trajectory samples of the test driver, and extract the gaze point distribution, gaze duration, saccade path, saccade speed and focusing frequency of the test driver based on the eye movement trajectory samples to obtain eye movement spatiotemporal feature samples; The intent label for obtaining information corresponding to eye-tracking spatiotemporal feature samples is determined through manual annotation. Eye-tracking spatiotemporal feature samples are input into a pre-constructed deep neural network to obtain the predicted information acquisition intent; The loss value is determined based on the predicted information acquisition intent and the information acquisition intent label; The parameters of the deep neural network are updated using the backpropagation algorithm based on the loss value, resulting in a trained driver intention prediction model.

[0037] Specifically, eye movement trajectory samples of the test driver are obtained. Based on the eye movement trajectory samples, the driver's fixation point distribution, fixation duration, saccade path, saccade speed, and focusing frequency are extracted to obtain eye movement spatiotemporal feature samples. The information acquisition intent labels corresponding to the eye movement spatiotemporal feature samples are determined by manual annotation. The eye movement spatiotemporal feature samples are input into a pre-constructed deep neural network to obtain the predicted information acquisition intent. The loss value is determined based on the predicted information acquisition intent and the information acquisition intent label. The parameters of the deep neural network are updated based on the loss value through the backpropagation algorithm to complete one iteration of training. When the number of iterations reaches a preset threshold or the loss value is lower than the preset threshold, training is stopped, and the trained driver intent prediction model is obtained.

[0038] It should be noted that the driver intention prediction model in this embodiment of the invention is different from the traditional driver intention prediction model. Traditional driver intention prediction models mostly focus on the driver's control intention, while the driver intention prediction model in this embodiment of the invention predicts the driver's information acquisition intention based on eye-tracking data.

[0039] As an optional implementation, the information display priority of each instrument's information to be displayed is determined based on the security requirement level, scenario requirement level, and intent requirement level, specifically as follows: The security requirement level, scenario requirement level, and intent requirement level are weighted and summed according to preset weight coefficients to obtain the information display priority of the corresponding instrument information to be displayed.

[0040] Specifically, a weighted fusion algorithm is used to sum the security requirement level, scenario requirement level, and intent requirement level to obtain the information display priority of the corresponding instrument information to be displayed. The weight coefficients of the security requirement level, scenario requirement level, and intent requirement level can be preset. For example, the weight coefficient of the security requirement level is 0.4, the weight coefficient of the scenario requirement level is 0.3, and the weight coefficient of the intent requirement level is 0.4.

[0041] As a further optional implementation, the instrument information to be displayed is divided into core layer information, auxiliary layer information, and redundant layer information according to the information display priority, and the core layer information, auxiliary layer information, and redundant layer information are displayed in a hierarchical manner, specifically including: When the information display priority is greater than or equal to the preset first threshold, the corresponding instrument information to be displayed is determined as core layer information. When the information display priority is less than or equal to the preset second threshold and less than the first threshold, the corresponding instrument information to be displayed is determined to be auxiliary layer information. When the information display priority is less than the second threshold, the corresponding instrument information to be displayed is determined to be redundant layer information. The core layer information is displayed through the HUD head-up display area, the auxiliary layer information is displayed through the instrument panel, and the redundant layer information is displayed through the HUD head-up display area or the instrument panel after the target driver issues a wake-up command.

[0042] Specifically, in this embodiment of the invention, the instrument information to be displayed is divided into a core layer (8-10 points), an auxiliary layer (4-7 points), and a redundant layer (0-3 points) according to the information display priority. For example, the core layer contains safety-related information (such as fault warnings and obstacle prompts) and driver-demanded information (such as key navigation guidance), the auxiliary layer contains regular usage information (such as energy consumption and tire pressure), and the redundant layer contains low-frequency demand information (such as entertainment settings and weather details).

[0043] After information is layered, core layer information, auxiliary layer information, and redundant layer information can be displayed in different ways. For example, core layer information can be displayed through the HUD head-up display area, auxiliary layer information can be displayed through the instrument panel, and redundant layer information can be displayed through the HUD head-up display area or the instrument panel after the target driver issues a wake-up command.

[0044] In some optional embodiments, priority recalculation and sorting updates can be quickly triggered when eye-tracking trajectories or environmental data change; at the same time, a sorting stability threshold is set to avoid high-frequency switching in a short period of time (switching interval ≥ 0.5 seconds) and reduce visual interference.

[0045] The method steps of the embodiments of the present invention have been described above. It can be understood that the embodiments of the present invention determine the safety requirement level, scenario requirement level, and intent requirement level of each instrument information to be displayed based on vehicle status data, external environment data, and driver eye-tracking trajectory data, respectively. Then, the information display priority of each instrument information to be displayed is calculated. Based on this information display priority, the instrument information to be displayed is divided into core layer information, auxiliary layer information, and redundant layer information, thereby achieving priority display of instrument information with high importance and high demand, improving the efficiency and timeliness of driver information acquisition.

[0046] Reference Figure 2 This invention provides a vehicle instrument information priority sorting device based on eye-tracking trajectory, comprising: The data acquisition module is used to acquire vehicle status data, external environment data, and eye movement trajectory data of the target driver. The requirement level classification module is used to determine the safety requirement level of each instrument information to be displayed based on vehicle status data, the scenario requirement level of each instrument information to be displayed based on external environment data, and the intent requirement level of each instrument information to be displayed based on eye-tracking data. The priority determination module is used to determine the information display priority of each instrument information to be displayed based on the security requirement level, scenario requirement level, and intent requirement level. The information layering module is used to divide the instrument information to be displayed into core layer information, auxiliary layer information, and redundant layer information according to the information display priority, and to display the core layer information, auxiliary layer information, and redundant layer information in a hierarchical manner.

[0047] It is understood that the content of the above method embodiments is applicable to the present device embodiments. The specific functions implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0048] Reference Figure 3 This invention provides an electronic device, comprising: At least one processor; At least one memory for storing at least one program; When the above-mentioned at least one program is executed by the above-mentioned at least one processor, the above-mentioned at least one processor implements the above-mentioned method for prioritizing vehicle instrument information based on eye-tracking trajectory.

[0049] It is understood that the content of the above method embodiments is applicable to this device embodiment. The specific functions implemented by this device embodiment are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0050] This invention also provides a computer-readable storage medium storing a processor-executable computer program that, when executed by a processor, implements the above-described method for prioritizing vehicle instrument information based on eye-tracking trajectories.

[0051] This invention provides a computer-readable storage medium that can execute a vehicle instrument information priority sorting method based on eye-tracking trajectory provided in the method embodiment of this invention. It can execute any combination of the implementation steps of the method embodiment and has the corresponding functions and beneficial effects of the method.

[0052] This invention also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described method for prioritizing vehicle instrument information based on eye-tracking trajectories.

[0053] It is understood that the content of the above method embodiments is applicable to the embodiments of this program product. The specific functions implemented by the embodiments of this program product are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0054] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0055] The embodiments described in this invention are for the purpose of more clearly illustrating the technical solutions of the embodiments of this invention, and do not constitute a limitation on the technical solutions provided by the embodiments of this invention. As those skilled in the art will know, with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of this invention are also applicable to similar technical problems.

[0056] The terms "first," "second," "third," "fourth," etc. (if present) in the specification and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0057] In some alternative embodiments, the functions / operations mentioned in the block diagrams may not occur in the order shown in the operation diagrams. For example, depending on the functions / operations involved, two consecutively shown blocks may actually be executed substantially simultaneously, or the aforementioned blocks may sometimes be executed in reverse order. Furthermore, the embodiments presented and described in the flowcharts of this invention are provided by way of example to provide a more comprehensive understanding of the technology. The disclosed methods are not limited to the operations and logic flows presented herein. Alternative embodiments are contemplated in which the order of various operations is changed and sub-operations described as part of a larger operation are executed independently.

[0058] Furthermore, although the invention has been described in the context of functional modules, it should be understood that, unless otherwise stated, one or more of the aforementioned functions and / or features may be integrated into a single physical device and / or software module, or one or more functions and / or features may be implemented in a separate physical device or software module. It is also understood that a detailed discussion of the actual implementation of each module is unnecessary for understanding the invention. Rather, given the properties, functions, and internal relationships of the various functional modules in the apparatus disclosed herein, the actual implementation of the module will be understood within the scope of conventional skill of an engineer. Therefore, those skilled in the art can implement the invention as set forth in the claims using ordinary techniques without excessive experimentation. It is also understood that the specific concepts disclosed are merely illustrative and not intended to limit the scope of the invention, which is determined by the full scope of the appended claims and their equivalents.

[0059] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0060] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-including system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device.

[0061] More specific examples (a non-exhaustive list) of computer-readable media include: electrical connections (electronic devices) having one or more wires, portable computer disk drives (magnetic devices), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Furthermore, computer-readable media can even be paper or other suitable media on which the aforementioned program can be printed, because the aforementioned program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in computer memory.

[0062] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0063] In the foregoing description of this specification, references to terms such as "one embodiment," "another embodiment," or "some embodiments" indicate that a specific feature, structure, material, or characteristic described in connection with an embodiment or example is included in at least one embodiment or example of the present invention. In this specification, illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0064] Although embodiments of the invention have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the claims and their equivalents.

[0065] The above is a detailed description of the preferred embodiments of the present invention. However, the present invention is not limited to the above embodiments. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of the present invention. All such equivalent modifications or substitutions are included within the scope defined by the claims of the present invention.

Claims

1. A method for prioritizing vehicle instrument information based on eye-tracking trajectories, characterized in that, Includes the following steps: Acquire vehicle status data, external environment data, and eye movement trajectory data of the target driver; The safety requirement level of each instrument information to be displayed is determined based on the vehicle status data, the scene requirement level of each instrument information to be displayed is determined based on the external environment data, and the intent requirement level of each instrument information to be displayed is determined based on the eye movement trajectory data. The information display priority of each instrument information to be displayed is determined based on the security requirement level, the scenario requirement level, and the intent requirement level. Based on the information display priority, the instrument information to be displayed is divided into core layer information, auxiliary layer information, and redundant layer information, and the core layer information, the auxiliary layer information, and the redundant layer information are displayed in a hierarchical manner.

2. The method for prioritizing vehicle instrument information based on eye-tracking trajectories according to claim 1, characterized in that, The vehicle status data includes vehicle speed, tire pressure, remaining range, and driver assistance information. The step of determining the safety requirement level of each instrument display based on the vehicle status data specifically includes: The vehicle status data is input into a pre-trained vehicle fault prediction model to obtain the vehicle fault prediction result. Based on the vehicle fault prediction results, the corresponding information display safety classification rules are matched in the preset personalized configuration library; The corresponding security requirement level is obtained by matching the information type of each instrument to be displayed with the information display security classification rules.

3. The method for prioritizing vehicle instrument information based on eye-tracking trajectory according to claim 1, characterized in that, The external environment data includes road image data, radar detection data, and environmental weather conditions. The step of determining the scene requirement level for each of the instruments to be displayed based on the external environment data specifically includes: The current road type is identified based on the road image data, the distance to obstacles is identified based on the radar detection data, and the current visibility is determined based on the environmental weather conditions. The current driving scenario is determined based on the current road type, the distance to the obstacle, and the current visibility. Based on the current driving scenario, the corresponding information display scenario classification rules are obtained by matching them in the preset personalized configuration library; Based on the information type of each instrument information to be displayed, the corresponding scenario requirement level is obtained by matching it with the information display scenario classification rules.

4. The method for prioritizing vehicle instrument information based on eye-tracking trajectories according to claim 1, characterized in that, The step of determining the intent demand level for each of the instrument information to be displayed based on the eye-tracking data specifically includes: Based on the eye movement trajectory data, the target driver's gaze point distribution, gaze duration, saccade path, saccade speed, and focusing frequency are extracted to obtain eye movement spatiotemporal feature data; The eye-tracking spatiotemporal feature data is input into a pre-trained driver intention prediction model to obtain the target driver's information acquisition intention. Based on the information acquisition intent, the corresponding information display intent classification rules are obtained by matching in the preset personalized configuration library; Based on the information type of each instrument information to be displayed, the corresponding intent requirement level is obtained by matching it with the information display intent classification rule.

5. The method for prioritizing vehicle instrument information based on eye-tracking trajectory according to claim 4, characterized in that, The driver intention prediction model is trained through the following steps: Obtain eye movement trajectory samples of the test driver, and extract the gaze point distribution, gaze duration, saccade path, saccade speed and focusing frequency of the test driver based on the eye movement trajectory samples to obtain eye movement spatiotemporal feature samples; The information acquisition intent labels corresponding to the eye movement spatiotemporal feature samples are determined by manual annotation. The eye-tracking spatiotemporal feature samples are input into a pre-constructed deep neural network to obtain the intention to acquire predictive information; The loss value is determined based on the predicted information to obtain the intent and the information acquisition intent label; The parameters of the deep neural network are updated using the backpropagation algorithm based on the loss value to obtain the trained driver intention prediction model.

6. The method for prioritizing vehicle instrument information based on eye-tracking trajectory according to claim 1, characterized in that, The step of determining the information display priority of each of the instrument information to be displayed based on the security requirement level, the scenario requirement level, and the intent requirement level specifically involves: The security requirement level, the scenario requirement level, and the intent requirement level are weighted and summed according to preset weight coefficients to obtain the information display priority of the instrument information to be displayed.

7. A method for prioritizing vehicle instrument information based on eye-tracking trajectories according to any one of claims 1 to 6, characterized in that, The step of dividing the instrument information to be displayed into core layer information, auxiliary layer information, and redundant layer information according to the information display priority, and displaying the core layer information, auxiliary layer information, and redundant layer information in a hierarchical manner, specifically includes: When the information display priority is greater than or equal to a preset first threshold, the corresponding instrument information to be displayed is determined to be the core layer information. When the information display priority is less than or equal to a preset second threshold and less than the first threshold, the corresponding instrument information to be displayed is determined to be the auxiliary layer information. When the information display priority is less than the second threshold, the corresponding instrument information to be displayed is determined to be the redundant layer information. The core layer information is displayed through the HUD head-up display area, the auxiliary layer information is displayed through the instrument panel, and the redundant layer information is displayed through the HUD head-up display area or the instrument panel after the target driver issues a wake-up command.

8. A vehicle instrument information priority sorting device based on eye-tracking trajectory, characterized in that, include: The data acquisition module is used to acquire vehicle status data, external environment data, and eye movement trajectory data of the target driver. The demand level classification module is used to determine the safety demand level of each instrument information to be displayed based on the vehicle status data, determine the scene demand level of each instrument information to be displayed based on the external environment data, and determine the intent demand level of each instrument information to be displayed based on the eye tracking data. The priority determination module is used to determine the information display priority of each of the instrument information to be displayed based on the security requirement level, the scenario requirement level, and the intent requirement level. The information layering module is used to divide the instrument information to be displayed into core layer information, auxiliary layer information and redundant layer information according to the information display priority, and to display the core layer information, the auxiliary layer information and the redundant layer information in a hierarchical manner.

9. An electronic device, characterized in that, include: At least one processor; At least one memory for storing at least one program; When the at least one program is executed by the at least one processor, the at least one processor implements a method for prioritizing vehicle instrument information based on eye-tracking trajectories as described in any one of claims 1 to 7.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements a method for prioritizing vehicle instrument information based on eye-tracking trajectories as described in any one of claims 1 to 7.

Images