Environment information display method, vehicle, medium and product

Abstract

The invention relates to the technical field of vehicles, and discloses an environment information display method, a vehicle, a medium and a product. The method comprises the following steps: acquiring a driving scene of a vehicle, and acquiring a current environment complexity coefficient and a maximum environment complexity coefficient according to the driving scene; obtaining a cognitive load correlation parameter value, and obtaining a cognitive load value of the driver according to the cognitive load correlation parameter value, the current environment complexity coefficient and the maximum environment complexity coefficient; and obtaining a target rendering mode according to the cognitive load value, and realizing rendering display of the environment information based on the target rendering mode. According to the scheme of the embodiment, quantitative evaluation of the cognitive load of the driver can be achieved, dynamic adjustment of the rendering mode can be achieved, dynamic adjustment of the rendering information density can be achieved, and the user experience can be improved.

Description

Technical Field

[0001] This invention relates to the field of vehicle technology, and more particularly to a method for displaying environmental information, a vehicle, a medium, and a product. Background Technology

[0002] Surrounding-Reality (SR) is a technology that reconstructs the surrounding environment to create a visual interface. By integrating multi-dimensional information, SR technology can render details such as multi-lane lines, zebra crossings, and obstacles around a vehicle in real time, forming a high-precision environmental model. It can also dynamically display the predicted movement trajectories of traffic participants and support the overlay of key parameters such as distance and speed, transforming complex physical environments into a concise visual interface.

[0003] Currently, existing environmental information display methods based on real-time rendering (SR) typically employ a fixed rendering mode, rendering and displaying all environmental information simultaneously. However, this approach increases the driver's cognitive load, especially after long periods or long distances of driving when the cognitive load is already high, severely impacting the driver's user experience. Summary of the Invention

[0004] This invention provides a method, vehicle, medium, and product for displaying environmental information, which can achieve quantitative assessment of driver cognitive load, dynamic adjustment of rendering mode, dynamic adjustment of rendering information density, and improve user experience.

[0005] According to one aspect of the present invention, a method for displaying environmental information is provided, comprising:

[0006] Obtain the vehicle's driving scenario, and based on the driving scenario, obtain the current environment complexity coefficient and the maximum environment complexity coefficient;

[0007] Obtain the cognitive load associated parameter value, and obtain the driver's cognitive load value based on the cognitive load associated parameter value, the current environmental complexity coefficient, and the maximum environmental complexity coefficient;

[0008] Based on the cognitive load value, the target rendering mode is obtained, and based on the target rendering mode, the environmental information is rendered and displayed.

[0009] According to another aspect of the present invention, an environmental information display device is provided, comprising:

[0010] The coefficient acquisition module is used to acquire the vehicle's driving scenario and, based on the driving scenario, acquire the current environment complexity coefficient and the maximum environment complexity coefficient.

[0011] The cognitive load value acquisition module is used to acquire cognitive load associated parameter values, and to acquire the driver's cognitive load value based on the cognitive load associated parameter values, the current environmental complexity coefficient, and the maximum environmental complexity coefficient.

[0012] The environment rendering module is used to obtain the target rendering mode based on the cognitive load value, and to render and display environmental information based on the target rendering mode.

[0013] According to another aspect of the present invention, a vehicle is provided, the vehicle comprising:

[0014] At least one processor; and

[0015] A memory communicatively connected to the at least one processor; wherein,

[0016] The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the environmental information display method according to any embodiment of the present invention.

[0017] According to another aspect of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium storing a computer program for causing a processor to execute and implement the environmental information display method according to any embodiment of the present invention.

[0018] According to another aspect of the present invention, a computer program product is provided, comprising a computer program that, when executed by a processor, implements the environmental information display method described in any embodiment of the present invention.

[0019] The technical solution of this invention acquires the vehicle's driving scenario and, based on the driving scenario, acquires the current environmental complexity coefficient and the maximum environmental complexity coefficient; acquires cognitive load correlation parameter values ​​and, based on the cognitive load correlation parameter values, the current environmental complexity coefficient, and the maximum environmental complexity coefficient, acquires the driver's cognitive load value; acquires the target rendering mode based on the cognitive load value and, based on the target rendering mode, realizes the rendering display of environmental information; by calculating the driver's cognitive load value based on the cognitive load correlation parameter values, the current environmental complexity coefficient, and the maximum environmental complexity coefficient, and selecting the current rendering mode based on the cognitive load value, it is possible to achieve a quantitative assessment of the driver's cognitive load, dynamically adjust the rendering mode, dynamically adjust the rendering information density, and improve the user experience.

[0020] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 This is a flowchart of a method for displaying environmental information according to Embodiment 1 of the present invention;

[0023] Figure 2 This is a schematic diagram of the structure of an environmental information display system provided in Embodiment 1 of the present invention;

[0024] Figure 3 This is a flowchart of another method for displaying environmental information according to Embodiment 1 of the present invention;

[0025] Figure 4 This is a schematic diagram of the structure of an environmental information display device according to Embodiment 2 of the present invention;

[0026] Figure 5 This is a schematic diagram of the structure of a vehicle that implements the environmental information display method of this invention. Detailed Implementation

[0027] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0028] It should be noted that the terms "first," "second," "target," etc., used in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0029] Example 1

[0030] Figure 1 This is a flowchart illustrating a method for displaying environmental information according to Embodiment 1 of the present invention. This embodiment is applicable to situations where environmental information surrounding a vehicle is rendered and displayed on a vehicle infotainment interface. This method can be executed by an environmental information display device, which can be implemented in hardware and / or software. Typically, this environmental information display device can be configured within the vehicle's infotainment system. Figure 1 As shown, the method includes:

[0031] S110. Obtain the vehicle's driving scenario, and based on the driving scenario, obtain the current environment complexity coefficient and the maximum environment complexity coefficient.

[0032] In this embodiment, various environmental feature information can be obtained through maps or navigation information. Based on these environmental feature information and the correspondence between the driving scenario and the environmental feature information, the current driving scenario is determined. For example, the driving scenario could be an urban road construction scenario or a highway with stable lighting. Furthermore, based on the correspondence between the driving scenario and the range of environmental complexity coefficient values, the range of environmental complexity coefficient values ​​corresponding to the current driving scenario can be determined. The median of this range is determined as the current environmental complexity coefficient, and the maximum value of this range is determined as the maximum environmental complexity coefficient. The environmental complexity coefficient is a quantitative indicator used to describe the complexity of the driving environment.

[0033] In a specific example, the correspondence between the range of environmental complexity coefficients and driving scenarios can be shown in Table 1. Driving scenarios are divided into levels I-IV, with higher levels indicating greater complexity. Each level corresponds to a specific range of environmental complexity coefficients and environmental characteristics. As the level increases, the boundary values ​​of the range gradually increase. Environmental characteristics may include road type, driving area type, lane line type, number of dynamic targets, and weather conditions.

[0034] Table 1. Correspondence between the range of environmental complexity coefficients and driving scenarios.

[0035]

[0036] S120. Obtain the cognitive load associated parameter value, and obtain the driver's cognitive load value based on the cognitive load associated parameter value, the current environmental complexity coefficient, and the maximum environmental complexity coefficient.

[0037] The cognitive load-related parameter values ​​can be the values ​​of characteristic parameters that affect the driver's cognitive load (CL). These characteristic parameters may include pupil diameter change rate, skin conductance response, head micro-motion variability, steering deviation error, and voice stress index, etc. In this embodiment, the parameter values ​​corresponding to each parameter can be detected by pre-deployed sensors and detection devices, and used as cognitive load-related parameter values. Then, the current cognitive load value can be determined according to the mapping relationship between the preset cognitive load-related parameter values, the current environmental complexity coefficient, the maximum environmental complexity coefficient, and the cognitive load value. A higher CL value indicates a higher cognitive load and greater mental stress for the driver.

[0038] Optionally, obtaining cognitive load-related parameter values ​​may include: obtaining pupil diameter change rate, skin conductance response value, head micromotion variability, turning deviation error value, and speech stress index value;

[0039] Correspondingly, obtaining the driver's cognitive load value based on the cognitive load correlation parameter value, the current environmental complexity coefficient, and the maximum environmental complexity coefficient may include:

[0040] The driver's cognitive load value is obtained based on the pupil diameter change rate, skin conductance response value, head micro-motion variability, steering deviation error value, voice stress index value, current environmental complexity coefficient, and maximum environmental complexity coefficient.

[0041] In an optional example, the cognitive load-related parameter values ​​include pupil diameter change rate, skin conductance response value, head micromotion variability, turning deviation error value, and voice stress index value. In this embodiment, the mapping relationship between each cognitive load-related parameter value, the current environmental complexity coefficient, the maximum environmental complexity coefficient, and the cognitive load value can be preset. Therefore, after obtaining the various cognitive load-related parameter values, the current environmental complexity coefficient, and the maximum environmental complexity coefficient, the current cognitive load value can be obtained by looking up the preset mapping relationship.

[0042] Examples can be based on formulas Based on the pupil diameter change rate ΔPNorm Skin conductance response value (GSR) Norm Head micromotion variability H Norm Steering deviation error value T Norm Voice stress index value V Norm Current environmental complexity coefficient K env and the maximum environmental complexity coefficient K max The cognitive load value CL is calculated. Here, a1, a2, a3, a4, and a5 are weighting coefficients that can be adaptively adjusted according to the project. Typically, a1 = 0.3, a2 = 0.25, a3 = 0.2, a4 = 0.15, and a5 = 0.1. The environmental complexity coefficient is a dynamically adjustable parameter with a value range of 1.0 (highway) to 2.5 (intersection in densely populated urban areas).

[0043] It should be noted that the values ​​of all cognitive load-related parameters are in the range of [0,1]. Values ​​exceeding 1 are limited to the maximum value of 1, and values ​​less than 0 are limited to the minimum value of 0.

[0044] The advantage of the above setup is that by fusing multimodal perception data, the accuracy of cognitive load assessment can be significantly improved.

[0045] Optionally, obtaining the pupil diameter change rate may include: obtaining the pupil diameter monitoring value of the driver, and calculating the pupil diameter change rate based on the pupil diameter monitoring value, as well as a preset mean resting pupil diameter and baseline standard deviation of pupil diameter.

[0046] The pupil diameter change rate can be the relative change between the driver's real-time pupil diameter monitoring value and the baseline value, reflecting the degree of visual attention resource consumption. Pupil dilation is positively correlated with increased cognitive load. In an optional example, firstly, the driver's pupil diameter monitoring value can be obtained through an infrared pupil tracking module; then, it can be based on the formula ΔP... Norm =(P current -μ baseline ) / σ baseline According to the pupil diameter monitoring value P current Mean resting pupil diameter (μ) baseline and baseline standard deviation of pupil diameter σ baseline The rate of change of pupil diameter ΔP was calculated. Norm For example, μ baseline = 3.5 mm, σ baseline = 0.8 mm.

[0047] The advantage of the above settings is that they allow for accurate acquisition of the rate of change in pupil diameter.

[0048] Optionally, obtaining the skin conductance response value may include: obtaining the skin conductance value of the driver's palm through a steering wheel capacitive sensor, and calculating the skin conductance response value based on the palm skin conductance value and a pre-calibrated skin conductance baseline value.

[0049] Among them, the skin conductance response value GSR Norm GSR is a quantitative indicator of the degree of sympathetic nerve activation, directly reflecting emotional stress and cognitive conflict. In an optional example, the palmar skin conductance value GSR and the pre-calibrated baseline skin conductance value GSR0 can be substituted into the formula GSR. Norm = (GSR - GSR0) / GSR, the skin conductance response value GSR is calculated. Norm The baseline skin conductance value is the skin conductance value of the driver at rest, measured in microsieverts (µS).

[0050] The advantage of the above settings is that they allow for accurate acquisition of skin conductance response values.

[0051] Optionally, obtaining the head micro-motion variability may include: obtaining the current head sway frequency and current head sway amplitude through an onboard millimeter-wave radar, and calculating the head micro-motion variability based on the current head sway frequency, the current head sway amplitude, and a pre-calibrated maximum head sway amplitude.

[0052] The head micro-motion variability is a comprehensive indicator of head sway frequency and amplitude, measured in Hertz-millimeters (Hz·mm). Abnormally high-frequency micro-motions suggest inattention or decision-making difficulties. In one optional example, millimeter-wave radar can be used to detect the number of head swaying movements per unit time, as well as the amplitude of each movement. Then, based on the number of head swaying movements and the unit time, the current head sway frequency can be calculated, and the maximum amplitude of each movement can be determined as the current head sway amplitude. Finally, a formula can be used... Based on the current head sway frequency f t Current head sway amplitude and the pre-calibrated maximum head swing amplitude The head micromotion variability H was calculated. Norm The maximum head swing amplitude can be obtained through a 30-second resting calibration.

[0053] The advantage of the above settings is that they allow for accurate acquisition of the degree of variation in head micro-motion.

[0054] Optionally, obtaining the steering deviation error value may include: obtaining the actual steering angle and the expected steering angle, and calculating the steering deviation error value based on the actual steering angle, the expected steering angle and the preset maximum steering angle.

[0055] The steering deviation error value can be the root mean square error between the actual steering angle and the expected path planning angle, measured in degrees (°). It measures the driver's control capability, and an increase in error indicates overload. In this embodiment, the actual steering angle of the vehicle can be obtained using a steering wheel angle sensor, and the expected steering angle can be obtained based on the expected path. Then, it can be based on the formula... Based on the actual steering angle θ actual Expected steering angle θ expect and preset maximum steering angle θ max The steering deviation error value T is calculated. Norm For example, the default maximum steering angle can be 30 degrees.

[0056] The advantage of the above settings is that they allow for accurate acquisition of steering deviation error values.

[0057] Optionally, obtaining the speech stress index value may include: obtaining the fundamental frequency standard deviation, speech rate change rate, and the proportion of silence interval duration; and obtaining the speech stress index value based on the fundamental frequency standard deviation, the speech rate change rate, and the proportion of silence interval duration using a pre-trained long short-term memory network model.

[0058] In this embodiment, firstly, the driver's voice data is acquired via a microphone array, and the voice data is analyzed and processed to obtain the fundamental frequency standard deviation (reflecting pitch fluctuations), speech rate change rate (syllables / second), and the percentage of silence interval duration. Then, the fundamental frequency standard deviation, speech rate change rate, and percentage of silence interval duration are input into a pre-trained Long Short-Term Memory (LSTM) network model, and the speech stress index value output by the LTM network model is obtained. The LTM network model can be trained based on pre-labeled standard speech data.

[0059] The advantage of the above settings is that they allow for accurate acquisition of the voice stress index value.

[0060] S130. Based on the cognitive load value, obtain the target rendering mode, and based on the target rendering mode, realize the rendering and display of environmental information.

[0061] In this embodiment, a pre-defined correspondence between cognitive load value ranges and rendering modes can be established. Therefore, after determining the cognitive load value, the cognitive load value range within the pre-defined correspondence can be found, and the rendering mode corresponding to this cognitive load value range can be determined as the target rendering mode. Finally, the vehicle's environmental information can be rendered and displayed on the vehicle's infotainment system interface according to this target rendering mode.

[0062] In a specific example, the correspondence between cognitive load ranges and rendering modes is shown in Table 2. Cognitive load is divided into five levels, CL1-CL5, each corresponding to a different CL value range, information density, and rendering mode. As the level increases, the boundary values ​​of the value range gradually increase, the information density gradually decreases, and the rendering mode gradually simplifies. Rendering modes, from high to low, are full rendering, dynamic degradation, simplified rendering, emergency mode, and minimized display. Each rendering mode has pre-set rendering rules. When the CL value is too high, the system activates a mechanism to adjust simplified rendering elements and provides a voice prompt.

[0063] Table 2. Correspondence between cognitive load range and rendering mode

[0064]

[0065]

[0066] Optionally, after obtaining the CL value, it can be filtered, smoothed, and hysteresis-processed to obtain an updated CL value. The target rendering mode can then be determined based on this updated CL value. The filtering and smoothing process can use a first-order IIR filter, expressed as CL_FLT. i =α×CL i +(1-α)×CL_FLT i-1 α is 2.0-2.5.

[0067] The advantage of the above settings is that they can avoid frequent switching between different rendering modes, which increases the cognitive load on the user.

[0068] Optionally, based on this embodiment, the operations corresponding to the five rendering modes can be further improved. For example, non-critical vehicle models can be directly hidden, the display size of the corresponding local area can be enlarged by 150%, and pulse animation effects can be added. Secondly, a multi-level progressive warning mechanism in emergency situations can be further implemented. Taking the three-level warning as an example, when the rendering mode enters CL3, it enters the first-level warning (yellow warning), at which time the driver is alerted by the warning light or the flashing edge of the SR window; when the rendering mode enters CL4, it enters the second-level warning (orange warning), at which time the warning box size is enlarged to 150% and a voice alarm is issued; when the rendering mode enters CL5, it enters the third-level warning (red warning), at which time the current task is forcibly interrupted and the emergency rendering pipeline is started, non-critical vehicle models are hidden, and the braking warning is activated.

[0069] In one specific embodiment of this example, the structure of the environmental information display system can be as follows: Figure 2As shown, the system includes an infrared pupil tracking module, a capacitive steering wheel, a 60GHz millimeter-wave radar, a steering module, a microphone, an environmental recognition module, a vehicle controller, and a vehicle display screen. Based on this environmental information display system, the process for displaying environmental information can be as follows: Figure 3 As shown. First, through the infrared pupil tracking module, capacitive steering wheel, 60GHz millimeter-wave radar, steering module, microphone, and environmental recognition module, raw quantities such as cognitive load-related parameter values ​​and environmental complexity coefficients are detected and obtained. The raw quantities are then preprocessed, including normalization and limit settings, to obtain the formula input variables. Then, through the vehicle controller, based on the cognitive load formula and the above formula input variables, the CL value is calculated, and the CL value is filtered and hysteresis processed to obtain the final CL value. Finally, based on the final CL value, the target rendering mode is obtained, and based on the target rendering mode, the environmental information is rendered and displayed on the vehicle display screen.

[0070] In a specific example, the rate of change of pupil diameter ΔP Norm =0.80, skin conductance response value GSR Norm =0.73, Head micromotion variability H Norm =0.62, steering deviation error value T Norm =0.58, the driver is at a normal interaction intensity, the voice stress index value V Norm =0.48. Considering the current scenario is an urban road construction scene (assuming a sunny day), and the scene recognition comes from autonomous driving or maps, the scene level is determined to be Level IV according to Table 1, i.e., an extremely complex unstructured environment. Let the complexity coefficient of the current environment be K. env =2.2 (sunny day), maximum environmental complexity coefficient K max =2.5.

[0071] Based on the cognitive load formula, the real-time CL value is calculated according to the above parameter values. After filtering (filter coefficient α is 0.2 to 0.35) and hysteresis (H = 15%), the final CL value is 0.62. Looking up CL = 0.62 in Table 2, the cognitive load level is CL4, corresponding to the emergency mode, which involves switching to monochrome lane line rendering and simplifying vehicle models within a 30-meter range. Additionally, further processing can be added based on this mode, such as hiding non-critical vehicle models, enlarging construction area annotations by 150%, and simplifying voice prompts to "Construction ahead, keep left lane," etc.

[0072] The technical solution of this invention acquires the vehicle's driving scenario and, based on the driving scenario, acquires the current environmental complexity coefficient and the maximum environmental complexity coefficient; acquires cognitive load correlation parameter values ​​and, based on the cognitive load correlation parameter values, the current environmental complexity coefficient, and the maximum environmental complexity coefficient, acquires the driver's cognitive load value; acquires the target rendering mode based on the cognitive load value and, based on the target rendering mode, realizes the rendering display of environmental information; by calculating the driver's cognitive load value based on the cognitive load correlation parameter values, the current environmental complexity coefficient, and the maximum environmental complexity coefficient, and selecting the current rendering mode based on the cognitive load value, it is possible to achieve a quantitative assessment of the driver's cognitive load, dynamically adjust the rendering mode, dynamically adjust the rendering information density, and improve the user experience.

[0073] Example 2

[0074] Figure 4 This is a schematic diagram of the structure of an environmental information display device provided in Embodiment 2 of the present invention. Figure 4 As shown, the device includes: a coefficient acquisition module 210, a cognitive load value acquisition module 220, and an environment rendering module 230; wherein,

[0075] The coefficient acquisition module 210 is used to acquire the vehicle's driving scenario and, based on the driving scenario, acquire the current environment complexity coefficient and the maximum environment complexity coefficient.

[0076] The cognitive load value acquisition module 220 is used to acquire cognitive load associated parameter values, and to acquire the driver's cognitive load value based on the cognitive load associated parameter values, the current environmental complexity coefficient, and the maximum environmental complexity coefficient.

[0077] The environment rendering module 230 is used to obtain the target rendering mode based on the cognitive load value, and to render and display environmental information based on the target rendering mode.

[0078] The technical solution of this invention acquires the vehicle's driving scenario and, based on the driving scenario, acquires the current environmental complexity coefficient and the maximum environmental complexity coefficient; acquires cognitive load correlation parameter values ​​and, based on the cognitive load correlation parameter values, the current environmental complexity coefficient, and the maximum environmental complexity coefficient, acquires the driver's cognitive load value; acquires the target rendering mode based on the cognitive load value and, based on the target rendering mode, realizes the rendering display of environmental information; by calculating the driver's cognitive load value based on the cognitive load correlation parameter values, the current environmental complexity coefficient, and the maximum environmental complexity coefficient, and selecting the current rendering mode based on the cognitive load value, it is possible to achieve a quantitative assessment of the driver's cognitive load, dynamically adjust the rendering mode, dynamically adjust the rendering information density, and improve the user experience.

[0079] Optionally, the cognitive load value acquisition module 220 is specifically used to acquire pupil diameter change rate, skin conductance response value, head micro-motion variability, turning deviation error value and voice stress index value;

[0080] The driver's cognitive load value is obtained based on the pupil diameter change rate, skin conductance response value, head micro-motion variability, steering deviation error value, voice stress index value, current environmental complexity coefficient, and maximum environmental complexity coefficient.

[0081] Optionally, the cognitive load value acquisition module 220 is specifically used to acquire the pupil diameter monitoring value of the driver, and calculate the pupil diameter change rate based on the pupil diameter monitoring value, as well as the preset mean resting pupil diameter and the baseline standard deviation of pupil diameter.

[0082] Optionally, the cognitive load value acquisition module 220 is specifically used to acquire the driver's palm skin conductance value through the steering wheel capacitive sensor, and calculate the skin conductance response value based on the palm skin conductance value and the pre-calibrated skin conductance baseline value.

[0083] Optionally, the cognitive load value acquisition module 220 is specifically used to acquire the current head sway frequency and the current head sway amplitude through the vehicle-mounted millimeter-wave radar, and calculate the head micro-motion variability based on the current head sway frequency, the current head sway amplitude and the pre-calibrated maximum head sway amplitude.

[0084] Optionally, the cognitive load value acquisition module 220 is specifically used to acquire the actual steering angle and the expected steering angle, and calculate the steering deviation error value based on the actual steering angle, the expected steering angle and the preset maximum steering angle.

[0085] Optionally, the cognitive load value acquisition module 220 is used to acquire the fundamental frequency standard deviation, speech rate change rate, and silent interval duration percentage.

[0086] The speech stress index value is obtained by using a pre-trained long short-term memory network model based on the fundamental frequency standard deviation, the speech rate change rate, and the proportion of silence interval duration.

[0087] The environmental information display device provided in the embodiments of the present invention can execute the environmental information display method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the method execution.

[0088] The collection, storage, use, processing, transmission, provision, and disclosure of user personal information involved in the technical solution disclosed herein comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0089] Example 3

[0090] Figure 5 A schematic diagram of the structure of a vehicle 30 that can be used to implement embodiments of the present invention is shown. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the invention described and / or claimed herein.

[0091] like Figure 5 As shown, vehicle 30 includes at least one processor 31 and a memory, such as read-only memory (ROM) 32 and random access memory (RAM) 33, communicatively connected to at least one processor 31. The memory stores computer programs executable by at least one processor. The processor 31 can perform various appropriate actions and processes based on the computer program stored in ROM 32 or loaded from storage unit 38 into RAM 33. RAM 33 can also store various programs and data required for the operation of vehicle 30. The processor 31, ROM 32, and RAM 33 are interconnected via bus 34. Input / output (I / O) interface 35 is also connected to bus 34.

[0092] Multiple components in vehicle 30 are connected to I / O interface 35, including: input unit 36; output unit 37, such as various types of displays, speakers, etc.; storage unit 38, such as disks, optical disks, etc.; and communication unit 39, such as network cards, modems, wireless transceivers, etc. Communication unit 39 allows vehicle 30 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0093] Processor 31 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 31 include, but are not limited to, central processing units, graphics processing units, various special-purpose artificial intelligence computing chips, various processors running machine learning model algorithms, digital signal processors, and any suitable processor, controller, microcontroller, etc. Processor 31 performs the various methods and processes described above, such as methods for displaying environmental information.

[0094] In some embodiments, the method for displaying environmental information may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 38. In some embodiments, part or all of the computer program may be loaded and / or installed on vehicle 30 via ROM 32 and / or communication unit 39. When the computer program is loaded into RAM 33 and executed by processor 31, one or more steps of the environmental information display method described above may be performed. Alternatively, in other embodiments, processor 31 may be configured to perform the environmental information display method by any other suitable means (e.g., by means of firmware).

[0095] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays, application-specific integrated circuits (ASICs), application-specific standard products (ASICs), systems-on-a-chip (SoCs), payload programmable logic devices, computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0096] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0097] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory, read-only memory, erasable programmable read-only memory, optical fibers, portable compact disk read-only memory, optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0098] To provide interaction with the user, the systems and technologies described herein can be implemented on vehicle 30, which includes a display device (e.g., a cathode ray tube or liquid crystal display) for displaying information to the user. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0099] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or middleware components (e.g., application servers), or frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0100] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact via a communication network. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server.

[0101] This embodiment may also include a computer program product, which includes a computer program that, when executed by a processor, implements the environmental information display method provided in any embodiment of the present invention.

[0102] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0103] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A method for displaying environmental information, characterized in that, include: Obtain the vehicle's driving scenario, and based on the driving scenario, obtain the current environment complexity coefficient and the maximum environment complexity coefficient; Obtain the cognitive load associated parameter value, and obtain the driver's cognitive load value based on the cognitive load associated parameter value, the current environmental complexity coefficient, and the maximum environmental complexity coefficient; Based on the cognitive load value, the target rendering mode is obtained, and based on the target rendering mode, the environmental information is rendered and displayed.

2. The method according to claim 1, characterized in that, Obtain the values ​​of cognitive load-related parameters, including: Acquire pupil diameter change rate, skin conductance response value, head micromotion variability, turning deviation error value, and voice stress index value; The driver's cognitive load value is obtained based on the cognitive load correlation parameter value, the current environmental complexity coefficient, and the maximum environmental complexity coefficient, including: The driver's cognitive load value is obtained based on the pupil diameter change rate, skin conductance response value, head micro-motion variability, steering deviation error value, voice stress index value, current environmental complexity coefficient, and maximum environmental complexity coefficient.

3. The method according to claim 2, characterized in that, Obtain the rate of change of pupil diameter, including: The pupil diameter monitoring value of the driver is obtained, and the pupil diameter change rate is calculated based on the pupil diameter monitoring value, as well as the preset mean resting pupil diameter and the baseline standard deviation of pupil diameter.

4. The method according to claim 2, characterized in that, Obtain skin electrical conductivity values, including: The driver's palm skin conductivity value is obtained through the steering wheel capacitive sensor, and the skin conductivity response value is calculated based on the palm skin conductivity value and the pre-calibrated skin conductivity baseline value.

5. The method according to claim 2, characterized in that, Obtain the head micromotion variability, including: The vehicle-mounted millimeter-wave radar acquires the current head sway frequency and current head sway amplitude, and calculates the head micro-motion variability based on the current head sway frequency, the current head sway amplitude, and the pre-calibrated maximum head sway amplitude.

6. The method according to claim 2, characterized in that, Obtain the steering deviation error value, including: The actual steering angle and the expected steering angle are obtained, and the steering deviation error value is calculated based on the actual steering angle, the expected steering angle and the preset maximum steering angle.

7. The method according to claim 2, characterized in that, Obtain the voice stress index value, including: Obtain the fundamental frequency standard deviation, speech rate change rate, and percentage of silence interval duration; The speech stress index value is obtained by using a pre-trained long short-term memory network model based on the fundamental frequency standard deviation, the speech rate change rate, and the proportion of silence interval duration.

8. A vehicle, characterized in that, The vehicles include: At least one processor, and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the method for displaying environmental information according to any one of claims 1-7.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the method for displaying environmental information as described in any one of claims 1-7.

10. A computer program product, characterized in that, It includes a computer program that, when executed by a processor, implements the method for displaying environmental information according to any one of claims 1-7.

Images