Interaction method, display device, storage medium and vehicle

Abstract

The invention relates to the technical field of projection display, in particular to an interaction method, display equipment, a storage medium and a vehicle. When the vehicle-related payment point is identified, the HUD display device triggers the projection of the first virtual image element, and the first virtual image element not only serves as an interaction element for the user to control the vehicle-related payment point, but also can provide verification information, prompt information and the like of the vehicle-related payment point for the user, so that the user performs authorization operation in a credible environment, and the user experience is improved. Therefore, equipment such as a barrier gate in the vehicle-related payment point can be simply controlled, and additional operation is not needed. According to the invention, the safety and reliability of vehicle-related payment are improved, the process is simplified, and the user experience is enhanced.

Description

Technical Field

[0001] The present application relates to the field of projection display technology, and in particular to an interaction method, a display device, a storage medium and a vehicle. Background Art

[0002] With the continuous advancement of automobile electrification, networking and intelligence, users' demands for automobiles are no longer limited to driving experience, but also put forward high requirements for scenario-based applications. In particular, when users drive their vehicles through toll booths or parking lot entrances and exits, they urgently need efficient and safe payment methods to achieve non-contact free-flow charging, reduce the complexity of vehicle-related payments and avoid congestion caused by the payment process. Summary of the invention

[0003] The purpose of this application is to provide an interactive method, a display device, a storage medium and a vehicle, which solve the technical problems in the prior art of complex payment process, low security and reliability, and poor user experience for vehicle-related payments.

[0004] In order to solve the above technical problems, this application adopts the following technical solutions.

[0005] In a first aspect, the present application provides an interaction method, comprising: In response to a vehicle-related payment point being present around the vehicle, the HUD display device projects at least a first virtual image element, where the first virtual image element is projected near the vehicle-related payment point to guide the user to establish a context associated with the vehicle-related payment point; The first virtual image element at least reflects the matching features with the vehicle-related payment point to support the user to authenticate with the vehicle-related payment point, and obtain at least part of the control function of the vehicle-related payment point in response to the authorization operation input by the user.

[0006] In an optional implementation of the first aspect, the first virtual image element also reflects the transaction price information of the vehicle-related payment point.

[0007] According to the above description, an optional implementation method combines the virtual and real aspects of the first virtual image element with the vehicle-related payment point, provides users with intuitive payment entrances and supports one-click operation, establishes a trust connection with users, and optimizes the entire vehicle process and experience of passing through vehicle-related payment points.

[0008] In an optional implementation of the first aspect, the responding to the presence of a vehicle-related payment point around the vehicle includes: The identification identity of the vehicle-related payment point is determined by a camera on the vehicle.

[0009] In an optional implementation of the first aspect, determining the identification of the vehicle-related payment point by using a camera on the vehicle includes: The determination is made in conjunction with a positioning device on the vehicle.

[0010] In an optional implementation of the first aspect, determining the identification of the vehicle-related payment point by using a camera on the vehicle includes: The identification identity of the vehicle-related payment point is determined based on at least one of the position of the vehicle, the orientation of the camera, and the shape, surface texture, and size of the vehicle-related payment point that are collected.

[0011] In an optional implementation of the first aspect, the responding to the presence of a vehicle-related payment point around the vehicle includes: The identification identity of the vehicle-related payment point is determined by communicating with the UWB module on the vehicle and the UWB module on the vehicle-related payment point.

[0012] According to the above description, the optional implementation method can accurately determine the identification identity of the vehicle-related payment point, thereby connecting with the computing device or server corresponding to the identification identity, so as to initiate the payment and control process.

[0013] In an optional implementation of the first aspect, in response to there being a vehicle-related payment point around the vehicle, the HUD display device projects at least a first virtual image element, comprising: The vehicle-related payment point includes a first vehicle-related payment point and a second vehicle-related payment point, and the HUD display device supports projecting a first virtual image element corresponding to the first vehicle-related payment point and a second virtual image element corresponding to the second vehicle-related payment point; In response to the authorization operation input by the user, obtaining at least part of the control function of the vehicle-related payment point includes: Upon receiving the user's authorization operation on the first virtual image element, obtaining at least part of the control function of the first vehicle-related payment point; Upon receiving the user's authorization operation on the second virtual image element, at least part of the control function of the second vehicle-related payment point is acquired.

[0014] In an optional implementation manner of the first aspect, the receiving an authorization operation of the user on the first virtual image element or the receiving an authorization operation of the user on the second virtual image element includes: The first virtual image element or the second virtual image element is selected by default according to the orientation relationship between the vehicle and the first vehicle-related payment point or the second vehicle-related payment point.

[0015] In an optional implementation manner of the first aspect, the receiving an authorization operation of the user on the first virtual image element or the receiving an authorization operation of the user on the second virtual image element includes: The first virtual image element or the second virtual image element is selected according to the switching instruction input by the user.

[0016] According to the above description, an optional implementation supports the simultaneous presentation of multiple vehicle-related payment points around the vehicle, and interacts with the corresponding vehicle-related payment points based on the user's true intention to use the corresponding vehicle-related payment points to eliminate payment ambiguity.

[0017] In an optional implementation of the first aspect, the first virtual image element is projected near the vehicle-related payment point to guide the user to establish a context associated with the vehicle-related payment point, including: When the vehicle-related payment point is directly opposite to the projection area of ​​the HUD display device, the first virtual image element is configured to display an information mark above or beside the vehicle-related payment point according to the position of the vehicle-related payment point; When the vehicle-related payment point deviates from the projection area of ​​the HUD display device, the first virtual image element is configured to display an arrow mark pointing in a specific direction according to the positional relationship between the vehicle and the vehicle-related payment point.

[0018] According to the above description, the optional implementation methods can flexibly adopt different presentation forms according to the virtual-real fit relationship between the projection area and the vehicle-related payment points, giving priority to more intuitive information identification forms to improve comprehensibility and conspicuousness, while getting rid of the limitations of the projection area size on the entire interaction.

[0019] In an optional implementation of the first aspect, the first virtual image element is presented in an animation manner.

[0020] According to the above description, the optional implementation method can improve the user's attention and avoid ignoring the interactive information about the vehicle-related payment point.

[0021] In an optional implementation of the first aspect, in response to there being a vehicle-related payment point around the vehicle, the HUD display device projects at least the first virtual image element, and the interaction method includes: A specific button in the vehicle automatically switches to receive an authorized operation input by the user.

[0022] In an optional implementation of the first aspect, the specific button in the vehicle automatically switches to receive the authorization operation input by the user, including: The specific button includes a play button, a volume button, an answer button or a menu button; In response to there being a vehicle-related payment point around the vehicle, the original function of the specific button is temporarily disabled to receive the authorization operation input by the user.

[0023] In an optional implementation of the first aspect, the specific button in the vehicle automatically switches to receive the authorization operation input by the user, including: The specific key supports receiving a user's selection configuration among a play key, a volume key, an answer key, and a menu key.

[0024] According to the above description, an optional implementation method receives the user's authorization operation through the vehicle-mounted button, ensuring that the user can make the entire interaction process controllable through the last step of the card point operation, thereby improving the user's trust in it and the quick experience of nearby authorization.

[0025] In an optional implementation of the first aspect, in response to the authorization operation input by the user, obtaining at least part of the control function of the vehicle-related payment point includes: Communicate with an external device and receive a user authorization operation forwarded by the external device.

[0026] In an optional implementation of the first aspect, the external device is a mobile phone terminal, and the forwarding authorization operation includes a mobile phone shake operation.

[0027] In an optional implementation manner of the first aspect, the communicating with the external device and receiving the user authorization operation forwarded by the external device includes: The specified amount is automatically deducted from the account bound to the external device.

[0028] According to the above description, an optional implementation method uses a mobile phone or other personal device to issue an authorization operation, making the interaction process such as payment more convenient.

[0029] In an optional implementation of the first aspect, the specific button in the vehicle automatically switches to receive the authorization operation input by the user, including: The specific key supports fingerprint recognition to determine the identity of the user who inputs the authorized operation.

[0030] In an optional implementation of the first aspect, in response to the authorization operation input by the user, obtaining at least part of the control function of the vehicle-related payment point includes: The identity of the user entering the authorized operation is determined by a driver monitoring system in the vehicle.

[0031] In an optional implementation of the first aspect, in response to the authorization operation input by the user, obtaining at least part of the control function of the vehicle-related payment point includes: The account from which the specified amount is deducted is determined based on the identified user identity.

[0032] According to the above description, the optional implementation method can further improve the security of the entire interaction process, and the selection of the deduction account is more flexible.

[0033] In an optional implementation of the first aspect, in response to there being a vehicle-related payment point around the vehicle, the HUD display device projects at least the first virtual image element, and the interaction method includes: The voice assistant in the vehicle automatically wakes up to receive the authorization operation input by the user.

[0034] According to the above description, the optional implementation method can automatically trigger the voice assistant without a wake-up word, which improves the fluency of the voice assistant receiving user authorization operations and provides a better user experience.

[0035] In an optional implementation of the first aspect, the vehicle-related payment point is a gate at a toll station or a parking lot entrance or exit.

[0036] In an optional implementation of the first aspect, in response to the authorization operation input by the user, obtaining at least part of the control function of the vehicle-related payment point includes: The gate is automatically opened to support the passage of the vehicle.

[0037] In an optional implementation of the first aspect, in response to the authorization operation input by the user, obtaining at least part of the control function of the vehicle-related payment point includes: Upon receiving the control operation input by the user, the barrier is closed or opened again.

[0038] In an optional implementation manner of the first aspect, the authorization operation and the control operation share the same key.

[0039] In an optional implementation manner of the first aspect, upon receiving the control operation input by the user, closing the barrier or reopening the barrier comprises: When the barrier is open, other vehicles cannot identify the vehicle-related payment point; When the barrier is closed, other vehicles support identifying the vehicle-related payment point to support the control function of seizing the vehicle-related payment point.

[0040] According to the above description, the optional implementation method supports users to autonomously control switches such as barriers according to the driving conditions of their own vehicles after obtaining control authority of the vehicle-related payment points to prevent other vehicles from illegally passing through.

[0041] In an optional implementation of the first aspect, in response to the authorization operation input by the user, obtaining at least part of the control function of the vehicle-related payment point includes: Automatically deduct the specified amount from the linked account.

[0042] According to the above description, the optional implementation method can meet the scenario of vehicle payment and pass, and improve the smoothness of the payment process.

[0043] In an optional implementation of the first aspect, the matching feature is an identifier that directs payment.

[0044] In an optional implementation of the first aspect, the matching feature is an identifier with the same content symbol as that set for the vehicle-related payment point.

[0045] According to the above description, the optional implementation method supports using simple verification content to assist users in determining the matching of vehicle-related payment points, thereby reducing users' doubts about their accuracy and authenticity.

[0046] In an optional implementation of the first aspect, after acquiring at least part of the control function of the vehicle-related payment point in response to the authorization operation input by the user, the interaction method includes: In response to the vehicle moving away from the vehicle-related payment point, the control function of the vehicle-related payment point is reclaimed.

[0047] In an optional implementation manner of the first aspect, the control function of the vehicle-related payment point being reclaimed includes: The barrier gate is automatically closed to regain control of the passage to the vehicle-related payment point.

[0048] According to the above description, the optional implementation method can reliably support the passage of continuous vehicles before and after the vehicle-related payment point, ensuring smooth handover of control of the vehicle-related payment point.

[0049] In an optional implementation of the first aspect, in response to there being a vehicle-related payment point around the vehicle, the HUD display device projects at least a first virtual image element, comprising: The HUD display device supports the installation of identification programs under unified security authorization.

[0050] In an optional implementation of the first aspect, in response to there being a vehicle-related payment point around the vehicle, the HUD display device projects at least a first virtual image element, comprising: The recognition program installed on the HUD display device is a smart contract added to the blockchain.

[0051] According to the above description, the optional implementation method prevents the illegal installation of programs that implement the interactive method in the HUD display device through security management, resulting in the payment of fees and the control of vehicle-related payment points being exploited by illegal third parties.

[0052] In a second aspect, the present application provides a display device, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor implements the steps of the interaction method described in the first aspect when executing the computer program.

[0053] In a third aspect, the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the interaction method described in the first aspect are implemented.

[0054] In a fourth aspect, the present application provides a vehicle comprising the display device described in the second aspect or the computer-readable storage medium described in the third aspect.

[0055] Compared with the prior art, when the present application identifies a vehicle-related payment point, the HUD display device triggers the projection of a first virtual image element, which not only serves as an interactive element for the user to control the vehicle-related payment point, but also can provide the user with verification information and prompt information of the vehicle-related payment point, so that the user can make an authorization operation in a trusted environment, thereby simply realizing the control of the gate and other equipment in the vehicle-related payment point without additional operations. The present application improves the security and reliability of vehicle-related payment, simplifies the process, and enhances the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] In order to more clearly illustrate the technical solution of the present application, the following is a brief introduction to the drawings required for use in the description of the technical solution. Obviously, the drawings described below are only some examples recorded in the present application, and a person of ordinary skill in the art can obtain other drawings based on these drawings without creative work.

[0057] Figure 1 It is a schematic diagram of the license plate payment scenario in the prior art.

[0058] Figure 2 Schematic diagram of ETC payment scenario in the prior art.

[0059] Figure 3 Schematic diagram of HUD projection display in some examples of this application.

[0060] Figure 4 Schematic diagram of the interaction architecture in some examples of this application.

[0061] Figure 5 Schematic diagrams are presented for the first virtual image element in some examples of the present application.

[0062] Figure 6 This is a schematic diagram of transaction price information in the first virtual image element in some examples of the present application.

[0063] Figure 7 This is a schematic diagram of limited presentation of the first virtual image element in some examples of the present application.

[0064] Figure 8 This is a schematic diagram of the digital matching features in the first virtual image element in some examples of the present application.

[0065] Fig. 9 Schematic diagram of shape matching features in the first virtual image element in some examples of the present application.

[0066] Fig.10 Schematic diagrams are presented for more than two virtual image elements in some examples of the present application.

[0067] Fig.11 This is a schematic diagram showing a wide range of more than two virtual image elements in some examples of this application.

[0068] Fig.12 This is a schematic diagram of key receiving operations in some examples of this application.

[0069] Fig.13 This is a schematic diagram of key operation prompts in some examples of this application.

[0070] Fig.14 This is a schematic diagram of key function switching in some examples of this application.

[0071] Fig.15 This is a schematic diagram of mobile phone receiving operation in some examples of this application.

[0072] Fig.16 Schematic diagram of voice assistant receiving operations in some examples of this application.

[0073] Fig.17 This is a timing diagram of the interaction method in some examples of this application.

[0074] Fig.18 Schematic diagram of HUD display device module in some examples of this application.

[0075] Fig.19 This is a schematic diagram of the composition of the HUD display device in some examples of this application.

[0076] Fig. 20 This is a schematic diagram of projection display in a vehicle in some examples of the present application. DETAILED DESCRIPTION

[0077] The present application will be described in detail below in conjunction with the accompanying drawings, but the content of the description is merely some examples recorded in the present application and does not limit the present application. Changes in structure, method, function, etc. made by ordinary technicians in this field based on these examples are included in the protection scope of the present application.

[0078] It should be noted that the same reference numerals or marks may be used in different examples, but these do not represent an absolute connection in structure or function. In addition, the "first", "second", etc. mentioned in each example are only for the convenience of description, and do not represent an absolute distinction in structure or function, nor can they be understood as indicating or implying the relative importance or the number of corresponding objects. Unless otherwise specified, the "at least one" that may be involved in the description refers to one or more, and "more than one" refers to two or more.

[0079] In addition, when expressing a feature, the character " / " can indicate that there is an or relationship between the front and back associated objects. For example, head-up display / head-up display can be expressed as head-up display or head-up display. When expressing an operation, the character " / " can indicate that there is a division relationship between the front and back associated objects. For example, the magnification M=L / P can be expressed as L (virtual image size) divided by P (image source size). Moreover, the "and / or" in different examples is only to describe the association relationship between the front and back associated objects. This association relationship can include three situations. For example, a concave mirror and / or a convex mirror can be expressed as the existence of a concave mirror alone, the existence of a convex mirror alone, and the existence of both a concave mirror and a convex mirror.

[0080] In traditional vehicle-related payment scenarios, such as when a vehicle passes through a toll booth or a parking lot entrance or exit, payment is collected manually. The disadvantage of this is that the efficiency is very low, and vehicles often have to wait in line. With the advent of electronic technology, there are also some automated payment methods. Among them, license plate payment has gradually become popular in the payment process of highway toll booths, and parking lot payment in cities has become more widely popular. Figure 1 As shown in the figure, the recognition and payment of license plate payment is generally done by automatically identifying the metal license plate on the vehicle through a high-definition camera, and managing the fees according to the account bound to the recognized license plate number. Since metal license plates often have the problem of duplicate license plates, the bound account is often a temporary payment account or a time-based subscription account. Users are not willing to directly bind financial accounts, such as bank cards, Alipay, etc. This will bring many inconveniences to the entire payment process. For example, users need to scan the code to pay every time they leave the parking lot. Figure 2As shown in the figure, this is another ETC (Electronic Toll Collection) payment method, which was first used for toll booths on highways, and has also been tried in some parking lots in cities in recent years. It is mainly implemented through wireless radio frequency communication between the OBU (On board Unit) installed on the windshield of the vehicle and the RSU (Road Side Unit) at the entrance and exit of the toll booth and parking lot, and payment is realized by exchanging information over the air. However, the recognition range of wireless radio frequency communication is relatively large, and the recognition range will fluctuate due to the weather. This sometimes leads to vehicles completing payment at a long distance, while other vehicles may cut in line to get the ETC of the following vehicles. At the same time, the OBU is exposed on the windshield, and vehicles parked outdoors are sometimes stolen by some illegal RSUs, and there are certain hidden dangers in terms of security.

[0081] For the above-mentioned vehicle-related payment methods, the main reason for the security issues is that the vehicle, as the payment subject, is often charged passively by external devices, unlike the personal payment scenario, where the payment process is often initiated by the payment subject. This may also be related to the safety restrictions on the intensity of user intervention during driving. Accordingly, in the process of improving the vehicle-related payment process, it is necessary to give the in-car user the ability to actively authorize, and at the same time, it is also necessary to reduce the overly cumbersome processes similar to personal payment processes. After all, the user is driving and has certain requirements for driving safety. In some examples, an interactive channel between the in-car user and the external scene can be established by projecting the HUD display device in front of the windshield, and the user can issue specific authorization instructions based on the situation of the external scene and his own needs. Among them, HUD projection display mainly uses the optical reflection principle to reflect the imaging light to be displayed through the vehicle windshield into the viewer's eyes. The human eye can see the virtual image information in the opposite direction of the light, and the windshield is used as a display screen to display the vehicle's navigation instructions, vehicle speed, etc. In this example, the virtual-real integration display capability of the HUD display device can be used to mark the interactive information to be paid in the payment scene ahead, thereby improving the efficiency of vehicle-related payments and user trustworthiness, which will be described in detail below. Figure 3As shown, the HUD display device may include at least an optical machine 1, a first reflector 2, a second reflector 3, etc., wherein the optical machine 1 includes a backlight source and an image source (not shown), the backlight source is used to provide illumination light and adjust the brightness of the illumination light according to control, for example, the backlight source may be an LED (Light Emitting Diode), a laser, etc. Under the illumination light provided by the backlight source, the image source adjusts the corresponding display content according to control and projects the display light from the surface of the image source, for example, the image source may be an LCD (Liquid Crystal Display), a DMD (Digital Micromirror Devices), a MEMS (Micro-Electro-Mechanical System) micromirror, a LCOS (Liquid Crystal on Silicon), etc. The first reflector 2 and the second reflector 3 can project the display light projected by the optical machine 1 onto the windshield 4, so as to customize the optical path in a smaller space and meet different projection display requirements. The first reflector 2 and the second reflector 3 can be set as a concave mirror, a convex mirror, a concave lens, a convex lens, etc. according to the requirements of optical planning, and the surface shape of the lens can adopt a free-form surface. Optionally, at least one of the first reflector 2 and the second reflector 3 can also be adjusted to a certain degree of angle, so as to change the projection position of the display light on the windshield 4 to meet the needs of viewers of different heights. The display light of the optical machine 1 is finally reflected on the windshield 4 of the vehicle to form a virtual image 5. When the human eye 6 observes the virtual image 5 facing the windshield 4, it can feel a certain sense of depth, just like watching a real object at a specific distance outside the windshield. The virtual image 5 can be the navigation instruction content, vehicle speed, etc. as described above. It should be added that, according to the characteristics of different optical machines, the HUD display device can also be provided with a diffuser. In some examples, the HUD display device can also include Fresnel lenses, waveguide optical devices, diffractive optical devices, holographic optical devices, tapered optical fibers, etc.

[0082] As mentioned above, as a head-up display device, the advantage of the HUD display device over mobile phones, vehicle-mounted central control screens and other head-down display devices is that it can project the displayed information directly onto the real scene in front. Therefore, it can adopt an interactive method different from mobile phone payment, and build a context of trust through intuitive cooperation with the real scene, so that even a simpler authorization method will be accepted by users. Correspondingly, for vehicle-related payment points such as barriers, it is generally necessary to pay an equivalent amount or rely on specific permissions to achieve control, such as remotely opening the barrier. However, in this example, the information necessary for interaction can be projected near the barrier through the HUD display device. Users can use the matching features associated with the vehicle-related payment points in the projection information to determine whether the vehicle-related payment points meet their needs, allowing users to fully grasp the entire process information, and the last step of the authorization operation is an active choice made by the user based on the projection information, which makes users feel more at ease and more secure and reliable. Figure 4As shown, the interactive method performed by the HUD display device to meet the needs of vehicle-related payment can be divided into two levels, namely the perception layer 932 and the interaction layer 931 above the perception layer 932. Optionally, a middleware layer can be set between the interaction layer 931 and the perception layer 932. The perception layer 932 is responsible for acquiring the external scene and scanning the vehicle-related payment points around the vehicle, such as the gate. It can use the camera to analyze the shape of the gate within the field of view. In order to distinguish the vehicle-related payment points with high similarity, such as multiple toll gates in the toll station, more reference features can be introduced for comparative analysis, such as the surface texture and size of the gate, and the gate of the specific identification identity can also be determined by combining parameters such as position and orientation. Correspondingly, each gate will register its own image features and scene features when entering the network. When identifying, the pre-registered data can be called for comparison to determine the identification identity. Furthermore, the relative position relationship between the gate and the vehicle can be determined based on image analysis, which serves as the basis for the projection information of the HUD display device and the gate to be displayed in conjunction with each other. In addition, the identification of the vehicle-related payment point can also be achieved through the UWB (Ultra Wide Band) module. UWB uses nanosecond non-sinusoidal narrow pulses to transmit data, so it occupies a wide spectrum range. It can support accurate positioning, such as determining the direction and distance of the vehicle-related payment point relative to the vehicle, and can also support data communication between the vehicle-related payment point and the vehicle, such as the vehicle-related payment point can directly send its own identification identity. It should be noted that the module setting in the perception layer 932 is not limited to the above example, and can also include an acceleration sensor to accurately determine the state of the vehicle itself. When the perception layer 932 submits the identified results to the interaction layer 931, the projection module in the interaction layer 931 will control the projection content and projection position of the virtual image element, thereby guiding the user to understand the controllable capabilities around the vehicle, and the payment module will exchange payment information with the corresponding server to complete operations such as amount deduction. Optionally, the interaction layer 931 can also include a communication module for interacting with the vehicle-related payment point for control instructions, as well as security management operations, to ensure that the entire interaction process is protected from illegal use by illegal third parties. In more examples, the interaction layer 931 and the perception layer 932 can be built on the whole formed by the HUD display device and the vehicle computer. For example, the perception layer 932 can reuse the sensor equipment owned by the vehicle computer, and the interaction layer 931 can also reuse the computing resources and communication resources owned by the vehicle computer. In some examples, in order to prevent the entire interaction process from being used by illegal third parties, the installation of the corresponding programs of the interaction layer 931 and the perception layer 932 will also be subject to security rules, especially the payment module will not be illegally deducted, and the scanning camera, UWB module, etc. will not identify illegal vehicle-related payment points.Specifically, the installation of these recognition programs on the HUD display device or the car computer requires the permission of the system, and it is forbidden to install them directly by side loading, such as downloading and installing them only from authorized store programs. In some examples, these recognition programs are smart contracts in the blockchain that have been authenticated by consensus. The released programs cannot be tampered with by illegal third parties, and therefore cannot be illegally used.

[0083] In some examples, such as Figure 5 As shown in the figure, when a vehicle approaches a specific parking lot, the parking lot will manage the passage of the vehicle through the gate. The gate will be recognized by the sensor equipment in the vehicle. For details, please refer to Figure 4 For example, the camera in the vehicle can capture images around the vehicle and determine the barrier with a specific identification identity based on the appearance characteristics of the barrier. Specifically, the corresponding identification identity can be obtained based on the location of the barrier, the unique pattern of the barrier, etc. For example, the unique rust texture on the barrier can uniquely determine the barrier set at a specific location, so that a connection can be established using the registered communication information corresponding to the identification identity. Optionally, corresponding UWB modules are provided on both the barrier and the vehicle for communication between the two. The vehicle can communicate with the surrounding barrier and obtain the identification identity fed back. It can also determine the direction and distance of the barrier from the vehicle based on parameters such as the strength of the UWB signal, so as to serve as coordinate information based on the projection information. In this example, in response to the presence of a barrier around the vehicle, a first virtual image element will be projected through the front projection area 50 of the windshield 4. It can be a sign prompting a charge. The user can know through the sign that the barrier has controllable capabilities and that the payment behavior of the barrier is trustworthy. At the same time, the interaction layer 931 (refer to Figure 4) is in a state of waiting to receive authorization operations and responds to user authorization operations at any time. In this example, since the projection area 50 is in a positive relationship with the barrier to be controlled, the HUD display device has the ability to directly project the first virtual image element on or beside the barrier, such as superimposing it on the actual barrier, and the viewing effect is like a specific identification light on the barrier is lit. Accordingly, when the first virtual image element is projected on or beside the barrier, the user can easily understand that it is information related to the barrier when viewing this projection result, thereby establishing an associated context with the barrier. At the same time, the first virtual image element in this example uses the RMB symbol. As a feature with specific connotations, it can match the payment behavior of the barrier. Further, the matching feature of the RMB symbol can be used exclusively on the parking lot barrier, so that the user can draw a matching conclusion through a simple comparison and verification, and make the next step of whether to authorize. In more examples, when the first virtual image element is projected on or beside the barrier, it will be presented in an animated manner, such as the RMB symbol will rotate, thereby attracting the user's attention, preventing the user from ignoring a specific parking lot entrance and exit, and making a timely decision on whether to enter. Figure 6 As shown, the first virtual image element may also include transaction price information required for passage. In this example, in addition to the barrier symbol as a matching feature, there is also a 5 yuan identification symbol. Accordingly, the transaction price information itself may also be used as a matching feature for user verification. If the toll of this barrier is obviously not a regular price, it will be promptly falsified by the user, and the corresponding authorization operation will not be performed.

[0084] In some examples, such as Figure 7 As shown in FIG. 1 , the FOV (Field of View) of the HUD display device is limited and does not meet the field of view determined by the entire windshield. The projection area 50 is only projected in front of the driver's seat. Accordingly, the projection information can only be projected in front of the left side of the vehicle, while the actual gate is in front of the right side. If the Figure 5 , Figure 6The projection method of the example can only be presented when the vehicle moves to the right to face the barrier. In this way, the vehicle has often arrived at the barrier and the user does not have enough time to issue the authorization operation. The only way to slow down the process is to stop the vehicle, which inevitably affects the user experience. In this example, the first virtual image element adopts a limited presentation form and is also projected near the barrier to establish a context associated with a specific vehicle-related payment point. It at least prompts the user that there are vehicle-related payment points around the vehicle and has certain direction information. At the same time, text content is used to describe the matching features associated with the right front barrier, such as the name of the barrier. Furthermore, an arrow mark can be used to point to the actual position of the barrier. The arrow mark reflects the relative position relationship between the barrier and the vehicle. The direction pointed by the arrow mark will also be dynamically adjusted according to the movement of the vehicle. Figure 4 The perception layer 932 in the projection area 50 provides. Accordingly, the arrow mark points not only include up, down, left, and right, but also can adjust the front and back degrees according to the distance between the vehicle and the barrier, which is specifically reflected by the arrow mark of stereoscopic vision. Optionally, the first virtual image element can also be presented only by an arrow mark, without the above-mentioned text description. An arrow mark represents a barrier, and the direction indicated by the arrow mark can be used as a matching feature. If the direction is consistent with the actual position of the barrier, a certain matching verification effect is achieved. Optionally, the arrow mark can also be presented in a flashing animation to attract the user's attention. In some examples, in the projection area 50, the arrow mark method can not only present the vehicle-related payment points directly in front of the windshield, but also support the vehicle-related payment points on the side or rear of the vehicle through virtual image elements. This also plays a certain role in navigation of vehicle-related payment points, helping users (that is, drivers) to locate undiscovered vehicle-related payment points around the vehicle. In more examples, when a vehicle approaches a specific barrier, the projection area 50 often changes from a direction away from the barrier to a direction facing the barrier. Therefore, when the projection area 50 projects the virtual image element corresponding to the barrier, the projection area 50 may change from a direction away from the barrier to a direction facing the barrier. Figure 7 The example is presented in the form of Figure 5 , Figure 6 The presentation forms in the example are switched, and the information presentation becomes more intuitive. Regardless of the presentation form, it supports receiving authorization operations issued by users.

[0085] In some examples, such as Figure 8As shown, the first virtual image element can also include more matching features, thereby further strengthening the user's trust in the entire interaction process. In this example, each barrier is assigned a unique number (123 in the figure) within a specific range, such as a certain geographical area. This number can represent a specific barrier, and this number can be affixed to the actual barrier. The number information is also included in the projected first virtual image element. Users can judge the credibility of the payment interface by comparing whether the number on the actual barrier is consistent with the projected number. Fig. 9 As shown, the matching feature can also be represented by a symbol of a specific arrangement. The arrangement of this symbol can follow certain rules. Optionally, it can uniquely point to a specific barrier. Similarly, after the barrier is installed, this symbol can be attached to the surface of the barrier and can be seen normally by the human eye. The first virtual image element of the projection also includes the symbol on the barrier. The consistency of the symbol indicates that there is no problem with the recognition and projection, and the user can rest assured to initiate the next step of payment. It should be noted that the matching features used for comparison in the above examples are not limited to digital numbers and arrangement shapes, and can also be presented in other ways. It is preferred to use those symbol that are easier for users to understand, simpler to compare, and uniquely representative for marking.

[0086] In some examples, such as Fig.10As shown, in different scenarios, there may be multiple vehicle-related payment points around the vehicle. Taking the toll station as an example, there are multiple toll gates in front of the vehicle. Generally, as the driver, the user will select the corresponding toll gate according to the queue situation of different toll gates when the vehicle is constantly approaching the toll station. Accordingly, when issuing the authorization operation, it is necessary to distinguish different vehicle-related payment points (i.e., gates). In this example, the front of the windshield 4 includes the gate T1 on the left and the gate T2 on the right. Different gates correspond to different toll gates. The first virtual image element corresponding to the gate T1 and the second virtual image element corresponding to the gate T2 are projected in the projection area 50. Since the gate T1 is opposite to the projection area 50, the first virtual image element uses a more intuitive information logo. Projecting it next to the gate T1 allows the user to understand at a glance that the gate T1 supports interactive control. At the same time, the barrier T2 deviates from the projection area 50, and the second virtual image element cannot be directly projected on or beside the barrier T2. Therefore, the second virtual image element can be marked with an arrow at the right edge of the projection area 50, and the pointing position is facing the actual barrier T2. In some examples, no matter what form the first virtual image element and the second virtual image element are in, when they are displayed at the same time, they will have a default virtual image element as the control focus, that is, if the user issues an authorization operation, it is determined to control the default virtual image element. Optionally, the default virtual image element can identify the virtual image element that is projected first, or the virtual image element corresponding to the vehicle-related payment point that the vehicle is facing most directly can be identified as the default. In more examples, input devices such as buttons in the vehicle can also be used to switch the focus of control, which can flexibly meet the specific needs of users. For example Fig.11 As shown, the projection area 50 is a large FOV, relative to Fig.10 For example, the projection area 50 is not only facing the barrier T1, but also facing the barrier T2, so the first virtual image element and the second virtual image element are both intuitively marked with information, and the user can find two controllable barriers at a glance. The first virtual image element and the second virtual image element are like settings that are integrated with the barriers. Optionally, the first virtual image element as the default control focus can be marked in an outer circle frame, and the user can choose whether to directly control the barrier T1 according to actual needs. If the user needs to switch to the second virtual image element corresponding to the barrier T2, the input device can also be used to switch, and accordingly, the outer circle will also be framed from the first virtual image element to the second virtual image element.

[0087] As described above, once the virtual image element corresponding to the vehicle-related payment point is projected in the projection area 50, it indicates that the user has the ability to take over the corresponding vehicle-related payment point at any time. At this time, an input device provided to the user for operation can be configured to receive the authorization operation issued by the user. Specifically, one-key operation can be supported to simplify the interaction process. Optionally, when there are multiple virtual image elements at the same time, that is, there are multiple vehicle-related payment points around the vehicle, the user will only obtain the control of the vehicle-related payment point in the control focus when issuing the authorization operation, and other virtual image elements may disappear after the authorization operation is issued, and will not respond. In some examples, a button for receiving authorization operations can be set separately in the vehicle. Preferably, it can be configured on the steering wheel for the user to press while driving the vehicle. Further, a fingerprint recognition function can be set on the button, and the authorization operation will only be issued in response to the pressing of a user with a specific identity, or the pressing of users with different identities will trigger payment of accounts bound to different users. Fig.12 As shown, button 933 can be an original play button, volume button, answer button or menu button on the steering wheel, etc., which has entertainment functions. Button 933 can serve the original entertainment operation under normal conditions. However, once it is recognized that there are vehicle-related payment points around the vehicle and there are corresponding virtual image elements in the projection area 50, button 933 will switch its function and no longer respond to the entertainment function, that is, the original function will be temporarily closed. If the user operates button 933 after the switch, it will be recognized that the user has entered an authorized operation, which not only reduces the occupation of the button, but also supports multiple functions with one button without unnecessary operations. In a specific example, one of the multiplexing play button, volume button, answer button or menu button can support flexible configuration by the user. After the user configures it, once the vehicle-related payment point is identified, the configured button can support receiving the authorized operation at any time, while other unconfigured buttons still maintain the original input function.

[0088] Specifically, when the button 933 switches the function input according to the changes in recognition and projection, it will also display corresponding prompt information in the projection area 50, such as "Press the volume button to trigger payment". Under the guidance of the prompt information, the user triggers the authorization operation, which also eliminates the confusion of operation under key reuse and improves the user experience. Correspondingly, the object pressed in the prompt information can also be adaptively adjusted according to the actual configuration. Fig.13 As shown, the prompt information is compared with Fig.12The example is not directly displayed in the projection area 50, but is separated from the virtual image elements in the projection area 50, so as to reduce the interference of the indication and matching of the vehicle-related payment points. Specifically, the HUD display device supports the projection of the first focal plane and the second focal plane. The first focal plane may be the range corresponding to the projection area 50, and its virtual image distance is farther, such as about 10 meters, which is suitable for fitting display with the real object, while the second focal plane may be the range corresponding to the projection area 51, and its virtual image distance is closer. In this example, the prompt information is displayed in the projection area 51. Since this area normally displays driving parameters such as vehicle speed, the user can see the payment-related prompt information while viewing this information. The information distribution is more organized, and there is no serious out-of-focus situation when the user sees it with peripheral vision, so it is easier to view. In more examples, the prompt information can also be displayed in the central control screen or other locations, or the user driving the vehicle can be reminded by voice prompts.

[0089] As described above, after the user sees the first virtual image element and the prompt information, if he really needs to pass through the corresponding vehicle-related payment point, such as entering from the entrance of the parking lot where the corresponding barrier is located, he can press the above-mentioned button. Taking the barrier as an example, in response to the user authorization operation input by the pressing behavior, the control function of the barrier is obtained, and the barrier can be opened under triggering. If the vehicle enters from the entrance of the parking lot, the barrier will be automatically closed, that is, the control right of the barrier can be taken back. In a specific example, between obtaining the authorization operation and obtaining the control function, the user's account will also be operated. For example, at the entrance of the parking lot, the time of vehicle entry is recorded, and at the exit of the parking lot or the toll station, the specified amount can be deducted. Accordingly, the user's account can be bound to a financial account such as a bank card or Alipay, so direct deduction can be supported. In this example, in the entire interactive process, only one step of authorization operation is retained for user control. This control only relies on the convenient input device around the user to perform one-key operation, and the previous and subsequent recognition, payment and other actions can be automatically executed, so it not only optimizes the experience but also ensures safety, and improves the user's ability to be informed and autonomy. In some examples, in addition to opening the barrier, the user can also obtain full control of the barrier for a period of time, that is, the user can close the barrier again by inputting a control operation with a specific key. When the vehicle arrives at the barrier, the user can input a control operation again to open the barrier, which can effectively prevent vehicles from cutting in line from passing through. Optionally, the key for inputting the authorization operation and the control operation can be the same, such as Fig.14As shown, the button can be an entertainment function button on the steering wheel. As mentioned above, it can be a function button such as audio adjustment and phone answering. Under normal circumstances, this button receives the original operation of implementing the above entertainment function. When the gate is detected, the button will be switched to receive the authorization operation, and correspondingly used to trigger the payment of fees, opening the gate and other actions. Optionally, when the button responds to the detection of the gate, it will only switch for the vehicle-related payment point detected in front of the vehicle, and the vehicle-related payment points sensed on both sides or the rear will not switch, so as to avoid the problem that the original function of the button is occupied and cannot be used for a long time. When the user inputs the authorization operation to trigger the acquisition of the gate control right, the button is switched to receive the control operation, that is, the button becomes the remote switch of the corresponding gate. The user can operate the button to trigger the gate to close when the gate is open, and operate the button to trigger the gate to open when the gate is closed. When the vehicle leaves the gate, the control function of the gate will be withdrawn, and the user cannot control the gate by pressing the button, and the button will be switched to receive the original operation, that is, if it is an audio key, it will still be restored to adjust the volume of the playback, which greatly improves the utilization rate of the button and the convenience for the user.

[0090] Furthermore, in order to prevent users from abusing the control of vehicle-related payment points, taking the barrier as an example, if the user does not pass through the barrier for a long time after obtaining the control of the barrier, it will inevitably cause queuing problems, which is inconsistent with the congestion caused by slow charging. It is caused by the users in the vehicle themselves. Therefore, the control of the barrier is not unlimited when it is temporarily transferred. In some examples, the control of a specific vehicle can be timed. If the time exceeds a certain threshold, it will automatically trigger the revocation of its control. Accordingly, the barrier will be automatically closed when the vehicle does not pass through the barrier. At the same time, if other vehicles issue authorization operations, their control will be handed over to other vehicles. In some examples, although the front vehicle obtains the control right of the barrier by issuing the authorization operation, the rear vehicle is also around the barrier, and the corresponding virtual image element will be displayed in the projection area corresponding to the rear vehicle. At this time, the user of the rear vehicle can determine whether he needs to seize the control right of the barrier according to the moving progress of the front vehicle. The method is no different from the ordinary authorization operation, which can be achieved by pressing a specific button, and the barrier will be triggered to unconditionally transfer the control right to the rear vehicle. The front vehicle cannot control the opening and closing of the barrier through operation. Optionally, in order to avoid the damage of the barrier to the passing vehicles, when the barrier is opened, the vehicles other than those occupying the control right cannot recognize its barrier. It can be that although the vehicle actively obtains the relevant information of the barrier through the sensor, it cannot establish a connection with the barrier, and the corresponding virtual image element will not be displayed in the corresponding projection area, and its control button cannot respond to the authorization operation input by the user. Only when the barrier is closed can other vehicles identify the barrier and display virtual image elements in the projection area to guide users in other vehicles to issue authorization operations. After responding to the authorization operations of other vehicles, the barrier will forcibly transfer control to the vehicle that has occupied the rear, so as to urge the vehicle in front to make a passage decision as soon as possible.

[0091] In some examples, such as Fig.15As shown, receiving the authorization operation or control operation input by the user is not limited to the buttons in the vehicle, but can also be an external device connected to the vehicle. Taking the mobile phone 934 as an example, it is generally strongly bound to a specific user. Through the mobile phone 934, not only can the corresponding authorization operation or control operation be received, but also the identity of the user who issues these operations can be obtained, and it can be bound to the payment account, etc., to realize payment by specific users. In this example, the logic of its recognition and projection is consistent with the above example. When the user sees the virtual image element representing the specific vehicle-related payment point in the projection area, the authorization operation can be issued by the mobile phone 934. The mobile phone 934 here can be carried by the user and automatically connected to the HUD display device or the car machine when getting on the car. Accordingly, when the corresponding virtual image element appears in the projection area, the HUD display device or the car machine can send a message notification to the connected mobile phone 934, telling the mobile phone 934 that if a specific operation of the mobile phone is received during this period, it can be sent to the HUD display device or the car machine as an authorization operation or a control operation, such as shaking. For users in the driving process, similar operations are relatively convenient for users to perform. In response to the user's shake operation, at least some of the control functions of the corresponding vehicle-related payment point can be obtained, such as automatically opening the gate in the above example. If payment is involved, the bound account can be determined based on the identity of the mobile phone user and the fee can be deducted. The user can also close the opened gate by shaking the mobile phone 934 again. When the vehicle drives to the gate, the mobile phone 934 can be shaken again to open the gate again. When the vehicle passes through the gate, the control of the gate will be taken back. At this time, the mobile phone 934 can be notified that there is no need to feedback the shaking action. Even if the user shakes the mobile phone 934, it will not initiate any control behavior on the gate. If some functions of the mobile phone 934 itself are triggered by shaking, it will also restore to the original triggering behavior.

[0092] In some examples, such as Fig.16 As shown, the authorization operation or control operation that receives user input may also be the voice assistant in the vehicle. Accordingly, when the vehicle-related payment points around the vehicle are identified, a corresponding virtual image element (the word "entrance") will appear in the projection area 50. At the same time, the voice assistant in the vehicle will be automatically turned on, and there is no need to use a wake-up word to trigger it. In order to prompt the user that the voice assistant has been turned on, the authorization instruction can be directly issued by voice command, and the logo that the voice assistant is triggered can be displayed in the projection area 50. Optionally, this prompt that the triggered logo can also be displayed on the central control screen 52 of the vehicle. The authorization operation is also received through the voice assistant capability of the vehicle computer, and the user's voice command is obtained through the vehicle computer. In more examples, the triggered logo can also refer to Fig.13The example is shown in projection area 51. Similarly, after receiving the user's authorization operation through voice input, the gate can be opened, etc. At this time, the voice assistant does not need to be turned off and will still receive the user's voice command, that is, it can further respond to the user's control command to continuously control the opening and closing of the gate. When the vehicle leaves the vehicle payment point and the control is taken back, the voice assistant is turned off. If the voice assistant is used normally, it needs to be triggered by the wake-up word. In addition, the mechanism of preemption through external devices and voice assistants is the same as Figure 12-14 The examples are similar and will not be repeated here.

[0093] like Fig.17 As shown, for the process of HUD display device assisting payment interaction, it can be a scene where the vehicle arrives at the vehicle-related payment point such as the barrier together with the vehicle. The HUD will identify the barrier. Optionally, the specific identification scan is achieved through the vehicle's own sensor equipment. Optionally, if the characteristic information is exchanged through UWB, etc., the characteristic information will also be encrypted to reduce interception by illegal third parties. Accordingly, when the HUD obtains the identification identity of the barrier, it can be compared with the comparison data owned by the HUD itself. Preferably, the identified characteristic information is sent to the operation server registered with the barrier. The operation server has the information registered after the installation of the specified barrier, so as to perform comparison and identification. When the HUD initiates a comparison with the operation server, it will also initiate an identity authentication to the operation server. The operation server determines whether the HUD initiating the comparison is a legal device. Only after it is determined to be legal will the comparison result be returned to the HUD, such as the identification identity of the barrier and the account for receiving the payment. After receiving the comparison result, the HUD will project a virtual image element to fit it relatively according to the position of the barrier. For details, please refer to Figure 5-Figure 11 , waiting for the user's authorization. For details on receiving the user's authorization operation, please refer to Figure 12-16. When the user issues the authorization operation, if payment is involved, a deduction request will be initiated to the bank bound to the HUD. The payment account can be determined based on the account bound to the HUD, or based on the user's fingerprint or mobile phone owner in the above example. In more examples, the bound account can also be determined by scanning the identity of the driver's seat user through the vehicle's integrated driver monitoring system. Correspondingly, the recipient account for the transfer can be determined based on the identification of the gate, and this information can be the existing information registered by the gate on the operation server. It should be noted that initiating a payment is not limited to the deduction of a monetary amount, but can also be the consumption of certain tokens or membership rights. After the deduction is completed, the payment completion message can be sent to the HUD and the operation server corresponding to the gate respectively. The operation server will also send a payment timestamp to the corresponding gate. The gate can transfer the control right to the HUD for control based on the payment timestamp. As mentioned above, a certain control time range can be set. Once the HUD obtains the control right of the gate, a control virtual connection will be established between the HUD and the gate. This virtual connection can ensure the control interaction between the two. The user can issue a control operation through the input device on the vehicle, and the gate can also respond to its operation to open or close. When the vehicle leaves the barrier, the virtual control connection it established will also be released, that is, the control of the barrier will be taken back.

[0094] When the HUD display device implementing the interactive method in the above example is applied to the vehicle, the HUD display device can be used as a payment auxiliary terminal of the vehicle to establish a convenient channel between the user in the vehicle and the vehicle-related payment point outside the vehicle. The user can control the vehicle-related payment point through a simple authorization operation to pass smoothly. Fig.18 As shown, the HUD display device integrated in the vehicle can be powered and data by the vehicle computer 92, or the HUD display device itself can provide power and generate data. The HUD display device can specifically include a processor 91, an Ethernet interface 901, a CAN (Controller Area Network) interface 902, a power management module 903, a running memory 904, a storage memory 905, a temperature detection 906, a motor 907, a backlight source 908, an image source 909, a positioning module 910, a radar 911, a camera 912, etc. It should be noted that Fig.18 The modules listed in the description are merely exemplary and do not constitute any limitation. In some examples, the HUD display device may further include other modules. In addition, the above modules may be implemented in one or more hardware in different examples, or a single module may be implemented by a combination of multiple hardware.

[0095] The processor 91 serves as the control center of the HUD display device, and includes one or more processing units of any type, including but not limited to a microcontroller, a microcontroller, a DSP (Digital Signal Processor) or any combination thereof. The processor 91 is used to generate operation control signals according to a computer program, realize the control of other modules, and cooperate with corresponding modules to process the acquired or inherent data, instructions, etc.

[0096] The Ethernet interface 901 is a network data connection port for local area network communication, which defines a series of software and hardware standards. Multiple electronic devices can be connected together through the Ethernet interface 901. In this example, the processor 91 can exchange information with the vehicle computer 92 through the Ethernet interface 901, such as sending data to the vehicle computer 92 or receiving data sent by the vehicle computer 92.

[0097] The CAN interface 902 is a network data connection port of the controller area network, which provides a standard bus for the control system and embedded industrial control inside the car, and realizes the communication interaction between the control nodes. In this example, the processor 91 can also exchange information with the vehicle computer 92 through the CAN interface 902. Optionally, the processor 91 can also connect to other external devices through the CAN interface 902. In some examples, the processor 91 can also be provided with a GPIO (General-purpose input / output) interface to improve the compatibility of peripheral connections.

[0098] The power management module 903 is connected to the vehicle computer 92 and can receive the power provided by the vehicle computer 92 to provide a regulated power supply for each module of the HUD display device, thereby ensuring that the processor 91 and each module operate under normal voltage supply and avoid damage due to overvoltage.

[0099] The running memory 904 is used to store computer programs executed by the processor 91, temporarily stored calculation data, data exchanged with the storage memory, etc. The running memory 904 can be a memory such as SDRAM (Synchronous Dynamic Random-access Memory).

[0100] The storage memory 905 is used to store resources such as display content related to the HUD display device, and long-term storage of running programs and data, etc. The storage memory 905 can be a memory such as Flash (flash memory). In some examples, the processor 91 can also provide an interface to access an external memory.

[0101] Temperature detection 906 is used to detect the temperature inside the HUD display device, and may specifically include several temperature sensors. Since the resistance value of the temperature sensor changes with the change of temperature, the processor 91 can determine the resistance value of the temperature sensor at the corresponding temperature according to the voltage change between each temperature sensor and the voltage divider resistor under a fixed power supply voltage, thereby reversely deducing the temperature at the location of the temperature sensor. In some examples, the processor 91 can control several temperature sensors through the GPIO interface, and several temperature sensors can be set at different locations inside the HUD display device. The processor 91 can use time-sharing detection to obtain the temperature values ​​fed back by several temperature sensors respectively.

[0102] Motor 907 is used to drive the optical lens in the HUD display device to rotate under the control of processor 91, so as to achieve the change of the corresponding optical path. For example, when the backflow of sunlight causes the temperature rise on the surface of the image source, the motor can be used to drive the optical lens to prevent the external sunlight from reaching the surface of the image source. In some examples, the processor 91 can also drive the fan provided on the HUD display device through motor 907 to increase the speed of air exchange between the inside and outside of the HUD display device to achieve heat dissipation. Specifically, the motor 907 is connected to the processor 91 through a motor driver chip, and the motor driver chip provides high-performance power output for the motor 907, and can also communicate and control with the processor 91 through interfaces such as SPI (Serial Peripheral Interface).

[0103] The backlight source 908 is used to provide illumination light and adjust the brightness of the illumination light according to the control of the processor 91, so as to adjust the projection display brightness of the entire HUD display device. The backlight source 908 cooperates with the image source 909 to realize the main function of optical machine projection display. The backlight source 908 can be an LED (Light Emitting Diode), a laser, etc. Specifically, the backlight source 908 is connected to the processor 91 through a backlight driver chip, and the backlight driver chip provides a driving voltage for the backlight source 908 and controls the brightness of the backlight source 908 under the pulse width signal output by the processor 91.

[0104] The image source 909 is used to display an image of corresponding content and project the display light corresponding to the image according to the control of the processor 91. The image source 909 can be an LCD (Liquid Crystal Display), a DMD (Digital Micromirror Devices), a MEMS (Micro-Electro-Mechanical System) micromirror, a LCOS (Liquid Crystal on silicon), etc.

[0105] The positioning module 910 is used to monitor the position of the HUD display device and the corresponding vehicle. The positioning module 910 can be a global navigation satellite system such as GPS (Global Positioning System) and Beidou Satellite Navigation System. By measuring the distance between the satellite and the receiver on the positioning module 910 at different positions, the corresponding position and orientation data are determined. In some examples, the positioning module 910 may also include an inertial navigation system, which is based on Newton's laws of mechanics. By measuring the acceleration of the positioning module 910 in the inertial reference system, integrating it over time, and transforming it into the navigation coordinate system, the speed, yaw angle, and position data in the navigation coordinate system are obtained. Optionally, the inertial navigation system can assist the global navigation satellite system to achieve more accurate positioning and provide the processor 91 with corresponding position information.

[0106] The radar 911 is used to determine the position of a target object through electromagnetic waves, and can usually determine the distance of the target object from the vehicle where the radar 911 is located.

[0107] Camera 912 includes a vehicle body camera and an in-vehicle camera, wherein the vehicle body camera is used to determine the position of the target object through visual recognition, and the vehicle body camera can be a monocular camera or a binocular camera. The biggest difference between a monocular camera and a binocular camera is that a binocular camera can capture images from two different perspectives, thereby obtaining distance information in three-dimensional space. The in-vehicle camera is used to identify the behavior status of the driver and passengers in the vehicle, including fatigue detection, distraction detection, expression recognition, gesture recognition, sight tracking, etc. In this example, the in-vehicle camera can also implement eye tracking.

[0108] In some examples, the positioning module 910, radar 911 and camera 912 can also be directly connected to the vehicle computer 92, and are not directly connected to the processor 91 of the HUD display device. For example, the vehicle computer 92 itself is integrated with a positioning module for position tracking and a radar and camera for automatic driving. The HUD display device can obtain the collected data of the positioning module, radar and camera in real time through communication with the vehicle computer 92.

[0109] In some examples, such as Fig.19As shown, the display device implementing the above-mentioned interaction method may specifically include a processor 81, a memory 82, an input device 83 and an output device 84, wherein the input device 83 may include an operation button connected to the display device, etc., and the display device may receive input control instructions and data through the input device 83. The output device 84 may include an image source integrated on the display device, etc., and the display device may output corresponding instructions or data to the output device 84. Further, the memory 82 stores a computer program running on the processor 81, and the processor 81 implements the interaction method of the above example when executing the computer program. In some examples, a computer-readable storage medium, a computer-readable storage medium storing a computer program, and the computer program implements the interaction method of the above example when executed by the processor.

[0110] like Fig. 20 As shown, the vehicle can be provided with the above-mentioned HUD display device. Specifically, the HUD display device is integrated inside the center console 10, such as in front of the steering wheel. The corresponding display light is projected onto the opposite vehicle windshield 4 through the projection window 102 of the HUD display device. The effect of the viewer observing the front of the windshield 4 from the cockpit is that the virtual image in the projection area 50 can be directly seen. In addition to basic information such as vehicle speed, the virtual image content can also respond to the vehicle-related payment points around the vehicle to project interactive information to assist users in payment and passage, so that users can trust the entire interactive process and compress the complexity of the process. In some examples, the vehicle can also distribute the program of the interactive method in the above example through the computer-readable storage medium of the above example, so that the vehicle-mounted HUD display device can be conveniently updated and upgraded. It should be noted that the above-mentioned vehicle is not limited to a car as a means of transportation, but can also include a bus, a truck, an excavator, a motorcycle, a train, a high-speed rail, a ship, a yacht, an airplane, a spacecraft, etc. The projected windshield is not limited to the front windshield of the car, but can also be a transparent surface at other positions.

[0111] In combination with the above examples, the technical solutions involved in the present application can be directly embodied as hardware, a software module executed by a control unit, or a combination of the two, that is, one or more steps and / or one or more step combinations, which can correspond to various software modules of a computer program flow, or to various hardware modules, such as ASIC (Application Specific Integrated Circuit), FPGA (Field-Programmable Gate Array) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or any appropriate combination thereof. For the convenience of description, the above description is divided into various modules and described separately according to their functions. Of course, when implementing the present application, the functions of each module can be implemented in the same or more software and / or hardware.

[0112] Through the description of the above examples, those skilled in the art can clearly understand that the present application can be implemented by means of software plus a necessary general hardware platform. Based on this understanding, the technical solution involved in the present application can be essentially or partly contributed to the prior art in the form of a software product. The software is executed by a microcontroller unit and, depending on the required configuration, may include one or more microcontroller units of any type, including but not limited to a microcontroller unit, a microcontroller, a DSP (Digital Signal Processor) or any combination thereof. The software is stored in a memory, such as a volatile memory (such as a random access memory, etc.), a non-volatile memory (such as a read-only memory, a flash memory, etc.) or any combination thereof.

[0113] In summary, when the vehicle-related payment point is identified, the HUD display device triggers the projection of the first virtual image element, which not only serves as an interactive element for the user to control the vehicle-related payment point, but also can provide the user with verification information and prompt information of the vehicle-related payment point, so that the user can make authorization operations in a trusted environment, thereby simply realizing the control of the gate and other equipment in the vehicle-related payment point without additional operations. This application improves the security and reliability of vehicle-related payment, simplifies the process, and enhances the user experience.

[0114] It should be understood that although this specification includes some examples, none of these examples contains only one independent technical solution, and this description of the specification is only for the purpose of clarity. A person skilled in the art should take the specification as a whole, and the technical solutions in each example can also be appropriately combined to form other implementation methods that can be understood by a person skilled in the art.

[0115] The series of detailed descriptions listed above are only specific descriptions of feasible implementation methods of the present application. They are not intended to limit the scope of protection of the present application. All equivalent implementation methods or modifications that do not deviate from the teachings of the present application should be included in the scope of protection of the present application.

Claims

1. An interactive method, characterized in that: include: In response to a vehicle-related payment point being present around the vehicle, the HUD display device projects at least a first virtual image element, where the first virtual image element is projected near the vehicle-related payment point to guide the user to establish a context associated with the vehicle-related payment point; The first virtual image element at least reflects the matching features with the vehicle-related payment point to support the user to authenticate with the vehicle-related payment point, and obtain at least part of the control function of the vehicle-related payment point in response to the authorization operation input by the user.

2. The interactive method according to claim 1, characterized in that: In response to there being a vehicle-related payment point around the vehicle, the HUD display device projects at least a first virtual image element, including: The vehicle-related payment point includes a first vehicle-related payment point and a second vehicle-related payment point, and the HUD display device supports projecting a first virtual image element corresponding to the first vehicle-related payment point and a second virtual image element corresponding to the second vehicle-related payment point; In response to the authorization operation input by the user, obtaining at least part of the control function of the vehicle-related payment point includes: Upon receiving the user's authorization operation on the first virtual image element, obtaining at least part of the control function of the first vehicle-related payment point; Upon receiving the user's authorization operation on the second virtual image element, at least part of the control function of the second vehicle-related payment point is acquired.

3. The interactive method according to claim 1, characterized in that: The first virtual image element is projected near the vehicle-related payment point to guide the user to establish a context associated with the vehicle-related payment point, including: When the vehicle-related payment point is directly opposite to the projection area of ​​the HUD display device, the first virtual image element is configured to display an information mark above or beside the vehicle-related payment point according to the position of the vehicle-related payment point; When the vehicle-related payment point deviates from the projection area of ​​the HUD display device, the first virtual image element is configured to display an arrow mark pointing in a specific direction according to the positional relationship between the vehicle and the vehicle-related payment point.

4. The interactive method according to claim 1, characterized in that: In response to there being a vehicle-related payment point around the vehicle, the HUD display device projects at least the first virtual image element, and the interaction method includes: A specific button in the vehicle automatically switches to receive an authorized operation of the user input.

5. The interactive method according to claim 1, characterized in that: In response to there being a vehicle-related payment point around the vehicle, the HUD display device projects at least the first virtual image element, and the interaction method includes: The voice assistant in the vehicle automatically wakes up to receive the authorization operation input by the user.

6. The interactive method according to claim 1, characterized in that: In response to the authorization operation input by the user, obtaining at least part of the control function of the vehicle-related payment point includes: The gate is automatically opened to support the passage of the vehicle.

7. The interactive method according to claim 1, characterized in that: In response to the authorization operation input by the user, obtaining at least part of the control function of the vehicle-related payment point includes: Automatically deduct the specified amount from the linked account.

8. A display device, characterized in that: The method comprises a memory, a processor and a computer program stored in the memory and running on the processor, wherein the processor implements the steps of the interaction method according to any one of claims 1 to 7 when executing the computer program.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the interaction method according to any one of claims 1 to 7 are implemented.

10. A means of transport, characterized in that: Includes the display device of claim 8 or the computer-readable storage medium of claim 9.

Images