Interaction method, display device, storage medium and vehicle

Abstract

The present application relates to the field of projection display technology, and in particular to an interactive method, display device, storage medium, and vehicle. When a vehicle-related payment point is identified, the present application triggers the projection of a first virtual image element by the HUD display device. The first virtual image element not only serves as an interactive element for the user to control the vehicle-related payment point, but also can provide the user with verification information and prompt information of the vehicle-related payment point, so that the user can make authorization operations in a trusted environment, thereby simply realizing the control of the gate and other equipment in the vehicle-related payment point without the need for additional operations. The present application improves the security and reliability of vehicle-related payments, simplifies the process, and enhances the user experience.

Description

Technical Field

[0001] The present application relates to the field of projection display technology, and in particular to an interaction method, a display device, a storage medium, and a vehicle. Background Art

[0002] With the continuous advancement of electric, connected, and intelligent vehicles, users' demands for vehicles are no longer limited to the driving experience, but also place high demands on scenario-based applications. In particular, when driving through toll booths or parking lot entrances and exits, users urgently need efficient and secure payment methods to achieve contactless free-flow toll collection, reduce the complexity of vehicle-related payments, and avoid congestion during the payment process. Summary of the Invention

[0003] The purpose of this application is to provide an interactive method, a display device, a storage medium and a vehicle, which solve the technical problems in the prior art of vehicle-related payment, such as the complicated payment process, low security and reliability, and poor user experience.

[0004] In order to solve the above technical problems, this application adopts the following technical solutions.

[0005] In a first aspect, the present application provides an interaction method, comprising:

[0006] In response to a vehicle-related payment point being located around the vehicle, the HUD display device projects at least a first virtual image element, where the first virtual image element is projected near the vehicle-related payment point to guide the user to establish a context associated with the vehicle-related payment point;

[0007] The first virtual image element at least reflects the matching features with the vehicle-related payment point to support the user to authenticate with the vehicle-related payment point, and obtain at least part of the control functions of the vehicle-related payment point in response to the authorization operation input by the user.

[0008] In an optional implementation of the first aspect, the first virtual image element further reflects transaction price information of the vehicle-related payment point.

[0009] According to the above description, the optional implementation method combines the virtual and real aspects of the first virtual image element with the vehicle-related payment point, provides users with an intuitive payment entrance and supports one-click operation, establishes a trust connection with the user, and optimizes the entire vehicle's process and experience of passing through the vehicle-related payment point.

[0010] In an optional implementation of the first aspect, the responding to the presence of a vehicle-related payment point around the vehicle includes:

[0011] The identification identity of the vehicle-related payment point is determined by a camera on the vehicle.

[0012] In an optional implementation of the first aspect, determining the identification of the vehicle-related payment point by a camera on the vehicle includes:

[0013] The determination is performed in conjunction with a positioning device on the vehicle.

[0014] In an optional implementation of the first aspect, determining the identification of the vehicle-related payment point by a camera on the vehicle includes:

[0015] The identification identity of the vehicle-related payment point is determined based on at least one of the position of the vehicle, the orientation of the camera, and the shape, surface texture, and size of the vehicle-related payment point collected.

[0016] In an optional implementation of the first aspect, the responding to the presence of a vehicle-related payment point around the vehicle includes:

[0017] The identification identity of the vehicle-related payment point is determined by communicating with the UWB module on the vehicle and the UWB module on the vehicle-related payment point.

[0018] According to the above description, the optional implementation method can accurately determine the identification identity of the vehicle-related payment point, thereby connecting with the computing device or server corresponding to the identification identity, so as to initiate the payment and control process.

[0019] In an optional implementation of the first aspect, in response to there being a vehicle-related payment point around the vehicle, the HUD display device projecting at least a first virtual image element includes:

[0020] The vehicle-related payment point includes a first vehicle-related payment point and a second vehicle-related payment point, and the HUD display device supports projecting a first virtual image element corresponding to the first vehicle-related payment point and a second virtual image element corresponding to the second vehicle-related payment point;

[0021] In response to the authorization operation input by the user, obtaining at least part of the control function of the vehicle-related payment point includes:

[0022] Upon receiving the user's authorization operation on the first virtual image element, obtaining at least part of the control functions of the first vehicle-related payment point;

[0023] Upon receiving the user's authorization operation on the second virtual image element, at least part of the control function of the second vehicle-related payment point is acquired.

[0024] In an optional implementation of the first aspect, the receiving the user's authorization operation on the first virtual image element or the receiving the user's authorization operation on the second virtual image element includes:

[0025] The first virtual image element or the second virtual image element is selected by default according to the orientation relationship between the vehicle and the first vehicle-related payment point or the second vehicle-related payment point.

[0026] In an optional implementation of the first aspect, the receiving the user's authorization operation on the first virtual image element or the receiving the user's authorization operation on the second virtual image element includes:

[0027] The first virtual image element or the second virtual image element is selected according to the switching instruction input by the user.

[0028] According to the above description, an optional implementation method supports the simultaneous presentation of multiple vehicle-related payment points around the vehicle, and interacts with the corresponding vehicle-related payment points based on the user's true intention to use the corresponding vehicle-related payment points to eliminate payment ambiguity.

[0029] In an optional implementation of the first aspect, the first virtual image element is projected near the vehicle-related payment point to guide the user to establish a context associated with the vehicle-related payment point, including:

[0030] When the vehicle-related payment point is facing the projection area of ​​the HUD display device, the first virtual image element is configured to display an information mark above or next to the vehicle-related payment point according to the position of the vehicle-related payment point;

[0031] When the vehicle-related payment point deviates from the projection area of ​​the HUD display device, the first virtual image element is configured to display an arrow mark pointing in a specific direction according to the positional relationship between the vehicle and the vehicle-related payment point.

[0032] According to the above description, the optional implementation method can flexibly adopt different presentation forms according to the virtual and real fitting relationship between the projection area and the vehicle-related payment point, giving priority to a more intuitive information identification form to improve comprehensibility and conspicuousness, while getting rid of the limitation of the projection area size on the entire interaction.

[0033] In an optional implementation of the first aspect, the first virtual image element is presented in an animation manner.

[0034] According to the above description, the optional implementation method can improve the user's attention and avoid ignoring the interactive information about the vehicle-related payment point.

[0035] In an optional implementation of the first aspect, in response to there being a vehicle-related payment point around the vehicle, the HUD display device projects at least the first virtual image element, and the interaction method includes:

[0036] A specific button in the vehicle automatically switches to receive an authorized operation input by the user.

[0037] In an optional implementation manner of the first aspect, the specific button in the vehicle automatically switches to receiving the authorized operation of the user input, including:

[0038] The specific buttons include a play button, a volume button, an answer button or a menu button;

[0039] In response to there being a vehicle-related payment point around the vehicle, the original function of the specific button is temporarily disabled to receive the authorization operation input by the user.

[0040] In an optional implementation manner of the first aspect, the specific button in the vehicle automatically switches to receiving the authorized operation of the user input, including:

[0041] The specific key supports receiving the user's selection configuration among the play key, volume key, answer key and menu key.

[0042] According to the above description, an optional implementation method can receive the user's authorization operation through the vehicle-mounted button, ensuring that the user can control the entire interaction process through the last step of the card point operation, thereby increasing the user's trust in it and the quick experience of nearby authorization.

[0043] In an optional implementation of the first aspect, in response to the authorization operation input by the user, obtaining at least part of the control functions of the vehicle-related payment point includes:

[0044] Communicate with an external device and receive a user authorization operation forwarded by the external device.

[0045] In an optional implementation of the first aspect, the external device is a mobile phone terminal, and the forwarding authorization operation includes a mobile phone shaking operation.

[0046] In an optional implementation manner of the first aspect, the communicating with the external device and receiving the user authorization operation forwarded by the external device includes:

[0047] The specified amount is automatically deducted from the account bound to the external device.

[0048] According to the above description, an optional implementation method uses a mobile phone or other personal device to issue authorization operations, making the interaction process such as payment more convenient.

[0049] In an optional implementation manner of the first aspect, the specific button in the vehicle automatically switches to receiving the authorized operation of the user input, including:

[0050] The specific key supports fingerprint recognition to determine the identity of the user who inputs the authorized operation.

[0051] In an optional implementation of the first aspect, in response to the authorization operation input by the user, obtaining at least part of the control functions of the vehicle-related payment point includes:

[0052] The identity of the user entering the authorized operation is determined by a driver monitoring system in the vehicle.

[0053] In an optional implementation of the first aspect, in response to the authorization operation input by the user, obtaining at least part of the control functions of the vehicle-related payment point includes:

[0054] The account from which the specified amount is deducted is determined based on the identified user identity.

[0055] According to the above description, the optional implementation method can further improve the security of the entire interaction process, and the selection of the deduction account is more flexible.

[0056] In an optional implementation of the first aspect, in response to there being a vehicle-related payment point around the vehicle, the HUD display device projects at least the first virtual image element, and the interaction method includes:

[0057] The voice assistant in the vehicle automatically wakes up to receive the authorization operation input by the user.

[0058] According to the above description, the optional implementation method can automatically trigger the voice assistant without the need for a wake-up word, thereby improving the fluency of the voice assistant in receiving user-authorized operations and providing a better user experience.

[0059] In an optional implementation of the first aspect, the vehicle-related payment point is a gate at a toll station or a parking lot entrance or exit.

[0060] In an optional implementation of the first aspect, in response to the authorization operation input by the user, obtaining at least part of the control functions of the vehicle-related payment point includes:

[0061] The barrier gate is automatically opened to support the passage of the vehicle.

[0062] In an optional implementation of the first aspect, in response to the authorization operation input by the user, obtaining at least part of the control functions of the vehicle-related payment point includes:

[0063] Upon receiving the control operation input by the user, the barrier gate is closed or opened again.

[0064] In an optional implementation manner of the first aspect, the authorization operation and the control operation share the same key.

[0065] In an optional implementation manner of the first aspect, upon receiving the control operation input by the user, closing the barrier gate or reopening the barrier gate includes:

[0066] When the barrier is open, other vehicles cannot identify the vehicle-related payment point;

[0067] When the barrier is closed, other vehicles can identify the vehicle-related payment point to support the control function of seizing the vehicle-related payment point.

[0068] According to the above description, the optional implementation method supports users to obtain control authority of the vehicle-related payment point, and then independently control switches such as gates according to the driving conditions of their own vehicles to prevent other vehicles from passing by.

[0069] In an optional implementation of the first aspect, in response to the authorization operation input by the user, obtaining at least part of the control functions of the vehicle-related payment point includes:

[0070] Automatically deduct the specified amount from the linked account.

[0071] According to the above description, the optional implementation method can meet the scenario of vehicle payment and pass, and improve the smoothness of the payment process.

[0072] In an optional implementation of the first aspect, the matching feature is an identifier directing payment.

[0073] In an optional implementation of the first aspect, the matching feature is an identifier with the same content symbol as that of the vehicle-related payment point.

[0074] According to the above description, the optional implementation method supports using simple verification content to assist users in judging the matching of vehicle-related payment points, thereby reducing users' doubts about their accuracy and authenticity.

[0075] In an optional implementation of the first aspect, after obtaining at least part of the control functions of the vehicle-related payment point in response to the authorization operation input by the user, the interaction method includes:

[0076] In response to the vehicle moving away from the vehicle-related payment point, the control function of the vehicle-related payment point is reclaimed.

[0077] In an optional implementation manner of the first aspect, the reclaiming of the control function of the vehicle-related payment point includes:

[0078] The barrier gate is automatically closed to regain control of the passage to the vehicle-related payment point.

[0079] According to the above description, the optional implementation method can reliably support the passage of continuous vehicles before and after the vehicle-related payment point, ensuring smooth handover of control of the vehicle-related payment point.

[0080] In an optional implementation of the first aspect, in response to there being a vehicle-related payment point around the vehicle, the HUD display device projecting at least a first virtual image element includes:

[0081] The HUD display device supports the installation of identification programs under unified security authorization.

[0082] In an optional implementation of the first aspect, in response to there being a vehicle-related payment point around the vehicle, the HUD display device projecting at least a first virtual image element includes:

[0083] The recognition program installed on the HUD display device is a smart contract added to the blockchain.

[0084] According to the above description, the optional implementation method prevents the illegal installation of programs that implement the interactive method in the HUD display device through security management, resulting in the payment of fees and the control of vehicle-related payment points being exploited by illegal third parties.

[0085] In a second aspect, the present application provides a display device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor implements the steps of the interaction method described in the first aspect when executing the computer program.

[0086] In a third aspect, the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the interaction method described in the first aspect are implemented.

[0087] In a fourth aspect, the present application provides a vehicle comprising the display device described in the second aspect or the computer-readable storage medium described in the third aspect.

[0088] Compared to existing technologies, this application triggers the projection of a first virtual image element by the HUD display device upon identifying a vehicle-related payment point. This first virtual image element not only serves as an interactive element for users to control the vehicle-related payment point, but also provides users with verification information and prompt information related to the vehicle-related payment point, allowing users to make authorization operations in a trusted environment, thereby simply achieving control of equipment such as gates at the vehicle-related payment point without the need for additional operations. This application improves the security and reliability of vehicle-related payments, simplifies the process, and enhances the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0089] To more clearly illustrate the technical solution of this application, the following briefly introduces the drawings required for describing the technical solution. Obviously, the drawings described below are merely examples of the invention, and those skilled in the art can derive other drawings based on these drawings without inventive effort.

[0090] Figure 1 This is a schematic diagram of a license plate payment scenario in the prior art.

[0091] Figure 2 Schematic diagram of ETC payment scenario in existing technology.

[0092] Figure 3 Schematic diagram of HUD projection display in some examples of this application.

[0093] Figure 4 Schematic diagram of the interaction architecture in some examples of this application.

[0094] Figure 5 Schematic diagrams showing the first virtual image element in some examples of the present application.

[0095] Figure 6 This is a schematic diagram of transaction price information in the first virtual image element in some examples of this application.

[0096] Figure 7 This is a schematic diagram of the limited presentation of the first virtual image element in some examples of the present application.

[0097] Figure 8 This is a schematic diagram of the digital matching features in the first virtual image element in some examples of this application.

[0098] Figure 9 This is a schematic diagram of the shape matching features in the first virtual image element in some examples of this application.

[0099] Figure 10 Schematic diagrams are presented for two or more virtual image elements in some examples of this application.

[0100] Figure 11 This is a schematic diagram of the wide presentation of two or more virtual image elements in some examples of this application.

[0101] Figure 12 This is a schematic diagram of key receiving operations in some examples of this application.

[0102] Figure 13 This is a schematic diagram of key operation prompts in some examples of this application.

[0103] Figure 14 This is a schematic diagram of key function switching in some examples of this application.

[0104] Figure 15 This is a schematic diagram of mobile phone receiving operations in some examples of this application.

[0105] Figure 16 Schematic diagram of voice assistant receiving operations in some examples of this application.

[0106] Figure 17This is a timing diagram of the interaction method in some examples of this application.

[0107] Figure 18 Schematic diagram of the HUD display device module in some examples of this application.

[0108] Figure 19 This is a schematic diagram of the composition of the HUD display device in some examples of this application.

[0109] Figure 20 This is a schematic diagram of projection display in a vehicle in some examples of this application. DETAILED DESCRIPTION

[0110] The following will describe the present application in detail with reference to the accompanying drawings, but the description is merely some examples recorded in the present application and does not limit the present application. Any changes in structure, method or function made by ordinary technicians in this field based on these examples are included in the scope of protection of the present application.

[0111] It should be noted that the same reference numbers or indicia may be used in different examples, but these do not represent absolute structural or functional connections. Furthermore, the terms "first," "second," and so on, which may be mentioned in various examples, are merely for descriptive convenience and do not represent absolute structural or functional distinctions. They should not be construed as indicating or implying relative importance or the number of corresponding objects. Unless otherwise specified, the term "at least one" in the description refers to one or more, and "a plurality" refers to two or more.

[0112] Additionally, when representing features, the character " / " can indicate an OR relationship between the preceding and following objects. For example, heads-up display / head-up display can be represented as heads-up display or heads-up display. When representing operations, the character " / " can indicate a division relationship between the preceding and following objects. For example, magnification M = L / P can be represented as L (virtual image size) divided by P (image source size). Furthermore, the "and / or" in different examples is simply to describe the relationship between the preceding and following objects. This relationship can include three situations. For example, a concave mirror and / or a convex mirror can be represented as the concave mirror alone, the convex mirror alone, or the concave and convex mirrors simultaneously.

[0113] In traditional vehicle-related payment scenarios, such as when a vehicle passes through a toll booth or a parking lot entrance or exit, payment is collected manually. The disadvantage of this is that the efficiency is very low, and vehicles often have to wait in line. With the advent of electronic technology, there are also some automated payment methods. Among them, license plate payment has gradually become popular in the payment process of highway toll booths, and parking lot payment in cities has become widely popular. Figure 1As shown, the recognition and payment of license plate payment is generally done by automatically identifying the metal license plate on the vehicle through a high-definition camera, and managing the fees according to the account bound to the identified license plate number. Since there is often a problem of duplicate license plates on metal license plates, the bound account is often a temporary payment account or a time-based subscription account. Users are not willing to directly bind financial accounts, such as bank cards, Alipay, etc. This will bring many inconveniences to the entire payment process. For example, users need to scan the code to pay every time they leave the parking lot. Figure 2 As shown, this is another ETC (Electronic Toll Collection) payment method, first used at toll booths on highways and, in recent years, also being tested in some urban parking lots. It primarily utilizes radio frequency communication between an on-board unit (OBU) mounted on the vehicle's windshield and a roadside unit (RSU) located at toll booths and parking lot entrances. Payment is then processed over the air through information exchange. However, radio frequency communication has a relatively short recognition range and is subject to fluctuations due to weather. This can sometimes result in a vehicle paying from a considerable distance, while other vehicles may cut in line and exploit the ETC payment. Furthermore, since the OBU is exposed on the windshield, vehicles parked outdoors can sometimes be fraudulently charged by unauthorized RSUs, posing security risks.

[0114] The main reason for the security issues with the above-mentioned vehicle-related payment methods is that the vehicle, as the payment subject, is often charged passively by external devices. Unlike personal payment scenarios, the payment process is often initiated by the payment subject. This may also be related to the security restrictions on the intensity of user intervention during driving. Accordingly, in the process of improving the vehicle-related payment process, it is necessary to give the user in the car the ability to actively authorize. At the same time, it is also necessary to reduce the overly cumbersome processes similar to personal payment processes. After all, the user is driving and has certain requirements for driving safety. In some examples, an interaction channel between the user in the car and the external scene can be established by projecting the HUD display device in front of the windshield, and the user can issue specific authorization instructions based on the situation of the external scene and his own needs. Among them, HUD projection display mainly uses the optical reflection principle to reflect the imaging light to be displayed through the vehicle windshield into the viewer's eyes. The human eye can see the virtual image information along the opposite direction of the light, and use the windshield as a display screen to display the vehicle's navigation instructions, vehicle speed, etc. In this example, the virtual and real display capabilities of the HUD display device can be used to mark the interactive information to be paid in the payment scene ahead, thereby improving the efficiency of vehicle-related payments and user trustworthiness. This will be described in detail below. Figure 3As shown, the HUD display device may include at least an optical engine 1, a first reflector 2, and a second reflector 3. The optical engine 1 includes a backlight source and an image source (not shown). The backlight source is used to provide illumination light and adjust the brightness of the illumination light according to control. For example, the backlight source may be an LED (Light Emitting Diode) or a laser. Under the illumination provided by the backlight source, the image source adjusts the corresponding display content according to control and projects the display light from the surface of the image source. For example, the image source may be an LCD (Liquid Crystal Display), a DMD (Digital Micromirror Device), a MEMS (Micro-Electro-Mechanical System) micromirror, or an LCOS (Liquid Crystal on Silicon). The first reflector 2 and the second reflector 3 can project the display light projected by the optical machine 1 onto the windshield 4, realizing customization of the optical path in a smaller space while meeting different projection display requirements. The first reflector 2 and the second reflector 3 can be set to a concave mirror, a convex mirror, a concave lens, a convex lens, etc. according to the requirements of optical planning, and the surface shape of the lens can be a free-form surface. Optionally, at least one of the first reflector 2 and the second reflector 3 can also be adjusted to a certain degree of angle, thereby changing the projection position of the display light on the windshield 4 to meet the needs of viewers of different heights. The display light of the optical machine 1 is finally reflected on the windshield 4 of the vehicle to form a virtual image 5. When the human eye 6 observes the virtual image 5 facing the windshield 4, it can feel a certain sense of depth, just like looking at a real object at a specific distance outside the windshield. The virtual image 5 can be the navigation instructions content, vehicle speed, etc. as described above. It should be added that, according to the characteristics of different optical machines, the HUD display device can also be provided with a diffuser. In some examples, the HUD display device can also include Fresnel lenses, waveguide optical devices, diffraction optical devices, holographic optical devices, tapered optical fibers, etc.

[0115] As mentioned above, as a head-up display device, the advantage of the HUD display device over low-head display devices such as mobile phones and vehicle-mounted central control screens is that the displayed information can be projected directly on the real scene in front. Therefore, an interactive method different from mobile phone payment can be adopted, and a context of trust can be built through intuitive cooperation with the real scene, so that even a simpler authorization method will be accepted by users. Correspondingly, for vehicle-related payment points such as gates, it is generally necessary to pay an equivalent amount or rely on specific permissions to achieve control, such as remotely opening the gate. However, in this example, the information required for interaction can be projected near the gate through the HUD display device. Users can use the matching features associated with the vehicle-related payment point in the projection information to determine whether the vehicle-related payment point meets their needs, allowing users to fully grasp the entire process information, and the last step of the authorization operation is an active choice made by the user based on the projection information, which makes users feel more at ease and more secure and reliable. As Figure 4As shown, the interaction methods implemented by the HUD display device to meet vehicle payment requirements can be divided into two layers: a perception layer 932 and an interaction layer 931 above it. Optionally, a middleware layer can be provided between the interaction layer 931 and the perception layer 932. The perception layer 932 is responsible for acquiring the external scene and scanning payment points around the vehicle, such as barriers. It can use a camera to analyze the shape of the barriers within its field of view. To distinguish payment points with high similarity, such as multiple toll gates at a toll station, additional reference features can be introduced for comparative analysis, such as surface texture and dimensions on the barriers. Parameters such as position and orientation can also be combined to determine the specific barrier to be identified. Accordingly, each barrier registers its own image and scene features upon network access. During identification, pre-registered data can be used for comparison to determine the identity. Furthermore, image analysis can be used to determine the relative position of the barrier and vehicle, serving as the basis for aligning the HUD display's projection information with the barrier. Alternatively, identification of vehicle-related payment points can be achieved through UWB (Ultra Wide Band) modules. UWB utilizes nanosecond-scale, non-sinusoidal, narrow pulses to transmit data, thus occupying a wide spectrum. This allows for precise positioning, such as determining the direction and distance of the payment point relative to the vehicle. It also supports data communication between the payment point and the vehicle, for example, by allowing the payment point to directly transmit its own identification. It should be noted that the module configuration within the perception layer 932 is not limited to the above example and may also include accelerometers to accurately determine the vehicle's state. When the perception layer 932 submits the recognition results to the interaction layer 931, the projection module within the interaction layer 931 controls the projection content and location of the virtual image elements, guiding users to understand the controllable capabilities around the vehicle. The payment module then exchanges payment information with the corresponding server to complete operations such as payment deductions. Optionally, the interaction layer 931 may also include a communication module for communicating control commands with the payment point, as well as security management operations, to ensure that the entire interaction process is protected from unauthorized third-party exploitation. In further examples, the interaction layer 931 and the perception layer 932 can be built on top of the HUD display device and the vehicle computer. For example, the perception layer 932 can reuse the vehicle computer's sensor equipment, while the interaction layer 931 can also reuse the vehicle computer's computing power and communication resources. In some examples, to prevent the entire interaction process from being exploited by unauthorized third parties, the installation of the corresponding programs of the interaction layer 931 and the perception layer 932 will also be subject to security regulations. In particular, the payment module will not be illegally charged, and the scanning camera, UWB module, etc. will not identify illegal vehicle-related payment points.Specifically, installing these recognition programs on HUD displays or vehicle computers requires system permission and prohibits direct installation via sideloading. For example, they can only be downloaded and installed from authorized app stores. In some cases, these recognition programs are consensus-certified smart contracts on the blockchain. Once released, these programs cannot be tampered with by unauthorized third parties and therefore cannot be illegally exploited.

[0116] In some examples, such as Figure 5 As shown in the figure, when a vehicle approaches a specific parking lot, the parking lot will manage the passage of the vehicle through the gate. The gate will be recognized by the sensor equipment in the vehicle. For details, please refer to Figure 4 For example, the camera in the vehicle can capture images around the vehicle and determine the barrier as a specific identification identity based on the appearance characteristics of the barrier. Specifically, the corresponding identification identity can be obtained based on the location of the barrier, the unique pattern of the barrier, etc. For example, the unique rust texture on the barrier can uniquely determine the barrier set at a specific location, so that a connection can be established using the registered communication information corresponding to the identification identity. Optionally, corresponding UWB modules are provided on both the barrier and the vehicle for communication between the two. The vehicle can communicate with the surrounding barrier and obtain the feedback identification identity. It can also determine the direction and distance of the barrier from the vehicle based on parameters such as the strength of the UWB signal, thereby serving as coordinate information based on the projection information. In this example, in response to the presence of a barrier around the vehicle, a first virtual image element will be projected through the front projection area 50 of the windshield 4. It can be a sign prompting charging. The user can know through the sign that this barrier has controllable capabilities and that the payment behavior of this barrier is trustworthy. At the same time, the interaction layer 931 (refer to Figure 4) is in a state of waiting to receive authorization operations and is ready to respond to user authorization operations at any time. In this example, because the projection area 50 is directly opposite the barrier to be controlled, the HUD display device has the ability to project the first virtual image element directly onto or next to the barrier, for example, superimposed on the actual barrier, and the viewing effect is like a specific identification light on the barrier being illuminated. Accordingly, when the first virtual image element is projected onto or next to the barrier, the user can easily understand the projection result as information related to the barrier, thereby establishing a context associated with the barrier. At the same time, the first virtual image element in this example uses the RMB symbol, a feature with specific connotations that can match the payment behavior of the barrier. Furthermore, the matching feature of the RMB symbol can be specifically used for parking lot barriers. In this way, the user can conclude that it matches through simple comparison and verification, and then decide whether to authorize the next step. In more examples, the first virtual image element is presented with animation when projected onto or next to the barrier, such as the RMB symbol rotating, to attract the user's attention, prevent them from overlooking specific parking lot entrances and exits, and make a timely decision on whether to enter. Figure 6 As shown, the first virtual image element can also include the transaction price information required for passage. In this example, in addition to the barrier symbol as a matching feature, there is also a 5-yuan identification symbol. Accordingly, the transaction price information itself can also be used as a matching feature for user verification. If the toll of this barrier is obviously not a regular price, it will be promptly falsified by the user, and the corresponding authorization operation will not be performed.

[0117] In some examples, such as Figure 7 As shown, the FOV (Field of View) of the HUD display device is limited and does not meet the field of view determined by the entire windshield. The projection area 50 is only projected in front of the driver's seat. Accordingly, the projection information can only be projected in the left front of the vehicle, while the actual gate is in the right front. If the Figure 5 、 Figure 6The projection method of the example can only be presented when the vehicle moves to the right to face the barrier. In this way, the vehicle has often arrived at the barrier and the user does not have enough time to issue the authorization operation. The only way is to slow down the process by parking, which inevitably affects the user experience. In this example, the first virtual image element adopts a limited presentation form and is also projected near the barrier to establish a context associated with a specific vehicle-related payment point. It at least prompts the user that there are vehicle-related payment points around the vehicle and has certain direction information. At the same time, text content is used to describe the matching features associated with the right front barrier, such as the name of the barrier. Furthermore, an arrow mark can be used to point to the actual position of the barrier. The arrow mark reflects the relative position relationship between the barrier and the vehicle. The direction of the arrow mark will also be dynamically adjusted according to the movement of the vehicle. Specifically, it can be based on Figure 4 The perception layer 932 in the image processing unit is provided. Accordingly, the arrow mark points not only up, down, left, and right, but can also adjust the front and back directions according to the distance between the vehicle and the barrier, specifically reflected by the stereoscopic arrow mark. Optionally, the first virtual image element can also be presented using only an arrow mark, without the above-mentioned text description. One arrow mark represents one barrier, and the direction indicated by the arrow mark can be used as a matching feature. If the direction is consistent with the actual position of the barrier, a certain matching verification function is achieved. Optionally, the arrow mark can also be presented in a flashing animation to attract the user's attention. In some examples, in the projection area 50, the use of arrow marks can not only present the vehicle-related payment points directly in front of the windshield, but also support the presentation of vehicle-related payment points on the side or rear of the vehicle through virtual image elements. This also plays a certain role in vehicle-related payment point navigation, helping users (i.e., drivers) locate undiscovered vehicle-related payment points around the vehicle. In more examples, when a vehicle approaches a specific gate, the projection area 50 often changes from a direction away from the gate to a direction facing the gate. Therefore, when the projection area 50 projects the virtual image element corresponding to the gate, it can be accompanied by a change from a direction away from the gate to a direction facing the gate. Figure 7 The example presentation is Figure 5 、 Figure 6 The presentation format in the example switches, and the information presentation becomes gradually more intuitive. Regardless of the presentation format, it supports receiving authorization operations issued by the user.

[0118] In some examples, such as Figure 8As shown, the first virtual image element can also include more matching features, thereby further strengthening the user's trust in the entire interaction process. In this example, each gate will be assigned a number (123 in the figure) that is unique within a specific range, such as a certain geographical area. This number can represent a specific gate. This number can be affixed to the actual gate, and this number information is also included in the projected first virtual image element. Users can judge the credibility of the payment interface by comparing whether the number on the actual gate is consistent with the projected number. Figure 9 As shown, the matching features can also be represented by symbols of a specific arrangement. The arrangement of this symbol can follow certain rules. Optionally, it can uniquely point to a specific barrier gate. Similarly, after the barrier gate is installed, this symbol logo can be affixed to the surface of the barrier gate and can be normally seen by the human eye. The first virtual image element of the projection also includes the symbol logo on the barrier gate. The consistency of the symbol logo indicates that there is no problem with the recognition and projection, and the user can safely initiate the next operation such as payment. It should be noted that the matching features used for comparison in the above examples are not limited to digital numbers and arrangement shapes, and can also be presented in other ways. Preferably, they are marked with symbol logos that are easier for users to understand, simpler to compare, and uniquely representative.

[0119] In some examples, such as Figure 10As shown, in different scenarios, there may be multiple vehicle-related payment points around the vehicle. For example, a toll booth may include multiple toll gates in front of the vehicle. Generally, as the vehicle approaches the toll booth, the driver will select the corresponding toll gate based on the queue situation at each toll gate. Accordingly, when issuing authorization operations, it is necessary to distinguish between different vehicle-related payment points (i.e., gates). In this example, the front of the windshield 4 includes a left gate T1 and a right gate T2. Different gates correspond to different toll gates. A first virtual image element corresponding to gate T1 and a second virtual image element corresponding to gate T2 are projected in the projection area 50. Since gate T1 is directly opposite the projection area 50, the first virtual image element uses a more intuitive information identifier. Projecting it next to gate T1 allows the user to clearly understand that gate T1 supports interactive control. At the same time, the barrier T2 deviates from the projection area 50, and the second virtual image element cannot be directly projected on or next to the barrier T2. Therefore, the second virtual image element can be marked with an arrow on the right edge of the projection area 50, and the pointing position is facing the actual barrier T2. In some examples, no matter what form the first virtual image element and the second virtual image element take, when displayed at the same time, there will be a default virtual image element as the control focus, that is, if the user issues an authorization operation, it is determined to control the default virtual image element. Optionally, the default virtual image element can be identified as the virtual image element that is projected first, or the virtual image element corresponding to the vehicle-related payment point that the vehicle is facing most directly can be identified as the default. In more examples, input devices such as buttons in the vehicle can also be used to switch the focus of control, which can flexibly meet the specific needs of users. For example Figure 11 As shown, the projection area 50 is a large FOV, relative to Figure 10 For example, projection area 50 not only faces barrier T1 but also barrier T2. Therefore, the first and second virtual image elements are intuitively labeled, allowing users to easily identify the two controllable barrier gates. The first and second virtual image elements appear as if they were integral to the barrier gates. Optionally, the first virtual image element, serving as the default control focus, can be marked with an outer circle. Users can choose whether to directly control barrier T1 based on their needs. If they need to switch to the second virtual image element corresponding to barrier T2, they can do so using an input device. Accordingly, the outer circle will shift from the first virtual image element to the second virtual image element.

[0120] As described above, once the virtual image element corresponding to the vehicle-related payment point is projected in the projection area 50, it indicates that the user has the ability to take over the corresponding vehicle-related payment point at any time. At this time, an input device provided to the user for operation can be configured to receive the authorization operation issued by the user. Specifically, one-key operation can be supported to simplify the interaction process. Optionally, when there are multiple virtual image elements at the same time, that is, there are multiple vehicle-related payment points around the vehicle, the user will only obtain the control of the vehicle-related payment point that is in the control focus when issuing the authorization operation, and the other virtual image elements may disappear after the authorization operation is issued and will not respond. In some examples, a button for receiving authorization operations can be set separately in the vehicle. Preferably, it can be configured on the steering wheel for the user to press while driving the vehicle. Further, a fingerprint recognition function can be set on the button, and the authorization operation will only be issued in response to the pressing of a specific identity user, or the pressing of users with different identities will trigger payment of accounts bound to different users. As Figure 12 As shown, button 933 can be an entertainment function button such as the original play button, volume button, answer button or menu button on the steering wheel. Under normal conditions, button 933 can serve the original entertainment operation, but once it recognizes that there is a vehicle-related payment point around the vehicle and there is a corresponding virtual image element in the projection area 50, button 933 will switch its function and no longer respond to the entertainment function, that is, the original function will be temporarily closed. If the user operates button 933 after the switch, it will be deemed that the user has entered an authorization operation. This not only reduces the occupation of the button, but also allows the user to achieve multiple functions with one button without unnecessary operations. In a specific example, reusing one of the play button, volume button, answer button or menu button can support flexible configuration by the user. After the user configures it, once the vehicle-related payment point is identified, the configured button can support receiving authorization operations at any time, while other unconfigured buttons still maintain their original input functions.

[0121] Specifically, when the button 933 switches the function input according to the changes in recognition and projection, it will also display corresponding prompt information in the projection area 50, such as "Press the volume button to trigger payment". The user triggers the authorization operation under the guidance of the prompt information, which also eliminates the confusion of operations under key reuse and improves the user experience. Correspondingly, the object pressed in the prompt information can also be adaptively adjusted according to the actual configuration. Figure 13 As shown, the prompt information is compared with Figure 12The example is not directly displayed in the projection area 50, and is separated from the virtual image elements in the projection area 50, reducing the indication and matching interference of the vehicle-related payment points. Specifically, the HUD display device supports the projection of the first focal plane and the second focal plane. The first focal plane can be the range corresponding to the projection area 50, and its virtual image distance is farther, such as about 10 meters, which is suitable for fitting display with the real object, while the second focal plane can be the range corresponding to the projection area 51, and its virtual image distance is closer. In this example, the prompt information is displayed in the projection area 51. Since this area normally displays driving parameters such as vehicle speed, the user can see the payment-related prompt information while viewing this information. The information distribution is more organized, and there is no serious defocus when the user sees it through the peripheral vision, making it easier to view. In more examples, the prompt information can also be displayed in locations such as the central control screen, or the user driving the vehicle can be reminded through voice prompts.

[0122] As described above, after the user sees the first virtual image element and the prompt information, if they truly need to pass through the corresponding vehicle-related payment point, such as entering the parking lot entrance where the corresponding barrier is located, they can press the aforementioned button. Taking the barrier as an example, in response to the user authorization operation input by the pressing behavior, the control function of the barrier is obtained, and the barrier can be triggered to open. If a vehicle enters the parking lot entrance, the barrier will automatically close, which can be used to reclaim control of the barrier. In a specific example, between obtaining the authorization operation and obtaining the control function, the user's account will also be operated. For example, at the parking lot entrance, the time of vehicle entry is recorded, while at the parking lot exit or toll station, a specified amount can be deducted. Accordingly, the user's account can be bound to a financial account such as a bank card or Alipay, thus supporting direct deduction. In this example, only the authorization operation is left to the user during the entire interaction process. This control relies solely on the user's convenient input device to perform a one-click operation, while the previous and subsequent recognition, payment, and other actions can be performed automatically. This not only optimizes the user experience but also ensures security and improves user awareness and autonomy. In some examples, in addition to opening the barrier, full control of the barrier can also be obtained for a period of time. That is, the user can close the barrier again by inputting a control operation with a specific key. When the vehicle arrives at the barrier, the user can input the control operation again to open the barrier. This can effectively prevent the vehicle in front from cutting in line and cutting in. Optionally, the key for inputting the authorization operation and the control operation can be the same, such as Figure 14As shown, the button can be an entertainment function button on the steering wheel. As mentioned above, it can be a function button for audio adjustment, call answering, etc. Under normal circumstances, this button receives the original operation for implementing the aforementioned entertainment functions. However, when a barrier is detected, the button switches to receive authorization operations, which are used to trigger actions such as payment and barrier opening. Optionally, in response to barrier detection, the button switches only for the vehicle-related payment point detected in front of the vehicle, and does not switch for vehicle-related payment points detected on the sides or rear of the vehicle. This prevents the original function of the button from being occupied and unavailable for a long time. When the user enters the authorization operation to trigger the control of the barrier, the button switches to receive control operations, that is, the button becomes a remote switch for the corresponding barrier. The user can press the button to close the barrier when the barrier is open, and press the button to open the barrier when the barrier is closed. When the vehicle leaves the barrier, the barrier control function is retracted, and the user can no longer control the barrier through the button. The button switches to receive the original operation, that is, if it is an audio key, it will revert to adjusting the playback volume, greatly improving the utilization rate of the button and user convenience.

[0123] Furthermore, to prevent users from abusing control of vehicle-related payment points, for example, if a user does not pass through the barrier for a long time after obtaining control of the barrier, queuing problems will inevitably occur. This is inconsistent with the congestion caused by slow toll collection, which is caused by the users in the vehicle themselves. Therefore, the temporary transfer of control of the barrier is not unlimited. In some examples, the control of a specific vehicle can be timed. If the time exceeds a certain threshold, the control will be automatically revoked. Accordingly, the barrier will be automatically closed if the vehicle does not pass through the barrier. At the same time, if other vehicles issue authorization operations, the control will be transferred to other vehicles. In some examples, although the leading vehicle obtains control of the barrier by issuing an authorization operation, the following vehicle is also around this barrier, and the corresponding virtual image element will be displayed in the projection area corresponding to the following vehicle. At this time, the user of the following vehicle can determine whether he needs to seize the control of this barrier based on the movement progress of the leading vehicle. The method is no different from the ordinary authorization operation. It can be achieved by pressing a specific button, and the barrier will be triggered to unconditionally transfer the control to the following vehicle. The leading vehicle cannot control the opening and closing of the barrier through operation. Optionally, in order to avoid damage to passing vehicles caused by the closing of the barrier, when the barrier is open, vehicles other than those occupying the control right cannot recognize its barrier. It can be that although the vehicle actively obtains the relevant information of the barrier through the sensor, it cannot establish a connection with the barrier, and the corresponding virtual image element will not be displayed in the corresponding projection area. Its control button cannot respond to the authorization operation input by the user. Only when the barrier is closed can other vehicles identify the barrier and display virtual image elements in the projection area to guide users in other vehicles to issue authorization operations. After responding to the authorization operations of other vehicles, the barrier will forcibly transfer control to the vehicle that has taken over from behind, so as to urge the vehicle in front to make a passage decision as soon as possible.

[0124] In some examples, such as Figure 15As shown, receiving user input for authorized or controlled operations is not limited to buttons within the vehicle; it can also be done via external devices connected to the vehicle. For example, mobile phone 934 is typically strongly bound to a specific user. Mobile phone 934 not only receives the corresponding authorized or controlled operations, but also obtains the identity of the user issuing these operations. This allows it to be tied to a payment account, enabling payment based on a specific user. In this example, the recognition and projection logic is consistent with the previous example. When a user sees a virtual image element representing a specific vehicle-related payment point in the projection area, they can use mobile phone 934 to issue an authorization operation. Mobile phone 934 can be carried by the user and automatically connect to the HUD display device or vehicle computer upon boarding the vehicle. Accordingly, when the corresponding virtual image element appears in the projection area, the HUD display device or vehicle computer can send a message notification to the connected mobile phone 934, instructing it to send any specific mobile phone operations received during this period, such as a shake, as authorization or control operations to the HUD display device or vehicle computer. For users who are driving, such operations are relatively convenient. In response to the user's shake operation, at least some of the control functions of the corresponding vehicle-related payment point can be obtained, such as automatically opening the gate in the above example. If payment is involved, the bound account can be determined based on the identity of the mobile phone user and the fee can be deducted. The user can shake the mobile phone 934 again to close the opened gate. When the vehicle drives to the gate, the mobile phone 934 can be shaken again to open the gate again. When the vehicle passes through the gate, the control of the gate will be taken back. At this time, the mobile phone 934 can be notified that there is no need to respond to the shaking action. Even if the user shakes the mobile phone 934, it will not initiate any control behavior on the gate. If some functions of the mobile phone 934 itself are triggered by shaking, the original triggering behavior will be restored.

[0125] In some examples, such as Figure 16 As shown, the authorization operation or control operation that receives user input may also be the voice assistant in the vehicle. Accordingly, when the vehicle-related payment points around the vehicle are identified, the corresponding virtual image element (the word "entrance") will appear in the projection area 50. At the same time, the voice assistant in the vehicle will be automatically turned on, and there is no need to use a wake-up word to trigger it to turn on. In order to prompt the user that the voice assistant has been turned on, the authorization instruction can be directly issued by voice command, and the logo that the voice assistant is triggered can be displayed in the projection area 50. Optionally, this logo that prompts that it has been triggered can also be displayed on the central control screen 52 of the vehicle. The authorization operation is also completed through the voice assistant capability of the vehicle computer, and the user's voice instruction is obtained through the vehicle computer. In more examples, the triggered logo can also refer to Figure 13The example is shown in the projection area 51. Similarly, after receiving the user's authorization operation through voice input, the gate can be opened, etc. At this time, the voice assistant does not need to be closed and will still receive the user's voice instructions, that is, it can further respond to the user's control instructions to continuously control the opening and closing of the gate. When the vehicle leaves the vehicle payment point and the control is taken back, the voice assistant is turned off. If the voice assistant is used normally, it needs to be triggered by the wake-up word. In addition, the mechanism of preemption through external devices and voice assistants is the same as Figure 12-14 The examples are similar and will not be repeated here.

[0126] like Figure 17 As shown, for the process of HUD display device assisting payment interaction, it can be a scene where the vehicle arrives at the vehicle-related payment point such as the barrier together with the vehicle. The HUD will identify the barrier. Optionally, the specific identification scan is achieved through the vehicle's own sensor equipment. Optionally, if the feature information is exchanged through UWB, etc., the feature information will also be encrypted to reduce interception by illegal third parties. Accordingly, when the HUD obtains the identification identity of the barrier, it can be compared with the comparison data owned by the HUD itself. Preferably, the identified feature information is sent to the operation server registered with the barrier. The operation server has the information of the designated barrier registered after installation, so as to perform comparison and identification. When the HUD initiates a comparison to the operation server, it will also initiate identity authentication to the operation server. The operation server determines whether the HUD initiating the comparison is a legal device. Only after it is determined to be legal will the comparison result be returned to the HUD, such as the identification identity of the barrier and the account for receiving the payment. After receiving the comparison result, the HUD will project a virtual image element to fit it relatively according to the position of the barrier. For details, please refer to Figure 5-Figure 11 , waiting for the user's authorization. For specific operations of receiving the user's authorization, please refer to Figure 12-16Once the user authorizes an action, if payment is involved, a debit request is initiated to the HUD's linked bank. The payment account can be determined based on the HUD's linked account, or based on the user's fingerprint or phone number, as in the example above. In further examples, the linked account can be determined by scanning the driver's seat user's identity through the vehicle's integrated driver monitoring system. Accordingly, the recipient account for the transfer can be determined based on the gate's identified identity. This information can be existing information registered with the gate's operating server. It should be noted that initiating a payment is not limited to deducting a monetary amount; it can also involve the consumption of certain tokens or membership privileges. After the deduction is completed, a payment completion message can be sent to the HUD and the gate's corresponding operating server. The operating server will also send a payment timestamp to the corresponding gate. The gate can then transfer control to the HUD based on the payment timestamp. As mentioned above, a specific control time range can be set. Once the HUD obtains control of the gate, a virtual control connection is established between the HUD and the gate. This virtual connection ensures control interaction between the two. Users can issue control operations through the vehicle's input devices, and the gate can respond by opening or closing. When the vehicle leaves the gate, the control virtual connection it established will also be released, that is, the control of the gate will be taken back.

[0127] When the HUD display device that implements the interactive method in the above example is applied to the vehicle, the HUD display device can be used as the vehicle's payment auxiliary terminal to establish a convenient channel between the user inside the vehicle and the vehicle-related payment point outside the vehicle. The user can control the vehicle-related payment point through simple authorization operations to achieve smooth passage. Figure 18 As shown, the HUD display device integrated in the vehicle can be powered and data-generated by the vehicle computer 92, or the HUD display device itself can provide power and generate data. The HUD display device may include a processor 91, an Ethernet interface 901, a CAN (Controller Area Network) interface 902, a power management module 903, a running memory 904, a storage memory 905, a temperature detection module 906, a motor 907, a backlight source 908, an image source 909, a positioning module 910, a radar 911, a camera 912, etc. It should be noted that Figure 18 The modules listed in the description are merely exemplary and do not constitute any limitation. In some examples, the HUD display device may further include other modules. In addition, the modules described above may be implemented in one or more hardware components in different examples, or a single module may be implemented by a combination of multiple hardware components.

[0128] Processor 91, serving as the control center of the HUD display device, includes one or more processing units of any type, including but not limited to a microcontroller, a microcontroller, a DSP (Digital Signal Processor), or any combination thereof. Processor 91 is used to generate operational control signals based on a computer program to control other modules and collaborate with corresponding modules to process acquired or inherent data and instructions.

[0129] The Ethernet interface 901 is a network data connection port for local area network communication, which defines a series of software and hardware standards. Multiple electronic devices can be connected together through the Ethernet interface 901. In this example, the processor 91 can exchange information with the vehicle computer 92 through the Ethernet interface 901, such as sending data to the vehicle computer 92 or receiving data sent by the vehicle computer 92.

[0130] CAN interface 902 is a network data connection port for the controller area network, providing a standard bus for the vehicle's internal control system and embedded industrial control, enabling communication and interaction between control nodes. In this example, processor 91 can also exchange information with vehicle computer 92 via CAN interface 902. Optionally, processor 91 can also connect to other external devices via CAN interface 902. In some examples, processor 91 may also be provided with a GPIO (General-purpose input / output) interface to improve compatibility with peripheral connections.

[0131] The power management module 903 is connected to the vehicle computer 92 and can receive the power provided by the vehicle computer 92 to provide a regulated power supply for each module of the HUD display device, ensuring that the processor 91 and each module operate under normal voltage supply and avoid damage due to overvoltage.

[0132] The running memory 904 is used to store computer programs executed by the processor 91, as well as temporarily stored operation data, data exchanged with the storage memory, etc. The running memory 904 can be a memory such as SDRAM (Synchronous Dynamic Random-access Memory).

[0133] The storage memory 905 is used to store resources such as the display content of the HUD display device, as well as long-term storage of running programs and data. The storage memory 905 can be a memory such as Flash. In some examples, the processor 91 can also provide an interface to access external memory.

[0134] Temperature detection 906 is used to detect the temperature within the HUD display device. Specifically, it may include multiple temperature sensors. Because the resistance of a temperature sensor changes with temperature, the processor 91 can determine the resistance of each temperature sensor at a corresponding temperature based on the voltage change between each temperature sensor and the voltage divider resistor under a fixed power supply voltage, thereby reversely inferring the temperature at the location of the temperature sensor. In some examples, the processor 91 can control multiple temperature sensors via the GPIO interface. Multiple temperature sensors can be located at different locations within the HUD display device. The processor 91 can use time-sharing detection to obtain the temperature values ​​fed back by multiple temperature sensors.

[0135] The motor 907 is used to drive the optical lens in the HUD display device to rotate under the control of the processor 91, thereby realizing the change of the corresponding optical path. For example, when the backflow of sunlight causes the temperature of the image source surface to rise, the motor can be used to drive the optical lens to prevent external sunlight from reaching the image source surface. In some examples, the processor 91 can also drive the fan provided on the HUD display device through the motor 907 to increase the speed of air exchange between the inside and outside of the HUD display device to achieve heat dissipation. Specifically, the motor 907 is connected to the processor 91 through a motor driver chip. The motor driver chip provides high-performance power output for the motor 907, and can also communicate and control with the processor 91 through interfaces such as SPI (Serial Peripheral Interface).

[0136] Backlight source 908 is used to provide illumination and adjust its brightness according to the control of processor 91, thereby adjusting the projection display brightness of the entire HUD display device. Backlight source 908 works in conjunction with image source 909 to implement the primary function of optical projection display. Backlight source 908 can be an LED (Light Emitting Diode), laser, or other device. Specifically, backlight source 908 is connected to processor 91 via a backlight driver chip. The backlight driver chip provides driving voltage for backlight source 908 and controls its brightness based on the pulse width signal output by processor 91.

[0137] Image source 909 is used to display an image of corresponding content and project display light corresponding to the image under the control of processor 91. Image source 909 can be an LCD (Liquid Crystal Display), a DMD (Digital Micromirror Devices), a MEMS (Micro-Electro-Mechanical System) micromirror, or an LCOS (Liquid Crystal on Silicon).

[0138] The positioning module 910 is used to monitor the position of the HUD display device and the corresponding vehicle. The positioning module 910 can be a global navigation satellite system such as the GPS (Global Positioning System) and the Beidou satellite navigation system. By measuring the distance between the satellite and the receiver on the positioning module 910 at different locations, the corresponding position and orientation data are determined. In some examples, the positioning module 910 may also include an inertial navigation system. Based on Newton's laws of mechanics, the acceleration of the positioning module 910 in the inertial reference system is measured, integrated over time, and transformed into a navigation coordinate system to obtain data such as speed, yaw angle, and position in the navigation coordinate system. Optionally, the inertial navigation system can assist the global navigation satellite system in achieving more accurate positioning and provide the processor 91 with corresponding position information.

[0139] The radar 911 is used to determine the position of a target object through electromagnetic waves, and can usually determine the distance of the target object from the vehicle where the radar 911 is located.

[0140] Camera 912 includes a vehicle body camera and an in-vehicle camera. The vehicle body camera is used to determine the location of a target object through visual recognition. The vehicle body camera can be a monocular camera or a binocular camera. The main difference between a monocular camera and a binocular camera is that a binocular camera can capture images from two different perspectives, thereby obtaining distance information in three-dimensional space. The in-vehicle camera is used to identify the behavior of the driver and passengers in the vehicle, including fatigue detection, distraction detection, expression recognition, gesture recognition, and gaze tracking. In this example, the in-vehicle camera can also specifically implement eye tracking.

[0141] In some examples, the positioning module 910, radar 911 and camera 912 can also be directly connected to the vehicle computer 92, and are not directly connected to the processor 91 of the HUD display device. For example, the vehicle computer 92 itself integrates a positioning module for position tracking and a radar and camera for automatic driving. The HUD display device can obtain the collected data of the positioning module, radar and camera in real time through communication with the vehicle computer 92.

[0142] In some examples, such as Figure 19As shown, the display device that implements the above-mentioned interaction method may specifically include a processor 81, a memory 82, an input device 83 and an output device 84, wherein the input device 83 may include an operation button connected to the display device, etc., and the display device may receive input control instructions and data through the input device 83. The output device 84 may include an image source integrated on the display device, etc., and the display device may output corresponding instructions or data to the output device 84. Furthermore, the memory 82 stores a computer program running on the processor 81, and the processor 81 implements the interaction method of the above example when executing the computer program. In some examples, a computer-readable storage medium stores a computer program, and the computer program implements the interaction method of the above example when executed by the processor.

[0143] like Figure 20 As shown, a vehicle can be equipped with the aforementioned HUD display device. Specifically, the HUD display device is integrated within the center console 10, for example, in front of the steering wheel. The HUD display device projects corresponding display light onto the vehicle's windshield 4 through its projection window 102. Viewers observing the area in front of the windshield 4 from the cockpit directly see a virtual image within the projection area 50. In addition to basic information such as vehicle speed, this virtual image can also project interactive information to assist users in making payments and accessing the vehicle in response to the presence of vehicle-related payment points around the vehicle, thereby enhancing user confidence in the entire interaction process and reducing its complexity. In some examples, the vehicle can also distribute a program for obtaining the interactive method described in the aforementioned example via the computer-readable storage medium described above, enabling convenient updates and upgrades to the vehicle's onboard HUD display device. It should be noted that the aforementioned vehicles are not limited to cars as a means of transportation, but may also include buses, trucks, excavators, motorcycles, trains, high-speed trains, ships, yachts, airplanes, spacecraft, and the like. The windshield onto which the projection is made is not limited to the front windshield of a car and may also be a transparent surface located elsewhere.

[0144] In conjunction with the above examples, the technical solutions involved in this application can be directly embodied as hardware, a software module executed by a control unit, or a combination of the two, that is, one or more steps and / or one or more step combinations, which can correspond to various software modules of a computer program flow, or to various hardware modules, such as ASICs (Application Specific Integrated Circuits), FPGAs (Field-Programmable Gate Arrays), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or any appropriate combination thereof. For the convenience of description, the above description is divided into various modules and described separately according to their functions. Of course, when implementing this application, the functions of each module can be implemented in the same or multiple software and / or hardware.

[0145] Through the description of the above examples, those skilled in the art can clearly understand that the present application can be implemented by means of software plus the necessary general-purpose hardware platform. Based on this understanding, the technical solution involved in this application, or the part that contributes to the existing technology, can be embodied in the form of a software product. The software is executed by a microcontroller unit and, depending on the required configuration, can include one or more microcontroller units of any type, including but not limited to a microcontroller unit, a microcontroller, a DSP (Digital Signal Processor), or any combination thereof. The software is stored in a memory, such as a volatile memory (such as a random access memory), a non-volatile memory (such as a read-only memory, a flash memory, etc.), or any combination thereof.

[0146] In summary, when a vehicle-related payment point is identified, the HUD display device triggers the projection of a first virtual image element. This first virtual image element not only serves as an interactive element for the user to control the vehicle-related payment point, but also provides the user with verification information and prompt information related to the vehicle-related payment point, allowing the user to make authorization operations in a trusted environment, thereby simply achieving control of the gate and other equipment in the vehicle-related payment point without additional operations. This application improves the security and reliability of vehicle-related payments, simplifies the process, and enhances the user experience.

[0147] It should be understood that although this specification includes some examples, none of these examples constitutes a single independent technical solution. This description is provided for clarity purposes only. Those skilled in the art should consider this specification as a whole. The technical solutions in the various examples may be appropriately combined to form other implementations that are understandable to those skilled in the art.

[0148] The series of detailed descriptions listed above are merely specific descriptions of feasible implementation methods of the present application. They are not intended to limit the scope of protection of the present application. Any equivalent implementation methods or modifications that do not deviate from the teachings of the present application should be included in the scope of protection of the present application.

Claims

1. An interactive method, characterized in that: include: In response to a vehicle-related payment point being located around the vehicle, the HUD display device projects at least a first virtual image element, where the first virtual image element is projected near the vehicle-related payment point to guide the user to establish a context associated with the vehicle-related payment point; The first virtual image element at least reflects a matching feature with the vehicle-related payment point, so as to support the user in performing authentication with the vehicle-related payment point by using the matching feature; In response to the authorization operation input by the user, at least part of the control function of the vehicle-related payment point is obtained, a control virtual connection is established between the HUD display device and the barrier gate to ensure control interaction between the two, and a specific button in the vehicle is switched to receive control operations, so that operating the button when the barrier gate is open triggers the barrier gate to close, and operating the button when the barrier gate is closed triggers the barrier gate to open; In response to the vehicle moving away from the vehicle-related payment point, the control function of the vehicle-related payment point is reclaimed, the established control virtual connection is released, and the vehicle-related payment point regains control of the passage of the vehicle-related payment point.

2. The interactive method according to claim 1, characterized in that In response to the presence of a vehicle-related payment point around the vehicle, the HUD display device projects at least a first virtual image element, including: The vehicle-related payment point includes a first vehicle-related payment point and a second vehicle-related payment point, and the HUD display device supports projecting a first virtual image element corresponding to the first vehicle-related payment point and a second virtual image element corresponding to the second vehicle-related payment point; In response to the authorization operation input by the user, obtaining at least part of the control function of the vehicle-related payment point includes: Upon receiving the user's authorization operation on the first virtual image element, obtaining at least part of the control functions of the first vehicle-related payment point; Upon receiving the user's authorization operation on the second virtual image element, at least part of the control function of the second vehicle-related payment point is acquired.

3. The interactive method according to claim 1, characterized in that: The first virtual image element is projected near the vehicle-related payment point to guide the user to establish a context associated with the vehicle-related payment point, including: When the vehicle-related payment point is facing the projection area of ​​the HUD display device, the first virtual image element is configured to display an information mark above or next to the vehicle-related payment point according to the position of the vehicle-related payment point; When the vehicle-related payment point deviates from the projection area of ​​the HUD display device, the first virtual image element is configured to display an arrow mark pointing in a specific direction according to the positional relationship between the vehicle and the vehicle-related payment point.

4. The interactive method according to claim 1, wherein: In response to the authorization operation input by the user, obtaining at least part of the control function of the vehicle-related payment point includes: Communicate with an external device and receive a user authorization operation forwarded by the external device.

5. The interactive method according to claim 1, characterized in that: In response to the presence of a vehicle-related payment point around the vehicle, the HUD display device projects at least the first virtual image element, and the interaction method includes: The voice assistant in the vehicle automatically wakes up to receive the authorization operation input by the user.

6. The interactive method according to claim 1, characterized in that: In response to the authorization operation input by the user, obtaining at least part of the control function of the vehicle-related payment point includes: Automatically opening the barrier to support the passage of the vehicle, and upon receiving the control operation input by the user, closing the barrier or reopening the barrier; When the barrier is open, other vehicles cannot identify the vehicle-related payment point; When the barrier is closed, other vehicles can identify the vehicle-related payment point to support the control function of seizing the vehicle-related payment point.

7. The interactive method according to claim 1, characterized in that: In response to the presence of a vehicle-related payment point around the vehicle, the HUD display device projects at least the first virtual image element, and the interaction method includes: The specific button in the vehicle automatically switches to receive the authorized operation input by the user; The specific keys include a play key, a volume key, an answer key or a menu key, and the original functions of the specific keys are temporarily disabled to receive the authorized operation input by the user.

8. A display device, characterized in that: The method comprises a memory, a processor and a computer program stored in the memory and running on the processor, wherein the processor implements the steps of the interaction method according to any one of claims 1 to 7 when executing the computer program.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the interactive method according to any one of claims 1 to 7 are implemented.

10. A means of transport, characterized in that: Includes the display device according to claim 8 or the computer-readable storage medium according to claim 9.

Images