System and method for preparing and displaying graphics on vehicle windshield

Abstract

FIELD: automotive engineering.SUBSTANCE: group of inventions relates to the technology of visually displaying graphics to the driver of a vehicle on the windshield. The system for displaying information on the windshield of a vehicle in real time comprises supporting communication channels for exchanging data: vehicle systems for receiving and / or transmitting data about vehicle systems and / or the environment, a device for receiving commands for displaying graphics, a computing module of an on-board computer (OBC) for processing a user request, a HUD display for displaying information on the windshield of a vehicle, an external data processing device configured to load an application into a software and computing system (SCS) and an SCS for providing information display on the windshield of a vehicle.EFFECT: functionality of systems, devices and methods for displaying information on the windshield (HUD display) is expanded while maintaining high safety during the operation of vehicles with HUD display functions due to the provision of a secure infrastructure for the implementation of applications from both the automaker and third-party developers for displaying graphics of various functionality on the windshield of the vehicle.8 cl, 1 dwg

Description

[0001] Technical field

[0002] The presented technical solution relates to vehicles, namely to the technology of visually displaying graphics to the driver of a vehicle on the windshield.

[0003] Technology Level

[0004] Currently, there are numerous systems designed to display various graphic elements on the windshield via a Head-Up Display (HUD). Typically, HUD applications in vehicles, particularly cars, are developed and implemented by vehicle manufacturers to control the application's operation and thus ensure safety.

[0005] A system for overlaying visual content (augmented reality) on a vehicle windshield is known, described in US 11640812. This solution provides for vehicle systems, graphical interfaces in a vehicle, as well as a system including a computer operatively connected to a network and containing a computing module, a program instruction storage module, operatively connected input / output devices, including a graphic system of an image output device, which provides for determining the coordinates of graphics on the windshield. As a result, augmented reality is displayed on the vehicle windshield, which can be perceived by the vehicle driver (Patent US 11640812; B60K 35 / 00; G01C 21 / 36; G02B 27 / 01; G09G 5 / 373; G09G 5 / 38; published 02.05.2023).

[0006] However, the known solution has a number of shortcomings, including limited functionality of the system and the method for displaying the projection on the windshield due to insufficient technical equipment to ensure the proper level of security when installing applications from both the car manufacturer and third-party developers for displaying graphics on the windshield of the vehicle, including, in addition to augmented reality, graphics of various functionality on the projection plane of the HUD display.

[0007] Disclosure of invention

[0008] The objective of the presented technical solution is to eliminate the above-mentioned shortcomings and ensure the display of various graphic elements (including augmented reality (AR), virtual reality (VR), user interface, text, graphics, objects, etc.) on the windshield through the Head-Up Display (HUD display).

[0009] The technical result of the invention is the expansion of the functional capabilities of systems, devices and methods for displaying information on the windshield (HUD display) while maintaining high safety during the operation of a vehicle with HUD display functions due to the provision of a secure infrastructure for the implementation of applications from both the automaker and third-party developers for displaying graphics of various functionality on the windshield of a vehicle.

[0010] To achieve the technical result, a system for displaying information on the windshield of a vehicle (TS) in real time is proposed, containing

[0011] supporting communication channels for data exchange: vehicle systems for receiving and / or transmitting data about vehicle systems and / or the environment, a device for receiving commands for displaying graphics, a HUD display, an on-board computer (OC), a computing module of the on-board computer (OC) for processing at least one user request, a HUD display for displaying information on the vehicle windshield, an external data processing device configured to load at least one application into the software and computing complex (SCC),

[0012] PVK for providing the output of information on the windshield of a vehicle, including an application storage module, a shader component for correcting and / or compensating for optical distortions in the projection on the windshield of the vehicle, a module for controlling the validity of at least one function call in accordance with the basic rules of applications, a module for preparing a visual scene, a module for launching at least one application, a human-machine interface control module for filtering the said request in accordance with the rules for processing applications, a PVC router for providing data exchange within the PVC.

[0013] Additionally, the system contains a graphical system within the BC, designed with the ability to process information for its visual presentation to the user.

[0014] Additionally, the system contains a module for storing pre-installed BC applications, containing at least one application integrated into the BC and configured with the possibility of sending control functions - data of the said application to the PVC.

[0015] Additionally, the system contains a TC router configured to receive data from TC systems and / or an external data processing device and transmit said data to the BC.

[0016] Additionally, the system contains routers BK-PVK as part of the BK and PVC-BK as part of the PVC, which are connectors or network devices of the second (channel) level, configured for transmitting and / or receiving data about the vehicle and control functions of the vehicle.

[0017] To achieve the technical result, a method for displaying information on the windshield of a vehicle in real time is also proposed, characterized by the fact that:

[0018] - at least one application is loaded into the software and computing complex (SCC) on the external data processing device;

[0019] - at least one user request is executed using a device for receiving commands for outputting graphics;

[0020] - using the computing unit of the on-board computer (BC), the mentioned request is processed and sent to the software and computing complex (SCC);

[0021] - via the PVC router, ensure the exchange of data within the PVC between the PVC computing module, the shader component, the function call validity control module in accordance with the basic application rules, the visual scene module, the application storage module, the application launcher module, and the human-machine interface control module;

[0022] - the computing module of the PVC transmits a request to the application storage module to launch the application; the data of this request is sent to the function call validity control module in accordance with the basic application rules to check the validity of the application for the vehicle before and / or after sending the request to launch the application to the application storage module;

[0023] - then send a request to the computing module of the HMI to launch the mentioned application, from where the request from the launched application is transmitted to the application launch module, where it is checked against the human-machine interface control module;

[0024] - then send the said request to the visual scene module, where the visual scene is composed while constantly checking with the human-machine interface control module;

[0025] - then send the said request to the shader component to correct and / or compensate for optical distortions in the projection onto the vehicle windshield, and then send the functions of the said application as a projection onto the vehicle windshield.

[0026] Additionally, the method comprises a step in which the human-machine interface control module filters out duplicate functionality of the application and disables functions that distract attention from the driver.

[0027] Furthermore, the method comprises a step in which, if the application is not confirmed to have passed validation, then the said application is not launched and / or does not launch the specific function that has not passed validation.

[0028] Below are explanations of some terms used in the description of the invention.

[0029] The system contains an external application processing module and a vehicle, which in turn contains vehicle systems, an on-board computer (OC), a software and computing complex (SCC) and input / output devices: a device for receiving commands for displaying graphics and a HUD display for displaying information on the vehicle windshield.

[0030] An external data processing device (e.g., ATOM Cloud) is an external database (server) capable of performing offline and manual verification of downloaded applications and providing the ability to download said applications to the PVC, as well as exchanging data with an automated workstation (AWS) for creating applications.

[0031] The vehicle systems shall be understood to mean all the subsystems and systems of the vehicle that transmit data on the state of the vehicle and / or the environment to the on-board computer (In-Vehicle Infotainment, IVI) and to the software and computing complex (SCC). Thus, the vehicle systems in the context of this invention are a set of devices and products installed in the vehicle and transmitting data on the state of the vehicle, on the vehicle components and on the environment. Information on the state of the vehicle includes such data as, for example, the status of the vehicle (in motion, parked, etc.), information on the state of the vehicle components includes such data as, for example, tire pressure, door position (closed / open), etc.; information on the environment includes such data as, for example, the temperature outside the vehicle, the weather forecast, etc., and the mentioned information may also vary depending on the systems, sensors, radars, detectors, cameras, etc. installed on the vehicle.

[0032] The on-board computer (OC) is functionally connected to the vehicle systems and output devices, such as the HUD display and a device for receiving commands for displaying graphics with a user interface installed on it. The OC comprises a computing module, a storage module for pre-installed applications, and a graphics system. The OC computing module is designed to process commands from the user (e.g., the vehicle driver) and to process information received from all available sources, including, but not limited to, the graphics system, vehicle systems, and the on-board computer. The graphics system is a module responsible for displaying information on devices installed in the vehicle, designed with the ability to process information for presentation to the user in visual form via, for example, a HUD display and / or SWP / SWD (steering wheel panel / steering wheel display).The pre-installed application storage module is a module that contains a collection of fundamental elements, namely, applications already integrated into the on-board computer (opening / closing door(s), player, light adjustment, ADAS functions such as cruise control, lane keeping control, etc.).

[0033] A software and computing system (SCS) for displaying information on a vehicle's windshield (such as the AR Engine of the ATOM electric vehicle) is designed to enable user interaction with the vehicle via a visual series through a HUD display and SWP / SWD. The SCS functionality is provided by receiving and converting information from vehicle systems and applications from the application storage module into a format suitable for the HUD display based on HMI standards and safety requirements. The SCS is adapted to the requirements of automotive technologies and can be used in any vehicle equipped with HUD systems. The SCS is functionally connected to the vehicle and receives data from it, in particular from the In-Vehicle Infotainment system. The SCS is designed to present an image in isometric view (3 image layers) using a UI layout (not shown in Figure 1).

[0034] The PVC includes an application storage module, a shader component for correcting and / or compensating for optical distortions in the projection onto the vehicle windshield, a module for monitoring the validity of function calls in accordance with the basic application rules, a visual scene preparation module, an application launch module, a human-machine interface control module for filtering requests in accordance with the application processing rules, and a PVC router for ensuring data exchange within the PVC.

[0035] - The application storage module is an application database within the PVC, which stores all third-party applications, i.e. at least one application downloaded from an external data processing device (for example, from ATOM cloud).

[0036] - The shader component is a module for correcting and / or compensating for optical distortions in the projection onto the vehicle windshield via the HUD display.

[0037] - The function call validity control module, in accordance with application base rules, is designed to control the validity of function calls within the PVC in accordance with policies. Policies are a set of application base rules that can be updated by the manufacturer on their workstation. For example, if a vehicle is equipped with an internal combustion engine, the function call validity control component, in accordance with application base rules, will not allow the battery charge notification function.

[0038] - The Visual Scene / Canvas module is responsible for the visual scene, i.e. the location of specific graphics on the windshield, and also communicates with the HMI control module to match the image being sent.

[0039] - RunTime is a module that pre-launches applications to perform appropriate checks before projecting them onto the HUD display.

[0040] - The Human Machine Interface Control Module (HMI Control Module) is designed for filtering. Specifically, it contains a set of rules that identifies application information, their rank / category / tag, when sending it to the visualization system (HUD or SWP). If the system contains applications with duplicate functionality, such as displaying vehicle speed, only one of them will be used for projection. If a user-installed application includes lighting effects (backlighting, HUD, spectrogram) while listening to music, then when driving, the driver-distracting feature (specifically, lighting effects) will be disabled, while music playback will continue.

[0041] According to one of the options, the PVC may additionally include a module of basic PVC applications - a module containing a collection of fundamental applications, namely, those integrated into the PVC by default (for example, “Vehicle Speed”, “Battery Charge”, applications for displaying ADAS graphics, and much more).

[0042] According to one of the options, the HUD may additionally include a HUD computing module - a module for collecting, processing, and sending information about the HUD for display.

[0043] Data transfer between the mentioned components can be carried out via routers, such as the TS router, the BK-PVK router, and the PVC-BK router. The TS router is configured to receive data from the vehicle systems and the application storage database and transmit the said data to the on-board computer (BC). The BK-PVK router is configured to connect the computing module of the on-board computer (BC) with the software and computing complex with the ability to transmit / receive data about the vehicle and control functions of the vehicle. The PVC-BK router is designed to connect the software and computing complex with the computing module of the on-board computer (BC) with the ability to transmit / receive data about the vehicle and control functions of the vehicle. In this case, the mentioned routers can be implemented both as connectors and as a network device of the second (data link) layer, i.e. a network bridge for combining segments / elements into a single network.

[0044] Device for receiving graphics output commands - a device containing a user interface for receiving commands. This may be a user's mobile device, a tablet / vehicle display, including a steering wheel tablet for receiving graphics output commands (SWP / SWD).

[0045] HUD display is an information display device designed to display symbolic and graphic information on the windshield of a vehicle.

[0046] The function call validity control module, in accordance with application base rules, is designed to control the validity of function calls within the PVC in accordance with policies. Policies are a set of application base rules that can be updated by the BKY manufacturer on their workstation. For example, if a vehicle is equipped with an internal combustion engine, the function call validity control component, in accordance with application base rules, will not allow the battery charge notification function.

[0047] The base application module and the pre-installed application storage module may include program instructions that can be executed by the processor to implement any element or action described herein.

[0048] Brief description of drawings

[0049] Fig. 1 shows a basic diagram of the system for displaying information on the windshield of a vehicle.

[0050] The following is a description of the implementation of the method of the declared technical solution - a step-by-step method for launching a downloaded application.

[0051] Using an external data processing device, autonomous and / or manual verification of applications downloaded from the application creation workstation is performed, followed by uploading these applications to the PVC, specifically the application storage module. During the autonomous verification phase, static code verification, object model compliance, and basic HMI compatibility verification are performed. Object model compliance includes checking the correctness of the list of properties, fields, etc., while the basic HMI compatibility verification includes checking the permissible application functionality (e.g., application tags) for bugs and vehicle safety. During the manual verification phase, test runs of the application are performed and the code is proofread.

[0052] Next, the downloaded application is launched to display its functions on the HUD display. To do this, the user interacts with the device via the user interface to receive commands for displaying graphics, selects the application to launch, and sends a request to the HUD. The HUD then processes the request, specifically, the request data is processed to visually present the information to the user via the graphics system and then transmitted to the HUD computing module, where it optionally receives data about the vehicle, its control functions, and / or the surrounding environment. Monitoring or data exchange is performed in real time to ensure the displayed image is consistent with actual conditions. The request is then routed through the HUD-BC routers (as part of the HUD) and the HUD-VC routers (as part of the HUD), which enable the transfer of the application request and information about the vehicle's systems from the vehicle to the HUD and vice versa.According to one option, the PVK also transmits control functions retrieved from the preinstalled app storage module (HUD on / off, door(s) opening / closing, light adjustment, ADAS functions such as cruise control, lane keeping assist, etc.). For example, when the "Media Library" app is launched, a request to launch the app, vehicle status data (parked / driving), and data from the preinstalled "Player" app are transmitted.

[0053] Next, in the PVC, the data is sent to the PVC router, which ensures the exchange of data between all PVC components.

[0054] The ATC computing module transmits information to the validity control module regarding at least one application function call in accordance with the application's basic rules, i.e., whether the application can be launched on the vehicle as a whole. The ATC computing module then receives the verification results and transmits a request to the ATM Adapter to launch the said application. The said application is then sent to the said validity control module for a second validation check. In one embodiment, the application's validity for the vehicle is checked before or after sending the application launch request to the application storage module.

[0055] Next, a request to pre-launch the application is sent to the VCS computing module. From there, the request from the running application is forwarded to the Runtime module, where it is also verified with the HMI control module in real time. This verification with the HMI control module filters out duplicate application functionality to prevent the repeated launch of an already active function, and also ensures that functions that could distract the driver while driving are disabled. This functionality is implemented based on the tags of the application in question and vehicle status information.

[0056] The scene is then composed in the Visual Scene (UI) Module, constantly checking with the HMI Control Module to filter the request according to application processing rules. After the scene is composed, the application is sent to the shader component, where distortion, chromatic aberration compensation is performed when projected onto the vehicle's windshield. The prepared display of the function(s) of the application is then sent to the graphics system for processing, from where it is then projected onto the vehicle's windshield via the HUD.

[0057] As a result, the functionality of systems, devices and methods for displaying information on the windshield (HUD display) is ensured while maintaining high safety during the operation of vehicles with HUD display functions due to the provision of a secure infrastructure for the implementation of applications from both the car manufacturer and third-party developers for displaying graphics of various functionality on the windshield of the vehicle.

Claims

1. A system for displaying information on a vehicle (V) windshield in real time, comprising supporting communication channels for exchanging data: VS systems for receiving and / or transmitting data on VS systems and / or the environment, a device for receiving commands for displaying graphics, an on-board computer (OC) computing module for processing at least one user request, a HUD display for displaying information on a VS windshield, an external data processing device configured to load at least one application into a software and computing system (SCS), a SCS for displaying information on a VS windshield, including an application storage module, a shader component for correcting and / or compensating for optical distortions in projection onto the VS windshield, a module for monitoring the validity of at least one function call in accordance with the basic rules of the applications, a visual scene preparation module,a module for launching at least one application, a human-machine interface control module for filtering said request in accordance with application processing rules, a PVC router for ensuring data exchange within the PVC.

2. A system for displaying information on the windshield of a vehicle according to paragraph 1, characterized in that it contains, as part of the BC, a graphic system designed with the ability to process information for its visual presentation to the user.

3. A system for displaying information on the windshield of a vehicle according to paragraph 1, characterized in that it contains a module for storing pre-installed BC applications, containing at least one application integrated into the BC and configured with the ability to send control functions - data of said application to the PVC.

4. A system for displaying information on the windshield of a vehicle according to paragraph 1, characterized in that it contains a vehicle router configured to receive data from vehicle systems and / or an external data processing device and transmit said data to the on-board computer.

5. The system for displaying information on the windshield of a vehicle according to paragraph 1, characterized in that it contains routers BK-PVK as part of the BK and PVCK-BK as part of the PVC, which are connectors or network devices of the second (channel) level, configured for transmitting and / or receiving data about the vehicle and control functions of the vehicle.

6. A method for displaying information on the windshield of a vehicle in real time, characterized by the fact that: - using a device for receiving commands for outputting graphics, at least one user request is executed; - using the computing unit of the on-board computer (BC), the said request is processed and sent to the software and computing complex (SCC); - via the PVC router, data exchange within the PVC is ensured between the PVC computing module, the shader component, the function call validity control module in accordance with the basic application rules, the visual scene module, the application storage module, the application launch module, and the human-machine interface control module; - a request to launch an application is transmitted from the computing module of the PVC to the application storage module; the data of said request is sent to the function call validity control module in accordance with the basic application rules to check the validity of the application for the TS before and / or after sending the request to launch the application to the application storage module; - then they send a request to the computing module of the HMI to launch the mentioned application, from where they transmit the request from the launched application to the application launch module, where they check it against the human-machine interface control module; - then send the said request to the visual scene module, where the visual scene is composed while constantly checking with the human-machine interface control module; - then the said request is sent to the shader component to correct and / or compensate for optical distortions in the projection onto the vehicle windshield, and then the functions of the said application are sent in the form of a projection onto the vehicle windshield.

7. A method for displaying information on the windshield of a vehicle according to paragraph 6, characterized in that the human-machine interface control module filters out duplicate functionality of the application and disables functions that distract attention from the driver.

8. A method for displaying information on the windshield of a vehicle according to paragraph 6, characterized in that if validation is not confirmed for an application, then said application does not launch and / or does not launch a specific function that has not been validated.

Images