AR-HUD-based software development system and method, electronic equipment and storage medium

Abstract

The invention provides a software development system and method based on AR-HUD, electronic equipment and a storage medium. The software development system comprises an external information data transmission module, an application program module and an HUD hardware module. Wherein the external information data transmission module is used for sending related information data of a target vehicle to the application program module; the application program module is used for performing logic function processing on the related information data and acquiring processed result data; and the HUD hardware module is used for carrying out U I and AR information display on the result data. According to the method and the device, the AR-HUD development complexity can be reduced, and the development efficiency is improved.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and more specifically, to a software development system, method, electronic device and storage medium based on AR-HUD. Background Art

[0002] With the rapid development of augmented reality (AR) and head-up display (HUD) technologies, AR-HUD products have been promoted and applied in vehicles. They can combine vehicle information and surrounding environment information, form corresponding virtual images based on various projection object information, and calculate the corresponding projection position to project it to the corresponding position in front of the driver, so that the driver can understand the current driving conditions and focus more on driving.

[0003] However, in the current AR-HUD technology development process, it is necessary to achieve accurate superposition of virtual images and real-world scenes, which requires developers to deeply understand and master a series of complex technologies. This not only places high demands on developers, but also fails to meet the customized needs of users. When connecting to the AR-HUD software development kit, various auto malls face high complexity and workload, resulting in problems such as high development difficulty, high cost and low efficiency.

[0004] Therefore, there is an urgent need for a software development solution based on AR-HUD that can improve the development efficiency of AR-HUD, reduce the development difficulty, and promote the interoperability and integration convenience of the entire industry. Summary of the invention

[0005] In view of the above problems, the purpose of this application is to provide a software development system based on AR-HUD to reduce the development complexity and workload of various automobile manufacturers when connecting to AR-HUD, reduce development difficulty, improve development efficiency, and accelerate the widespread application and promotion of ARHUD technology in the automotive industry.

[0006] In a first aspect, an embodiment of the present application provides a software development system based on AR-HUD, including: an external information data transmission module, an application module and a HUD hardware module; wherein:

[0007] The external information data transmission module is used to send relevant information data of the target vehicle to the application module;

[0008] The application module is used to perform logical function processing on the relevant information data and obtain the processed result data;

[0009] The HUD hardware module is used to display UI and AR information of the result data.

[0010] Further, the external information data transmission module sends the relevant information data via RFC;

[0011] The relevant information data includes the body domain, chassis domain, power domain, intelligent driving domain, and cockpit domain information of the target vehicle.

[0012] Furthermore, the application module includes a data processing layer, a data warehouse layer, a functional service layer, a view data layer and a view control layer; wherein,

[0013] The data processing layer is used to receive the relevant information data of the target vehicle, convert the relevant information data into internal data of the application module, and send the converted internal data to the data warehouse layer;

[0014] The data warehouse layer is used to store the internal data;

[0015] The functional service layer is used to execute business logic according to the internal data in the data warehouse, and form business view data, which is stored in the view data layer;

[0016] The view data layer is used to store the business view data to drive view update;

[0017] The view control layer is used to determine the display logic and display results of UI and AR according to the business view data provided by the view data layer;

[0018] The data processing layer, the data warehouse layer, the functional service layer, the view data layer and the view control layer are independent of each other.

[0019] Furthermore, the data processing layer includes an advanced driver assistance system adapter, a whole system adapter, a vehicle adapter, and a navigation adapter;

[0020] The data warehouse layer includes advanced driver assistance system storage, whole machine system storage, vehicle data storage, navigation data storage, and function storage;

[0021] The functional service layer includes a map navigation module, an auxiliary driving module, an infotainment module, a calibration service module, a vehicle information module, and a functional service module;

[0022] The view data layer includes an advanced driver assistance system view model, a whole system view model, a vehicle view model, a navigation view model, and a function view model;

[0023] The view control layer includes an AR entity controller and a UI view controller.

[0024] Further, the AR entity controller is used to operate the AR element display logic, and the AR elements include obstacles, navigation arrows, paved routes, guide lines, destinations, via points, and POIs;

[0025] The UI view controller is used to operate the UI element display logic, and the UI elements include vehicle speed, camera, lane information, road information, and TBT information.

[0026] Furthermore, the AR-HUD-based software development system also includes a common library connected to the functional service layer; wherein,

[0027] The common library includes a core algorithm library, a UI component library, a 3D component library, a graphics engine, a framework library and auxiliary tools; wherein,

[0028] The core algorithm library includes coordinate conversion algorithm, delay compensation algorithm, motion compensation algorithm and anti-shake algorithm;

[0029] The framework library includes base classes of the data processing layer, the data warehouse layer, the functional service layer, the view data layer and the view control layer, and the framework library is used to establish a communication mechanism between the data processing layer, the data warehouse layer, the functional service layer, the view data layer and the view control layer.

[0030] Furthermore, the AR-HUD-based software development system also includes an SDK layer located in the application module;

[0031] The SDK layer includes SDK functional services, SDK atomic services and SDK common libraries;

[0032] The SDK functional service includes the data warehouse layer, the calibration service module, the vehicle information module, the functional service module, and the functional view model;

[0033] The SDK atomic service includes the core algorithm library, the UI component library, and the 3D component library;

[0034] The SDK common library includes the graphics engine and the framework library;

[0035] In the data warehouse layer of the SDK functional service, the storage format and data structure of the data warehouse are defined by FrancaIDL.

[0036] In a second aspect, an embodiment of the present application provides a software development method based on AR-HUD, which uses the software development system based on AR-HUD in the first aspect to perform software development; wherein the method includes:

[0037] Sending relevant information data of the target vehicle to the application module through the external information data transmission module;

[0038] Performing logical function processing on the relevant information data through the application module, obtaining the processed result data and sending it to the HUD hardware module;

[0039] The result data is displayed in UI and AR information through the HUD hardware module.

[0040] In a third aspect, an embodiment of the present application provides an electronic device, including:

[0041] A processor and a memory, wherein the memory is used to store a computer program, and the processor is used to call and run the computer program stored in the memory to run the AR-HUD-based software development system in the first aspect.

[0042] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium for storing a computer program, wherein the computer program enables a computer to run the AR-HUD-based software development system in the first aspect.

[0043] By utilizing the above-mentioned AR-HUD-based software development system, method, electronic device and storage medium, the relevant information data of the target vehicle can be sent to the application module through the external information data transmission module, and the relevant information data can be processed by the application module through logical functions, such as data conversion, storage, logical operations, etc., and the processed result data can be obtained and sent to the HUD hardware module. Finally, the result data can be displayed as UI and AR information through the HUD hardware module, which can simplify the creation and optimization process of AR-HUD applications, so that both developers who are new to this field and experienced professional teams can understand and master the key technologies of AR-HUD system development more quickly, thereby reducing the troubles caused by technical barriers and improving the convenience and versatility of development.

[0044] In order to achieve the above and related purposes, one or more aspects of the present invention include the features that will be described in detail later. The following description and the accompanying drawings describe some exemplary aspects of the present invention in detail. However, these aspects indicate only some of the various ways in which the principles of the present invention can be used. In addition, the present invention is intended to include all these aspects and their equivalents. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] By referring to the following description in conjunction with the accompanying drawings, and with a more comprehensive understanding of the present invention, other objects and results of the present invention will become more clear and easy to understand. In the accompanying drawings:

[0046] Figure 1A functional block diagram of a software development system based on AR-HUD according to an embodiment of the present invention;

[0047] Figure 2 is a flowchart of a software development method based on AR-HUD according to an embodiment of the present invention;

[0048] Figure 3 A debugging principle diagram of a software development method based on AR-HUD according to an embodiment of the present invention;

[0049] Figure 4 is a schematic block diagram of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0050] In the following description, for the purpose of illustration, in order to provide a comprehensive understanding of one or more embodiments, many specific details are set forth. However, it is apparent that these embodiments may also be implemented without these specific details. In other examples, for ease of describing one or more embodiments, known structures and devices are shown in the form of block diagrams.

[0051] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product, or server that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products, or devices.

[0052] In order to solve the current AR-HUD technology development, which not only has high requirements for developers but also cannot meet the customized needs of users, each auto mall has high complexity and workload when connecting to the AR-HUD software development kit, resulting in the problems of great development difficulty, high cost and low efficiency. The embodiment of the present application designs a software development solution based on AR-HUD, constructs an external information data transmission module, an application module and a HUD hardware module, sends the relevant information data of the target vehicle to the application module through the external information data transmission module, performs data conversion, storage, logical operation and other processing on the relevant information data through the application module, and sends the processed result data to the HUD hardware module, and finally displays UI and AR information through the HUD hardware module, which can simplify the creation and optimization process of AR-HUD applications, so that both developers who are new to this field and experienced professional teams can understand and master AR-HUD system development more quickly, thereby reducing the troubles caused by technical barriers and improving the convenience and versatility of development.

[0053] In order to describe in detail the AR-HUD-based software development system and its use method in the present invention, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0054] Figure 1 The schematic principle of the AR-HUD software development system according to an embodiment of the present invention is shown.

[0055] like Figure 1 As shown, the AR-HUD-based software development system of the embodiment of the present invention includes three modules: an external information data transmission module (External info), an application module (AR-HUD App) and a HUD hardware module (HUD hardware); wherein the external information data transmission module is used to send relevant information data of the target vehicle to the application module; the application module is used to perform logical function processing on the relevant information data and obtain the processed result data; the HUD hardware module is used to perform UI (User Interface) and AR (Augmented Reality) information display on the result data. During the application process, the modules are independent of each other. After obtaining the relevant data, the information display of the corresponding results can be realized by integrating the application module, and the operation is more convenient and efficient.

[0056] Furthermore, the external information data transmission module can send relevant information data through RFC (Remote Procedure Call), and the relevant information data includes the target vehicle's body domain, chassis domain, power domain, intelligent driving domain, cockpit domain, etc. Among them, the so-called "domain" mainly refers to the collection of electronic and electrical architectures that control a large functional module of the car, and each domain is uniformly controlled by a domain controller.

[0057] Among them, the relevant information of the body domain includes: lights, wipers and washers, central door locks, window controls, PEPS (Power Entry Passive Start, i.e. keyless entry and start system) smart keys, low-frequency antennas, low-frequency antenna drivers, electronic steering column locks, IMMO (immobiliser, engine anti-theft locking system) antennas, gateway CAN (industrial bus), LIN (Local Interconnect Network) networks, Ethernet interfaces, TPMS (tire pressure monitoring system) and wireless receiving modules, etc.

[0058] Intelligent Driving Domain, the domain controller used in the field of autonomous driving can enable the vehicle to have the capabilities of multi-sensor fusion, positioning, path planning, and decision-making control. It usually requires multiple external cameras, millimeter-wave radars, lidars and other equipment. The functions completed include image recognition, data processing, etc., and there is no need to carry external industrial computers, control boards and other hardware, thereby providing support for different levels of computing power for autonomous driving. The core lies mainly in the processing power of the chip. The ultimate goal is to meet the computing power requirements of autonomous driving, simplify equipment, and greatly improve the integration of the system.

[0059] The cockpit domain mainly includes full LCD instrument, large screen central control system, in-vehicle infotainment system, head-up display system, streaming media rearview mirror, etc. The core control component is the domain controller. The cockpit domain controller (DCU, Drive Control Unit) realizes the integration of head-up display, instrument panel, navigation and other components through Ethernet / MOST (Media Oriented System Transport, information transmission system) / CAN (bus system). It not only has traditional cockpit electronic components, but also further integrates intelligent driving ADAS (Advanced Driving Assistance System, advanced driver assistance) system and vehicle networking V2X system, so as to further optimize intelligent driving, in-vehicle interconnection, infotainment and other functions.

[0060] The power domain is an intelligent powertrain management unit that uses CAN / FLEXRAY (bus system and communication system) to achieve transmission management, engine management, battery monitoring, and AC generator regulation.

[0061] The chassis domain is mainly related to the driving of the car, and is composed of the transmission system, driving system, steering system and braking system. The transmission system is responsible for transmitting the power of the engine to the drive wheels, and can be divided into mechanical, hydraulic and electric types. Among them, the mechanical transmission system is mainly composed of clutch, transmission, universal transmission device and drive axle, and the hydraulic transmission system is mainly composed of torque converter, automatic transmission, universal transmission device and drive axle; the driving system connects the various parts of the car into a whole and supports the whole car, such as the frame, suspension, wheels, and axles are all its parts; the steering system ensures that the car can drive straight or turn according to the driver's wishes; the braking system forces the road surface to exert a certain external force on the car wheels that is opposite to the direction of the car's driving, and performs a certain degree of forced braking on the car. Its function is to slow down and stop, and parking brake.

[0062] It can be seen that the body domain, chassis domain, power domain, intelligent driving domain, cockpit domain and other information of the above-mentioned target vehicle can be obtained through various channels. For example, as shown in the attached figure, the relevant information data can be obtained through DVR camera (corresponding information is Video info), intelligent driving system (corresponding information is ADAS info), DMS (Driver Monitoring System) (corresponding information is DMS info), IMU+GNSS (conventional measuring instrument + global navigation satellite system) (corresponding IMU&GNSS info), vehicle-mounted signal (corresponding information is DMS info), high-precision map / positioning / map navigation and other methods (corresponding Navi&Position&Map info), and the acquired data is sent to the application module through RFC (Remote Procedure Call) for processing.

[0063] Furthermore, the application module includes a data processing layer (adapter), a data warehouse layer (store), a functional service layer (Service), a view data layer (ViewModel) and a view control layer (View); wherein the data processing layer is used to receive relevant information data of the target vehicle, and convert the relevant information data into internal data of the application module, and send the converted internal data to the data warehouse layer; the data processing layer includes a variety of data converters such as an advanced driver assistance system adapter (System adapter), a whole system adapter (System adapter), a vehicle adapter (Vehicle adapter), and a navigation adapter (Navigation adapter). The function of this layer is to structurally convert external relevant information data, and combine FrancaIDL (Franca Interface Definition Language) to ensure the unification of the data structure entering the functional service layer and reduce the data requirements in the application process.

[0064] The data warehouse layer is used to store internal data, including advanced driver assistance system storage (ADAS store), whole system storage (System store), vehicle data storage (Vehicle store), navigation data storage (Navigationstore), function storage (Function store), etc. It can drive the function service layer to display TBT (Time-Based Triggering, which can trigger events in computers or other applications within a specified time interval), navigation arrows, destinations and other services.

[0065] Specifically, FrancaIDL can be used to pre-define data types in the data warehouse layer and generate corresponding header files. Then, when the data processing layer receives data sent by the external information data transmission module (for example, vehicle speed data, location data, etc.), it will send the data to the data warehouse layer through the FrancaIDL mechanism and assign values ​​to the header files, thereby completing the definitional storage of the data.

[0066] It can be seen that the conversion and organization of data requires the cooperation of the data warehouse layer and the data processing layer. FrancaIDL pre-defines the data type in the data warehouse layer, which belongs to the definition process, while the data processing layer belongs to the application of this mechanism. By referencing FrancaIDL, the data is stored in the header file of the data warehouse layer.

[0067] The functional service layer is used to execute business logic based on the internal data in the data warehouse and form business view data, which is stored in the view data layer. The functional service layer includes map navigation module, assisted driving module, infotainment module, calibration service module, vehicle information module, functional service module and other service types.

[0068] The view data layer is used to store business view data to drive view updates. The view data layer includes the advanced driver assistance system view model (ADAS ViewModel), the whole system view model (System ViewModel), the vehicle view model (Vehicle ViewModel), the navigation view model (Navigation ViewModel), and the function view model (Function ViewModel).

[0069] View control layer The view control layer includes AR entity controller (AR Entity) and UI view controller (UIView). This layer is used to determine the display logic and display results of UI and AR based on the business view data provided by the view data layer. Among them, the AR entity controller can be used to operate the display logic of AR elements, including obstacles, navigation arrows, paved routes, guide lines, destinations, via points, POIs, etc.; the UI view controller is used to operate the display logic of UI elements, including vehicle speed, camera, lane information, road information, TBT information, etc.

[0070] It should be noted that in the present application, the data processing layer, data warehouse layer, functional service layer, view data layer and view control layer are independent of each other and can collaborate through the interface definition language. Only the relevant information data is integrated within the functional service layer to reduce the correlation between layers, thereby improving flexibility in the development process.

[0071] In addition, the HUD hardware module may include hardware structures such as PGU (Power Generation Unit), drive board, motor and optical structure, which are used to intuitively display the display results of AR-HUD to users.

[0072] Furthermore, the AR-HUD-based software development system of the present application also includes a common library connected to the functional service layer; wherein the common library includes a core algorithm library, a UI component library, a 3D component library, a graphics engine, a framework library and auxiliary tools; wherein the core algorithm library includes a coordinate conversion algorithm, a delay compensation algorithm, a motion compensation algorithm, and an anti-shake algorithm; the UI component library includes components such as Text, Image, Box, Row, Column, Space, and Animator; the 3D component library includes components such as IText, ISprite, IGroup, IPoint, IPath, IGLLine, IGlowLine, and Animation; wherein Text is used to display text information and does not support user modification of text content; Image is used to display pictures; Box usually refers to a container component, which is used to wrap other components for unified management and layout; Row and Column belong to linear layout methods, and Row Represents horizontal arrangement of sub-elements, Column represents vertical arrangement of sub-elements, and the two are used together to achieve flexible layout; Space is used to adjust the spacing of layout spaces or filler components to control the spacing between components; Animation is used to achieve animation effects and enhance user experience; IText is used to generate documents and convert document formats, ISprite is an interface for rendering sprites and texture maps; IGroup is used to represent a group of input geometric objects, and IPoint represents an input point. These input data are used to control the behavior and appearance of 3D components; IPath represents a path; IGLLine is a basic element in the 3D component, used to represent a line. In 3D graphics programming, IGLLine is usually used to connect two or more vertices to form a line segment geometry, which can be a straight line or a curve; IGlowLine represents a luminous line; Animation is used to achieve animation effects and enhance user experience, etc.

[0073] The framework library includes base classes of the data processing layer, data warehouse layer, functional service layer, view data layer and view control layer. The framework library is used to establish communication mechanisms among the data processing layer, data warehouse layer, functional service layer, view data layer and view control layer. During the application process, users can flexibly set the UI component library and 3D component library according to production requirements to meet the usage needs of different users.

[0074] In a specific embodiment of the present application, it also includes an SDK (Software Development Kit) layer located in the application module, as shown in the red dotted line area in the attached figure; wherein the SDK layer includes SDK functional services, SDK atomic services and SDK common libraries; SDK functional services include data warehouse layer, calibration service module, vehicle information module, functional service module, functional view model; SDK atomic services include core algorithm library, UI component library, 3D component library; SDK common libraries include graphics engine and framework library; in the data warehouse layer of SDK functional services, the storage format and data structure of the data warehouse are defined by Franca IDL. Among them, the SDK layer also provides an atomic service interface, which mainly performs atomic operations on integer types and individual data bits, ensuring the atomicity of data operations in a multi-tasking environment, that is, the indivisibility of operations, thereby avoiding data competition and consistency problems, allowing developers to flexibly call these underlying services according to custom requirements to complete the conversion of data to visual information, and finally display the processed data on the optical machine of AR-HUD.

[0075] However, in response to different business needs, developers can use the SDK's built-in core algorithm library to perform fine-grained data processing, including but not limited to data filtering, time delay compensation, motion compensation, and anti-jitter processing, etc., to ensure data quality and stability; in the development process of the in-vehicle AR-HUD system, different automobile manufacturers can connect to various external data sources, and convert and organize the different types of data obtained according to the store data warehouse format pre-defined by FrancaIDL. The store here is equivalent to a standardized data storage and exchange center, which can specify the format and structure of all data when it is input into the SDK, so that it can be applied to various automobile manufacturers to meet the processing and application requirements of different data.

[0076] As a specific example, the functional-level service modules encapsulated in the SDK can simplify the software development process, such as: 1. Vehicle information function: Developers only need to pass relevant data such as the real-time latitude and longitude of the vehicle provided by the navigation system to the SDK according to the interface specifications defined in the store in the SDK. The SDK will use this information internally, including but not limited to the fusion of DR data, route data, and ADAS data, to calculate and optimize the position and posture of the vehicle, ensuring the best display effect of the vehicle position and status on the AR-HUD. 2. Path guidance function: For information such as navigation paths, navigation arrows, and destinations, developers only need to input relevant external data generated by the navigation system into the SDK according to the store interface. The SDK will convert these data and the vehicle position and posture data processed by the vehicle information function into points in the vehicle coordinate system through a coordinate conversion algorithm, and finally display the driving path, navigation arrow guidance, and destination signs in real time and accurately on the optical machine of the AR-HUD hardware module, providing drivers with an intuitive and vivid augmented reality navigation experience.

[0077] It can be seen that the AR-HUD-based software development system of this application adopts a hierarchical architecture, divides the data processing flow into multiple levels, and uses the pre-set standard interfaces, modular components, and easy-to-use atomic services of the data warehouse layer, so that both developers who are new to this field and experienced professional teams can more quickly understand and master the development technology of the AR-HUD system, thereby reducing the troubles caused by technical barriers. Therefore, the development system of this application is committed to significantly reducing the time cost of the AR-HUD project from conception to actual implementation. By providing one-stop resource support, a highly integrated development environment, and efficient debugging tools, it can help developers write code, test verification, and iterate functions more efficiently, thereby greatly shortening the product launch cycle and accelerating the widespread application and promotion of AR-HUD technology in the automotive industry.

[0078] Figure 2 A flowchart of a software development method based on AR-HUD provided in an embodiment of the present application is shown.

[0079] like Figure 2 As shown, the AR-HUD-based software development method implemented in the present application uses the AR-HUD-based software development system in the first aspect to develop software, and the method includes:

[0080] S110: Sending relevant information data of the target vehicle to the application module through the external information data transmission module;

[0081] S120: Performing logical function processing on the relevant information data through the application module, obtaining the processed result data and sending it to the HUD hardware module;

[0082] S130: Display the result data in UI and AR information through the HUD hardware module.

[0083] As an example, Figure 3 The debugging principle of the software development method based on AR-HUD according to the embodiment of the present application is shown, as shown in FIG. Figure 3 As shown, during the debugging process of the AR-HUD-based software development method in the embodiment of the present application, relevant information data can be sent through the vehicle computer, and an application program integrated with the AR-HUD-based software development system can be run through a PC, wherein the vehicle computer data can be received by the adapter in the AR-HUD-based software development system and decoded and converted into the data format required by the AR-HUD SDK interface, and stored in the data warehouse Store, the vehicle computer data is transmitted to a personal computer (PC), all data is processed on the PC side according to the same standard format, and sent to the SDK repository for display on display hardware, such as a monitor.

[0084] It can be seen that this method can eliminate the optical and mechanical dependence of HUD. After the function development is completed, it is only necessary to configure the optical and mechanical parameters of the corresponding real vehicle on the SDK side to complete the integration. This not only lowers the technical threshold for development, but also greatly saves development time.

[0085] Figure 4 An electronic device provided according to an embodiment of the present application is shown, comprising: a processor and a memory, wherein the memory is used to store a computer program, and the processor is used to call and run the computer program stored in the memory to run the AR-HUD-based software development system in the first aspect.

[0086] like Figure 4 As shown, the electronic device 1 may include a processor 10, a memory 11 and a bus, and may also include a computer program stored in the memory 11 and executable on the processor 10, such as an AR-HUD-based software development system program 12.

[0087] Among them, the memory 11 includes at least one type of readable storage medium, and the readable storage medium includes flash memory, mobile hard disk, multimedia card, card-type memory (for example: SD or DX memory, etc.), magnetic memory, disk, optical disk, etc. The memory 11 may be an internal storage unit of the electronic device 1 in some embodiments, such as a mobile hard disk of the electronic device 1. The memory 11 may also be an external storage device of the electronic device 1 in other embodiments, such as a plug-in mobile hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (SecureDigital, SD) card, a flash card (Flash Card), etc. equipped on the electronic device 1. Further, the memory 11 may also include both an internal storage unit of the electronic device 1 and an external storage device. The memory 11 can not only be used to store application software and various types of data installed in the electronic device 1, such as the code of the software development system program based on AR-HUD, but also can be used to temporarily store data that has been output or is to be output.

[0088] The processor 10 may be composed of an integrated circuit in some embodiments, for example, a single packaged integrated circuit, or a plurality of integrated circuits with the same or different functions, including one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and combinations of various control chips. The processor 10 is the control core (Control Unit) of the electronic device, and uses various interfaces and lines to connect the various components of the entire electronic device, and executes or executes programs or modules (such as software development system programs based on AR-HUD, etc.) stored in the memory 11, and calls the data stored in the memory 11 to execute various functions of the electronic device 1 and process data.

[0089] The bus may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. The bus is configured to realize connection and communication between the memory 11 and at least one processor 10, etc.

[0090] It can be seen that Figure 4Only an electronic device with components is shown. It can be understood by those skilled in the art that the structure shown does not constitute a limitation on the electronic device 1, and it can include fewer or more components than shown, or combine certain components, or arrange components differently. For example, although not shown, the electronic device 1 can also include a power supply (such as a battery) to power each component. Preferably, the power supply can be logically connected to the at least one processor 10 through a power management device, so that the functions of charging management, discharging management, and power consumption management are realized through the power management device. The power supply can also include any components such as one or more DC or AC power supplies, recharging devices, power failure detection circuits, power converters or inverters, power status indicators, etc. The electronic device 1 can also include a variety of sensors, Bluetooth modules, Wi-Fi modules, etc., which will not be repeated here.

[0091] Furthermore, the electronic device 1 may also include a network interface. Optionally, the network interface may include a wired interface and / or a wireless interface (such as a WI-FI interface, a Bluetooth interface, etc.), which is generally used to establish a communication connection between the electronic device 1 and other electronic devices.

[0092] Optionally, the electronic device 1 may further include a user interface, which may be a display, an input unit (such as a keyboard), or a standard wired interface or a wireless interface. Optionally, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, and an OLED (Organic Light-Emitting Diode) touch device. The display may also be appropriately referred to as a display screen or a display unit, which is used to display information processed in the electronic device 1 and to display a visual user interface.

[0093] It should be understood that the embodiment is for illustration only and is not limited to the scope of the patent application. The AR-HUD-based software development system program 12 stored in the memory 11 in the electronic device 1 is a combination of multiple instructions. When running in the processor 10, it can achieve:

[0094] Sending relevant information data of the target vehicle to the application module through the external information data transmission module;

[0095] Performing logical function processing on the relevant information data through the application module, obtaining the processed result data and sending it to the HUD hardware module;

[0096] The result data is displayed in UI and AR information through the HUD hardware module.

[0097] Furthermore, if the module / unit integrated in the electronic device 1 is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. The computer-readable medium may include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a mobile hard disk, a magnetic disk, an optical disk, a computer memory, and a read-only memory (ROM).

[0098] It should be understood that the method embodiment, the electronic device embodiment and the system embodiment may correspond to each other, and similar descriptions may refer to the system embodiment. To avoid repetition, they will not be described here.

[0099] In the several embodiments provided by the present invention, it should be understood that the disclosed devices, apparatuses and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of the modules is only a logical function division, and there may be other division methods in actual implementation.

[0100] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0101] In addition, each functional module in each embodiment of the present invention may be integrated into one processing unit, each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of hardware plus software functional modules.

[0102] It is obvious to those skilled in the art that the present invention is not limited to the details of the above exemplary embodiments, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.

[0103] Therefore, no matter from which point of view, the embodiments should be regarded as illustrative and non-restrictive, and the scope of the present invention is limited by the appended claims rather than the above description, so it is intended that all changes falling within the meaning and scope of the equivalent elements of the claims are included in the present invention. Any attached figure mark in the claims should not be regarded as limiting the claims involved.

[0104] In addition, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices stated in the system claim can also be implemented by one unit or device through software or hardware. The second and other words are used to indicate names, but not to indicate any particular order.

[0105] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the spirit and scope of the technical solution of the present invention.

[0106] As described above, the AR-HUD-based software development system according to the present invention is described by way of example with reference to the accompanying drawings. However, those skilled in the art should understand that various improvements can be made to the AR-HUD-based software development system proposed by the present invention without departing from the content of the present invention. Therefore, the protection scope of the present invention should be determined by the content of the attached claims.

Claims

1. A software development system based on AR-HUD, characterized in that: include: External information data transmission module, application module and HUD hardware module; among them, The external information data transmission module is used to send relevant information data of the target vehicle to the application module; The application module is used to perform logical function processing on the relevant information data and obtain the processed result data; The HUD hardware module is used to display UI and AR information of the result data.

2. The AR-HUD-based software development system according to claim 1, characterized in that: The external information data transmission module sends the relevant information data via RFC; The relevant information data includes the body domain, chassis domain, power domain, intelligent driving domain, and cockpit domain information of the target vehicle.

3. The AR-HUD-based software development system according to claim 1, characterized in that: The application module includes a data processing layer, a data warehouse layer, a functional service layer, a view data layer and a view control layer; wherein, The data processing layer is used to receive the relevant information data of the target vehicle, convert the relevant information data into internal data of the application module, and send the converted internal data to the data warehouse layer; The data warehouse layer is used to store the internal data; The functional service layer is used to execute business logic according to the internal data in the data warehouse, and form business view data, which is stored in the view data layer; The view data layer is used to store the business view data to drive view update; The view control layer is used to determine the display logic and display results of UI and AR according to the business view data provided by the view data layer; The data processing layer, the data warehouse layer, the functional service layer, the view data layer and the view control layer are independent of each other.

4. The AR-HUD-based software development system according to claim 3, characterized in that: The data processing layer includes an advanced driver assistance system adapter, a whole system adapter, a vehicle adapter, and a navigation adapter; The data warehouse layer includes advanced driver assistance system storage, whole machine system storage, vehicle data storage, navigation data storage, and function storage; The functional service layer includes a map navigation module, an auxiliary driving module, an infotainment module, a calibration service module, a vehicle information module, and a functional service module; The view data layer includes an advanced driver assistance system view model, a whole system view model, a vehicle view model, a navigation view model, and a function view model; The view control layer includes an AR entity controller and a UI view controller.

5. The AR-HUD-based software development system according to claim 4, characterized in that: The AR entity controller is used to operate the AR element display logic, and the AR elements include obstacles, navigation arrows, paved routes, guide lines, destinations, via points, and POIs; The UI view controller is used to operate the UI element display logic, and the UI elements include vehicle speed, camera, lane information, road information, and TBT information.

6. The AR-HUD-based software development system according to claim 4, characterized in that: It also includes a common library connected to the functional service layer; wherein, The common library includes a core algorithm library, a UI component library, a 3D component library, a graphics engine, a framework library and auxiliary tools; wherein, The core algorithm library includes coordinate conversion algorithm, delay compensation algorithm, motion compensation algorithm and anti-shake algorithm; The framework library includes base classes of the data processing layer, the data warehouse layer, the functional service layer, the view data layer and the view control layer, and the framework library is used to establish a communication mechanism between the data processing layer, the data warehouse layer, the functional service layer, the view data layer and the view control layer.

7. The AR-HUD-based software development system according to claim 6, characterized in that: Also included is an SDK layer located within the application module; The SDK layer includes SDK functional services, SDK atomic services and SDK common libraries; The SDK functional service includes the data warehouse layer, the calibration service module, the vehicle information module, the functional service module, and the functional view model; The SDK atomic service includes the core algorithm library, the UI component library, and the 3D component library; The SDK common library includes the graphics engine and the framework library; In the data warehouse layer of the SDK functional service, the storage format and data structure of the data warehouse are defined by FrancaIDL.

8. A software development method based on AR-HUD, characterized in that: Software development is performed using the AR-HUD-based software development system as described in any one of claims 1 to 7; wherein the method comprises: Sending relevant information data of the target vehicle to the application module through the external information data transmission module; Performing logical function processing on the relevant information data through the application module, obtaining the processed result data and sending it to the HUD hardware module; The result data is displayed in UI and AR information through the HUD hardware module.

9. An electronic device, characterized in that: include: A processor and a memory, wherein the memory is used to store a computer program, and the processor is used to call and run the computer program stored in the memory to run the AR-HUD-based software development system described in any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that: Used to store a computer program, wherein the computer program enables a computer to run the AR-HUD-based software development system according to any one of claims 1 to 8.

Images