A universal communication architecture for HUD products and implementation method

Abstract

The application discloses a general communication architecture and implementation method for a HUD product, the architecture adopts a layered decoupling+process isolation design, the layers include a business processing layer, a ZeroMQ communication layer, a data serialization layer, a communication management layer and a protocol abstraction layer; the protocol abstraction layer defines a unified ICommAdapter interface, encapsulates multiple vehicle-mounted buses such as CAN-FD, SOME / IP, IIC and FDBUS, and shields the differences between the underlying protocols; the communication management layer uniformly manages various communication adapters; the data serialization layer realizes efficient binary encoding and decoding based on Protobuf; the communication module independently runs in a process, asynchronously interacts with a Kanzi business process through a ZeroMQ publishing and subscribing mode, and eliminates polling overhead in combination with an event-driven callback mechanism. The application solves technical problems such as great development difficulty, long cycle, cross-platform adaptation difficulty and the like of existing ARHUD / 3DHUD / PHUD products caused by the diversity of communication protocols, and realizes one-time development and multi-platform adaptation.

Description

Technical Field

[0001] This invention relates to the field of vehicle HUD communication technology, specifically to a general communication architecture and implementation method for HUD products. It is a data interaction and communication architecture applicable to various vehicle head-up display devices such as AR-HUD, 3D-HUD, and PHUD, and vehicle domain controllers such as DMS (Driver Monitoring System) and ADAS (Advanced Driver Assistance Systems). Background Technology

[0002] With the rapid iteration of in-vehicle intelligent cockpit and intelligent driving technologies, head-up display products such as AR-HUD (Augmented Reality Head-Up Display), 3D-HUD (3D Head-Up Display), and PHUD (Photo-HUD (Photo-based Head-Up Display, or Projection HUD)) have become standard equipment in passenger vehicles. Currently, the industry faces significant technical challenges: different OEMs define inconsistent data acquisition interfaces and communication protocols for HUD software; and the communication protocols used by controllers in different domains (cockpit, intelligent driving, and body) vary greatly, including CAN-FD (Controller Area Network Flexible Data-Rate), SOME / IP (Scalable service-oriented middleware over IP), IIC, FDBUS, and many others. Software developers need to perform customized development for each type of protocol and each OEM's requirements, significantly increasing the workload of software development and lengthening the project development cycle. At the same time, the differences in optics, structure, and hardware platforms among different HUDs further exacerbate the difficulty of cross-hardware and cross-operating system adaptation.

[0003] Existing related patent solutions all have obvious limitations: 1. Application No. 202411508683.1 discloses a software development system, method, electronic device and storage medium based on AR-HUD. This AR-HUD software development system only proposes a data acquisition approach for multi-domain controllers, but does not provide a unified processing solution for multiple communication protocols, and cannot solve the problem of protocol fragmentation and adaptation. 2. Application No. 202210347470.X discloses a vehicle and its head-up display system, which deploys the ARHUD program to the domain controller, saving the cost of independent hardware, but does not consider the problems of insufficient computing power of the domain controller causing HUD rendering, vehicle domain program lag, and insufficient real-time performance. 3. Patent No. 202210123223.1 discloses a method, system and intelligent vehicle for implementing SOA in intelligent cockpit. It proposes an SOA communication scheme for intelligent cockpit, which only supports interoperability between multiple systems within a single hardware, and is only compatible with SOME / IP and DDS protocols. It cannot be adapted to underlying hardware communication buses such as IIC and SPI. 4. Application No. 202111308428.9 discloses a communication architecture and fusion method for the body domain and cockpit domain based on SOA, which realizes cross-hardware communication between the body domain and cockpit domain based on SOMEIP. However, the communication protocol is singular and cannot be compatible with other vehicle buses such as CAN-FD, IIC, and FDBUS, resulting in poor versatility.

[0004] In summary, existing technologies lack a universal HUD communication architecture that is compatible with multiple vehicle communication protocols, cross-operating systems, cross-hardware platforms, and has low latency. This makes it impossible to achieve one-time development for multi-platform vehicle adaptation, resulting in high industry development costs and low iteration efficiency. Summary of the Invention

[0005] This invention provides a general communication architecture and implementation method for HUD products to solve the technical problems of existing HUD communication solutions, such as fragmented communication protocols, difficulty in cross-platform adaptation, long development cycle, high data interaction latency, and poor scalability. Through a layered decoupled architecture design, it is compatible with all mainstream vehicle communication protocols such as CAN-FD, SOME / IP, IIC, and FDBUS, and realizes low-latency data interaction between HUD and vehicle systems such as DMS and ADAS. It achieves the technical goal of adapting to multiple vehicle models, multiple hardware, and multiple operating systems with a single development, thereby reducing the overall vehicle supporting R&D cost.

[0006] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is: a general communication architecture for HUD products, which adopts a layered and decoupled design, consisting of independent communication processes and business processes. The communication process and the business process complete cross-process data interaction through ZeroMQ (Zero Message Queue, a high-performance, low-latency asynchronous message queue library) middleware; the layered structure from top to bottom consists of a business processing layer, a ZeroMQ communication layer, a data serialization layer, a communication management layer, and a protocol abstraction layer; The protocol abstraction layer defines a unified communication adapter interface ICommAdapter. CAN-FD, SOME / IP, IIC, FDBUS, and custom external communication protocols each create independent communication subclasses. All subclasses inherit from ICommAdapter, and each subclass independently implements the protocol initialization, communication start / stop, message sending, and data receiving callback logic. The communication management layer is configured with CommManager, which is used to uniformly register communication adapters, start all communication adapters that inherit from ICommAdapter, and perform unified message distribution processing on external bus input data. The data serialization layer defines a unified serialization interface IProtocol, which uses the Protobuf (ProtocolBuffers, a serialized data format developed by Google for efficient transmission of structured data) serialization format to uniformly encode and decode structured vehicle data such as DMS, ADAS, and navigation, and converts heterogeneous raw data from multiple protocols into standard Protobuf message structures. The ZeroMQ communication layer adopts a publish-subscribe PUB / SUB asynchronous message mode and serves as a data transmission middleware between the communication management layer and the business processing layer, compatible with Linux, QNX, and Android automotive operating systems. The architecture has a built-in event-driven mechanism and adopts an asynchronous callback strategy. Each communication adapter registers a dedicated data callback function, and the callback processing is triggered directly when external data arrives, thus abandoning polling reading. The independent communication process runs its own dedicated process, isolated from the business process that deploys the Kanzi rendering program, to prevent communication services from competing for HUD rendering computing power.

[0007] The ICommAdapter unified interface includes a standardized function set: protocol initialization function, communication start function, communication stop function, message sending function, and data receiving callback function; different bus protocols only require rewriting the corresponding communication processing logic, and upper-layer services do not need to be aware of the differences in the underlying communication protocols.

[0008] The Protobuf serialization compresses and encodes in-vehicle structured data. Under the same dataset, the Protobuf encoded message size is reduced by 34% compared to the original format, and the transmission time is reduced by 21%.

[0009] A general communication architecture implementation method for HUD products includes the following steps: Step S1. Create CommManager, which serves as the unified management entry point for all communication adapters; Step S2. Based on the bus type of the external vehicle device, create a corresponding communication protocol adapter subclass, with all subclasses inheriting the unified ICommAdapter interface; Step S3. Register the communication adapter with CommManager, so that CommManager can uniformly listen to and distribute external bus data; Step S4. External DMS / ADAS / navigation devices transmit raw data to the subclass derived from the protocol abstraction layer. Different external communication protocols have a unique derived class to handle message sending and receiving separately. The adapter triggers an asynchronous callback function to receive the raw data. Step S5. The raw data is adapted into the corresponding Protobuf data structure according to different communication protocols, and serialized by Protobuf to be encapsulated into serialized data; Step S6. The independent communication process sends the serialized message to the business process through the ZeroMQ publisher; Step S7. The business process receives messages through the ZeroMQ subscriber, deserializes them using Protobuf, and then distributes them to the Kanzi rendering business layer to complete the HUD screen rendering. Step S8. The interactive commands generated by the business layer are sent back to the external vehicle equipment via ZeroMQ, data serialization layer, and protocol abstraction layer.

[0010] In step S4, the entire data reception process adopts event-driven asynchronous callback, without loop polling read operations, thus reducing CPU resource consumption.

[0011] In step S6, ZeroMQ adopts the PUB / SUB publish / subscribe mode, with the communication process acting as the publisher and the business processing process acting as the subscriber. The two establish a dedicated communication link through independent ports.

[0012] The present invention provides a general communication architecture and implementation method for HUD products designed using the above-mentioned technical solution. Compared with the prior art, its advantages are as follows: 1. High versatility, achieving unified adaptation of multiple protocols and platforms: Through the protocol abstraction layer ICommAdapter, all mainstream vehicle buses are uniformly encapsulated, shielding the differences of underlying CAN-FD, IIC, SOME / IP, and FDBUS protocols. One set of code can be adapted to different OEMs and different domain controller hardware without modification, truly realizing one-time development for multiple vehicle models and multiple hardware adaptation, solving the pain point of fragmented communication in the industry. 2. Low latency and high transmission efficiency: Event-driven asynchronous callbacks are used to eliminate polling overhead. Combined with Protobuf binary serialization to compress data volume and ZeroMQ high-performance asynchronous message middleware, the measured end-to-end transmission latency of batch DMS data is only 12ms, which meets the stringent timing requirements of AR-HUD real-time screen rendering. 3. High system compatibility: The architecture conforms to the development trend of automotive SOA software architecture, the communication logic and HUD rendering business are completely decoupled, the native C++ development is compatible with Linux, QNX and Android full automotive operating systems, and seamlessly connects to multiple data inputs such as DMS, ADAS and vehicle navigation; 4. High system stability: The communication module runs as an independent process in isolation, which will not preempt the computing power of Kanzi rendering and algorithm processing, thus avoiding HUD lag and vehicle program freezing due to insufficient computing power of the domain controller; 5. Outstanding commercial value: Significantly reduces the workload of customized development for automakers and HUD suppliers, reduces cross-domain integration and debugging costs, and shortens the delivery cycle of vehicle projects; it is compatible with AR / 3D-HUD products from mainstream HUD manufacturers such as Zejing, Huayang, and Crystal Optoelectronics, forming a differentiated product competitiveness; manufacturers can focus on core innovations in HUD optics and display algorithms, reducing the risk of multi-protocol integration and development. Attached Figure Description

[0013] Figure 1 This is a schematic diagram of the overall architecture of the present invention, which supports multiple communication protocols and multiple platforms. Figure 2 This is a block diagram of the communication layered architecture of the present invention; Figure 3 This is a schematic diagram of the registration process for the DMS communication adapter of the present invention. Figure 4 This is a schematic diagram of the simulated DMS eye point coordinate data output log of the present invention; Figure 5 This is a schematic diagram of the batch data serialization processing log of the main communication thread of the present invention; Figure 6 This is a schematic diagram of the timestamp recording at the data sending end of the present invention; Figure 7 This is a schematic diagram of the timestamp recording at the data receiving end of the business layer of the present invention; Figure 8 This is a schematic diagram of the low-latency transmission performance test embodiment of the present invention; Figure 9 This is a schematic diagram of the ZeroMQ publish-subscribe communication implementation process of this invention. Detailed Implementation

[0014] The following description, in conjunction with the accompanying drawings, details a general communication architecture and implementation method for HUD products according to the present invention.

[0015] Embodiment 1 of the present invention: DMS communication adapter registration and adaptation process like Figure 2As shown in the layered architecture, the communication architecture of this invention consists of a five-layered structure from top to bottom: business processing layer T006, ZeroMQ communication layer T005, data serialization layer T004, communication management layer T003, and protocol abstraction layer T002.

[0016] The protocol abstraction layer T002 defines the basic message sending and receiving processing for all communication protocols from external devices. Different protocol types allow for the development of custom protocol code. Specifically, it defines a unified ICommAdapter interface, which provides four basic functions: initialization, start / stop, sending, and callback. Independent communication classes are created for CAN-FD, SOME / IP, IIC, FDBUS, and custom external protocols, all inheriting from ICommAdapter, with only the underlying hardware interaction logic of the corresponding bus being overridden.

[0017] The communication management layer T003 has a built-in CommManager communication manager, which is responsible for registering all communication adapters, starting all communication adapters that inherit from ICommAdapter, and performing unified message distribution processing on external bus input data.

[0018] The data serialization layer T004 focuses on converting data from different communication protocols into protobuf data. It then serializes the converted protobuf data and populates it into a custom Message structure. Specifically, based on the IProtocol interface, it uses Protobuf to perform unified binary encoding and decoding of raw data from multiple protocols, standardizing heterogeneous bus data into a unified message structure.

[0019] ZeroMQ communication layer T005 acts as middleware for data interaction between the communication layer and the business processing layer T006, sending serialized data from the communication layer to the business processing layer T006 process.

[0020] This architecture enables efficient, low-latency data interaction between the HUD system and in-vehicle systems such as Driver Monitoring System (DMS) and ADAS through a protocol abstraction layer, event-driven mechanism, and process isolation architecture, solving the problems of low scalability, cross-platform difficulty, and insufficient real-time performance of traditional HUD communication architectures.

[0021] like Figure 3 As shown, the complete communication adaptation process for DMS devices is as follows: T009: Create a global CommManager instance to uniformly manage all external communication adapters and manage daily functions such as registration and message distribution for all communication protocols.

[0022] T008: Create a Driver Monitoring System (DMS) communicator. This communicator processes the acquired DMS data and can be adapted to different DMS communication modes and data structures. Specifically, based on the vehicle's DMS hardware communication bus type, create a corresponding DMS communication adapter instance. This adapter inherits from ICommAdapter and overrides the corresponding bus send / receive and callback logic.

[0023] T007: Register the Driver Monitoring System (DMS) communicator through the communication manager. This allows all data from the DMS to be perceived and processed by the algorithm module. Specifically, register the DMS communication adapter to the CommManager. After registration, all eye-point 3D coordinate data uploaded by the DMS will be captured and distributed uniformly by the CommManager. The upper-layer HUD algorithm and rendering module can directly perceive the data without modifying the underlying bus code.

[0024] Embodiment 2 of the present invention: Low-latency data transmission performance test This embodiment verifies the end-to-end data transmission latency performance of this architecture. To ensure low latency during data processing and transmission, asynchronous processing and efficient serialization methods are employed in the design. To facilitate recording and testing the time consumed from receiving data from the Driver Monitoring System (DMS) to its reception and normalization by the business end, 20 sets of data (starting point of time recording) are simulated from the DMS, and then uniformly sent to the business end via ZeroMQ (ending point of time recording). The test object is the DMS eye-point 3D coordinate data, and the test steps are as follows: Step 1. Read the simulated eye point coordinate values ​​from the generated Driver Monitoring System (DMS) JSON file, periodically generate sinusoidal variations in the three-dimensional X / Y / Z coordinates of the human eye, generating 20 sets of DMS data in a single batch. The data format is as follows: Figure 4 As shown in the log; Step 2. The independent communication process asynchronously receives data through the registered DMS adapter and triggers a callback function to complete the parsing of the raw data; Step 3. Data is sent to the serialization layer, and the Protobuf serialization interface is called to complete binary encoding, generating a fixed-length message buffer. The log is as follows: Figure 5 As shown, the buffer size is 300 bytes after serializing 20 sets of data at a time. Step 4. Add timestamps to the Driver Monitoring System (DMS) data acquisition section and the business layer's DMS data acquisition section to calculate the time interval from acquisition to receipt by the business layer for these 20 sets of data. The ZeroMQ publisher records the sending start timestamp and sends the serialized message to the business process via PUB mode; Step 5. The business process starts an independent receiving thread to receive data through the ZeroMQ SUB subscription client, records the receiving completion timestamp, performs Protobuf deserialization on the data, and pushes it to the Kanzi rendering layer for processing. Here, a separate thread needs to be created to process the received data, that is, to deserialize the data. Depending on the business characteristics, the deserialization function needs to be divided into different functions, with different functions handled in detail according to business needs. Step 6. Calculate the difference between the timestamps at both ends. The sending start time is 82,684,471 ms, the service reception completion time is 82,684,483 ms, and the overall transmission time for a single batch of 20 data groups is 12 ms.

[0025] The test environment is a PC-based simulated vehicle platform. The data is input offline using JSON simulation. In the actual vehicle hardware environment, the data transmission latency of this architecture is no higher than 12ms, which is far lower than the 100ms real-time hard target of the vehicle HUD industry. This ensures that the AR-HUD virtual image is synchronously superimposed on the real road scene without any lag.

[0026] Embodiment 3 of the present invention: Demonstrating how to create inter-process ZeroMQ communication to simulate driver monitoring system (DMS) data transmission. This invention selects ZeroMQ as the inter-process communication middleware. Its core advantages are lightweight and low resource consumption, cross-operating system compatibility, and high-throughput asynchronous transmission. It employs a publish-subscribe (PUB / SUB) model for data sending and receiving. The publisher (PUB) can be a device of any architecture, and the subscriber (SUB) can also be a device of any architecture. The PUB / SUB publish-subscribe model is used to complete the data interaction between the communication process and the business process. The implementation process is as follows: Figure 9 : 1. Create a ZeroMQ communication class and define the publisher's local server interface, binding it to the local TCP port 5556. The code is as follows: ZmqCommunicate communicate(ZmqCommunicate::Mode::PUBLISHER, "tcp: / / *:5556"); / / Use the local address as the server, using a different port. 2. Referring to Example 2, after serializing the Driver Monitoring System (DMS) data, the serialized data is sent out via ZeroMQ.

[0027] bool flag = communicate.send(buffer); 3. The business process creates a ZeroMQ subscriber instance, defines the subscriber interface, and listens on local port 5556: std::make_shared <zmqcommunicate>(ZmqCommunicate::Mode::SUBSCRIBER, "tcp: / / localhost:5556") 4. The subscriber reads the message buffer in a loop, completes deserialization, and delivers the message to the business rendering module: bool success = m_ComReceiver->receive(buffer) std::vector <std::shared_ptr <signaldata>>originData=m_ComReceiver->deserializeOriginData(buffer); / / Deserialize data ZmqCommunicate is a self-developed dedicated communication class based on ZeroMQ, used to unify the inter-process data transmission interface. ZmqCommunicate encapsulates the underlying ZeroMQ API, providing a unified send / receive interface while shielding the underlying TCP and IPC transmission details, enabling zero-adaptation development across hardware and operating systems.

[0028] Overall architecture, development language, and cross-platform features of this invention The entire communication architecture is developed using native C++ with no operating system dependency. ZeroMQ natively supports the three major automotive operating systems: Linux, QNX, and Android. The protocol abstraction layer completely isolates hardware bus differences, and the same set of engineering code can be directly adapted to all types of head-up display devices, including AR-HUD, 3D-HUD, and PHUD, and can connect to cockpit domain and intelligent driving domain controllers from different OEMs.< / signaldata> < / zmqcommunicate>

Claims

1. A general communication architecture for HUD products, characterized by: It adopts a layered and decoupled design, consisting of independent communication processes and business processes. The communication process and business process complete cross-process data interaction through ZeroMQ middleware. The layered structure from top to bottom is as follows: business processing layer, ZeroMQ communication layer, data serialization layer, communication management layer, and protocol abstraction layer. The protocol abstraction layer defines a unified communication adapter interface ICommAdapter. CAN-FD, SOME / IP, IIC, FDBUS, and custom external communication protocols each create independent communication subclasses. All subclasses inherit from ICommAdapter, and each subclass independently implements the protocol initialization, communication start / stop, message sending, and data receiving callback logic. The communication management layer is configured with CommManager, which is used to uniformly register communication adapters, start all communication adapters that inherit from ICommAdapter, and perform unified message distribution processing on external bus input data. The data serialization layer defines a unified serialization interface IProtocol, which uses the Protobuf serialization format to uniformly encode and decode DMS, ADAS, and navigation structured vehicle data, and converts multi-protocol heterogeneous raw data into standard Protobuf message structures. The ZeroMQ communication layer adopts a publish-subscribe PUB / SUB asynchronous message mode and serves as a data transmission middleware between the communication management layer and the business processing layer, compatible with Linux, QNX, and Android automotive operating systems. The architecture has a built-in event-driven mechanism and adopts an asynchronous callback strategy. Each communication adapter registers a dedicated data callback function, and the callback processing is triggered directly when external data arrives, thus abandoning polling reading. The independent communication process runs its own dedicated process, isolated from the business process that deploys the Kanzi rendering program, to prevent communication services from competing for HUD rendering computing power.

2. The general communication architecture for HUD products according to claim 1, characterized in that: The ICommAdapter unified interface includes a standardized function set: protocol initialization function, communication start function, communication stop function, message sending function, and data receiving callback function; different bus protocols only require rewriting the corresponding communication processing logic, and upper-layer services do not need to be aware of the differences in the underlying communication protocols.

3. The general communication architecture for HUD products according to claim 1, characterized in that: The Protobuf serialization compresses and encodes in-vehicle structured data. Under the same dataset, the Protobuf encoded message size is reduced by 34% compared to the original format, and the transmission time is reduced by 21%.

4. A method for implementing a general communication architecture for HUD products, applied to the general communication architecture described in any one of claims 1-3, comprising the following steps: Step S1. Create CommManager, which serves as the unified management entry point for all communication adapters; Step S2. Based on the bus type of the external vehicle device, create a corresponding communication protocol adapter subclass, with all subclasses inheriting the unified ICommAdapter interface; Step S3. Register the communication adapter with CommManager, so that CommManager can uniformly listen to and distribute external bus data; Step S4. External DMS / ADAS / navigation devices transmit raw data to a subclass derived from the protocol abstraction layer. Different external communication protocols have a unique derived class that handles message sending and receiving separately. The adapter triggers an asynchronous callback function to receive the raw data. Step S5. The raw data is adapted into the corresponding Protobuf data structure according to different communication protocols, and serialized by Protobuf to be encapsulated into serialized data; Step S6. The independent communication process sends the serialized message to the business process through the ZeroMQ publisher; Step S7. The business process receives messages through the ZeroMQ subscriber, deserializes them using Protobuf, and then distributes them to the Kanzi rendering business layer to complete the HUD screen rendering. Step S8. The interactive commands generated by the business layer are sent back to the external vehicle equipment via ZeroMQ, data serialization layer, and protocol abstraction layer.

5. The method for implementing a general communication architecture for HUD products according to claim 4, characterized in that: In step S4, the entire data reception process adopts event-driven asynchronous callback, without loop polling read operations, thus reducing CPU resource consumption.

6. A method for implementing a general communication architecture for HUD products according to claim 4, characterized in that... In step S6, ZeroMQ adopts the PUB / SUB publish / subscribe model, with the communication process acting as the publisher and the business processing process acting as the subscriber. The two establish a dedicated communication link through independent ports.

Images