A real-time distributed system for intelligent driving

Abstract

The present invention discloses a real-time distributed system for intelligent driving, comprising a perception layer, a decision layer and a control layer. The perception layer comprises a collection unit, which encapsulates external information collected by the collection unit into a DataWriter and then writes it into a predefined Topic. Subscribers subscribe to the Topic and obtain external messages from it. Publishers publish the external messages to the decision layer. The decision layer derives and publishes decision instructions based on the subscribed external messages and external input instructions. The control layer subscribes to the decision instructions, generates and publishes corresponding control instructions. The car adjusts its throttle, steering wheel, brake and gear according to the control instructions. The advantage of the present invention is that it meets the real-time requirements of the intelligent driving system.

Description

Technical Field

[0001] The present invention relates to the field of intelligent driving system architecture, and more specifically to a real-time distributed system for intelligent driving. Background Art

[0002] The real-time distributed architecture addresses challenges in handling high-volume traffic and protecting critical business operations. Using clustering technology, it distributes the load of large-scale concurrent requests across different devices. This improves the availability of backend services, and the distributed architecture promptly isolates faults to prevent them from impacting other critical business operations. This architecture also significantly enhances system availability by eliminating redundant or duplicative components, ensuring operational efficiency.

[0003] The intelligent driving system architecture consists of three layers: the perception layer, the decision layer, and the control layer. Each layer performs its own specific tasks, and only through their coordinated efforts can the intelligent driving system operate stably and safely. Currently, sensors, high-precision maps, and cloud computing are key to achieving autonomous driving. Sensors serve as the eyes of autonomous driving, observing changes in the surrounding environment. High-precision maps provide a comprehensive field of view for autonomous driving and can predict road conditions beyond its detection range. Cloud computing ensures the timely upload of sensor data, ensuring that high-precision maps are always up to date and distributed to vehicles. Cloud computing is the primary link for collecting and analyzing road data and training and updating decision models. The real-time distributed architecture offers high performance and low latency. When the intelligent driving system receives a command, it processes it at high speed, minimizing delays between the command and the system. Key technologies involved in intelligent driving include environmental perception, navigation and positioning, path planning, decision-making and control, and automatic control. The system must be able to perceive the safety of the surrounding environment and store it in high-precision maps. It also requires an understanding of road conditions and the ability to respond promptly to unexpected road conditions.

[0004] To ensure timely awareness of road conditions for intelligent driving vehicles, distributed systems require that changes and information on any node be promptly detected and accessed by other relevant nodes. Therefore, achieving real-time, on-demand information distribution within the system is a key technology and a key focus for researchers. Common technologies involved include CORBA (Common Object Request Broker Architecture), which is object- and service-centric and utilizes a client-server communication model. This complex communication mechanism requires establishing a connection for data transmission and reception, which cannot fully meet the real-time performance requirements of intelligent driving systems. Furthermore, it lacks QoS support for service policies and lacks the flexibility required for communication. The Java Message Service (JMS) includes both point-to-point and publish / subscribe messaging modes, providing reliable message transmission, transactions, and message filtering. However, it lacks application-level QoS policies and remains unsuitable for systems with high real-time requirements.

[0005] Chinese patent publication number CN112348201A discloses a method for implementing intelligent decision-making for a swarm of autonomous vehicles based on federated deep reinforcement learning. The method comprises: using deep reinforcement learning to observe the driving environment of the swarm and surrounding vehicles, generating training data within the processors of each vehicle, encrypting the trained data model and sending it to an edge computing server, which then trains an edge neural network and simultaneously sends the trained model to a cloud computing server. The cloud computing server then integrates the training data for each model and sends the trained model information to the edge computing server and vehicle terminals to update the model information on the vehicle terminals. This method utilizes a federated deep learning approach to implement intelligent decision-making for autonomous vehicles. This patent application helps mitigate the impact of big data communication conditions, resolves the imbalance of training data, and facilitates more intelligent allocation of computing loads and resources. It also enables powerful environmental awareness capabilities for swarm decision-making in autonomous vehicles. However, the model structure of this patent application is complex, and the computational process is time-consuming, which still cannot meet the real-time requirements of intelligent driving systems. Summary of the Invention

[0006] The technical problem to be solved by the present invention is that the existing control methods for intelligent driving cannot meet the real-time requirements of the intelligent driving system.

[0007] The present invention solves the above technical problems through the following technical means: a real-time distributed system for intelligent driving, including a perception layer, a decision layer and a control layer. The perception layer includes a collection unit. The external information collected by the collection unit is encapsulated into a DataWriter and then written into a predefined Topic. The subscriber subscribes to the Topic and obtains external messages from it. The publisher publishes the external message to the decision layer. The decision layer derives and publishes decision instructions based on the subscribed external messages and external input instructions. The control layer subscribes to the decision instructions to generate corresponding control instructions and publish them. The car adjusts its throttle, steering wheel, brake and gear according to the control instructions.

[0008] The perception layer, decision layer and control layer of the present invention adopt a publish / subscribe model centered on real-time data. The publish / subscribe model meets the real-time and high-speed exchange of information in complex systems. In a distributed network environment, it can dynamically and flexibly perform networking and on-demand multi-point distribution of data, solving the problem of efficient and real-time data distribution, and is very suitable for the use scenarios of intelligent driving systems.

[0009] Furthermore, the perception layer issues external messages to the decision layer, the decision layer issues decision instructions to the outside, and the control layer issues control instructions, all of which are achieved through a multi-threaded worker data distribution method.

[0010] Furthermore, the multi-threaded worker data distribution method includes:

[0011] The publisher adds a timestamp and corresponding index to the message data and sends the message to the polling pool. Multiple worker partitions receive messages from the polling pool, obtain the timestamp and index after unpacking, and then calculate the delay and determine whether there is packet loss. The delay and packet loss information are then encapsulated together with the message data and sent to the subscriber collector. The subscriber collector writes the delay and packet loss information into the log, counts the total number of received messages, and determines whether to continue receiving. If the delay and packet loss of the total received messages exceed the preset threshold, a termination signal is sent. The corresponding worker stops receiving messages after receiving the termination signal.

[0012] Furthermore, the message data is partitioned according to preset index segments, each worker sets a corresponding index segment, and each worker receives the message data of the corresponding index segment. When all indexes of the message data received by the worker are missing compared with the index corresponding to its index segment, it indicates that packet loss occurs.

[0013] Furthermore, the publisher and subscriber pre-set QoS (Quality of Service) templates, and the publisher and subscriber exchange templates with each other. Only when the service quality defined by the subscriber's QoS template is higher than the service quality defined by the publisher's QoS template, the publisher and subscriber can be successfully matched. The subscriber can then receive messages sent by the publisher.

[0014] Furthermore, the worker sets a corresponding deadline for receiving each message data. If the message data is not received within the deadline, it means that the data is delayed. The delay is obtained by subtracting the timestamp from the deadline for receiving the message data.

[0015] Furthermore, when the worker is no longer needed to receive messages, the automatic garbage collection mechanism provided by the CPython interpreter is used to release the worker resources and reclaim the memory space occupied by the worker.

[0016] Furthermore, in the absence of external input instructions, the starting address and destination address are input from the high-precision map, the path is planned through the path planning algorithm, and the planned path is input to the decision layer and the control layer. The decision layer derives decision instructions based on the planned path and issues them to the control layer, and the control layer generates corresponding control instructions based on the decision instructions.

[0017] Furthermore, the acquisition unit includes a camera, a light radar, a GPS and an inertial sensor.

[0018] Furthermore, the decision instructions include following a vehicle, overtaking, changing lanes, braking, and turning.

[0019] Furthermore, the control instructions generated by the control layer include throttle size, steering wheel angle, braking force, and forward and reverse movement.

[0020] The advantages of the present invention are:

[0021] (1) The perception layer, decision layer, and control layer of the present invention adopt a publish / subscribe model centered on real-time data. The publish / subscribe model satisfies the real-time and high-speed exchange of information in complex systems. In a distributed network environment, it can dynamically and flexibly perform networking and on-demand multi-point distribution of data, solving the problem of efficient and real-time data distribution, and is very suitable for the use scenarios of intelligent driving systems.

[0022] (2) The present invention adopts a multi-threaded worker data distribution method to realize data distribution. The polling pool serves as the worker status polling interface, which is responsible for orderly managing multiple buffer partitions. Multiple worker partitions receive messages from the polling pool to prevent the same worker from processing too many messages, thereby improving the concurrency of the publish / subscribe model. In addition, based on the traditional subscriber, a subscriber collector for message aggregation is designed and implemented. Different workers asynchronously aggregate the message results processed by each worker, thereby further improving the subscriber's message receiving capability.

[0023] (3) The multi-threaded worker data distribution method of the present invention receives messages through multi-threaded partitions, effectively solving the packet loss problem caused by the subscriber's inability to receive messages as fast as the publisher's sending messages.

[0024] (4) In the traditional publish-subscribe model, publishers and subscribers can only communicate in one direction, and subscribers are invisible to publishers, causing slow connection problems. The multi-threaded worker data distribution method of the present invention realizes two-way communication between publishers and subscribers by setting up a two-way channel. When delays and packet loss occur, it determines whether to continue receiving. When the delay and packet loss exceed the preset threshold, a termination signal is sent. The corresponding worker stops receiving messages after receiving the termination signal. Based on the two-way communication, the status synchronization between the publisher and the subscriber is achieved, and the publisher process and the subscriber process are controlled to execute synchronously, thereby preventing the packet loss caused by the slow connection problem. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 This is a DCPS model architecture diagram in a real-time distributed system for intelligent driving disclosed in an embodiment of the present invention;

[0026] Figure 2 This is an architecture diagram of a real-time distributed system for intelligent driving disclosed in an embodiment of the present invention;

[0027] Figure 3 A finite state machine for behavior planning in a real-time distributed system for intelligent driving disclosed in an embodiment of the present invention;

[0028] Figure 4 This is a workflow diagram of each layer in a real-time distributed system for intelligent driving disclosed in an embodiment of the present invention;

[0029] Figure 5 This is a flowchart of controller operations at the control layer in a real-time distributed system for intelligent driving disclosed in an embodiment of the present invention;

[0030] Figure 6 A schematic diagram of a SUB process in a real-time distributed system for intelligent driving disclosed in an embodiment of the present invention;

[0031] Figure 7 This is a schematic diagram of a slow connection problem in a real-time distributed system for intelligent driving disclosed in an embodiment of the present invention;

[0032] Figure 8 This is a schematic diagram of the multi-threaded worker data distribution process in a real-time distributed system for intelligent driving disclosed in an embodiment of the present invention. DETAILED DESCRIPTION

[0033] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0034] This invention primarily focuses on a novel real-time distributed architecture suitable for intelligent driving environments. The core concepts of this distributed architecture are parallel splitting and horizontal expansion. Parallel mechanisms are typically used in read operations with high real-time requirements. Its basic working process involves parallel calls to all nodes in a service cluster. Since these are read operations, all service nodes return identical data. Therefore, as long as one service node successfully returns a successful call, a response is returned to the client. During horizontal expansion, nodes are expanded horizontally, for example, from one node to multiple nodes. Each node runs an independent instance, and the nodes are interconnected via a network. As nodes expand, system processing power increases, and even if a single node fails, the entire cluster can still provide external services. Intelligent driving systems combine different modular components, which must be integrated into a reliable system to ensure the safety of intelligent driving and enable real-time analysis and accurate response to complex environments. DDS, as a data distribution service, is used in the publish-subscribe (DCPS) middleware model for distributed application communication and integration. DDS's most important feature is its data-centric nature, which distinguishes it from many other communication middleware platforms. Messages are passed between multiple nodes in DDS via shared memory. The following sections describe the solutions of this invention.

[0035] 1. Extract the car's communication information into multiple topics

[0036] The DDS architecture consists of the DLRL layer (Data Local Reconstruction) and the DCPS layer (Subscribe-Publish Model). The DLRL layer is built on the underlying DCPS layer. Through the services provided by the DCPS, it simplifies programming implementation and integrates data obtained by the intelligent driving perception layer, achieving seamless integration with local language structures. DCPS (Data-Centric Publish-Subscribe) is a data-centric subscription-publish model defined in the DDS standard. In this model, the parties that write data to the global data space are called Publishers and DataWriters. Similarly, the parties that read data from the global data space are called Subscribers and DataReaders. DDS data sharing is based on topics, allowing applications to determine the type of data contained in a topic without relying on other contextual information. For example, the lidar in an intelligent driving system captures a frame of data, which represents a topic. DDS can then send this frame of data to subscribers who have subscribed to the topic for processing. Furthermore, DDS can automatically store, publish, or subscribe to data according to user-defined methods, allowing applications to write or read data as if they were accessing local data. When the system is running, data is transmitted or stored on demand. The publisher of data only sends the data needed by the other party, and the subscriber only receives and stores the data currently needed by the local application. For intelligent driving, the vehicle driving state sensor and road condition sensor are the senders of data, which send the vehicle driving state and road condition to the state perception controller, the receiver. By understanding the vehicle driving state and road condition, a series of control decisions are made for the intelligent vehicle. The DCPS model in intelligent driving is as follows: Figure 1 shown.

[0037] The fundamental structure in DDS is the domain, which binds applications together for communication. Any two entities in DDS must interact within the same domain. This means they share the same DomainID upon initialization, and DomainIDs must be unique across domains. A DomainParticipant within a domain is the entry point for services. Any DDS application must first obtain a DomainParticipant, which then accesses other services such as Publishers, Subscribers, and Topics. DDS is based on a publish / subscribe model based on topics. A topic is a control signal for a car, such as a road obstacle. Surrounding cars are also topics. In DDS, a participant is registered, a publisher is generated, and a dataWriter is registered for a specific topic. This completes the publishing process. Similarly, the receiving process requires registering a participant, generating a subscriber, and then registering a dataReader for a specific topic.

[0038] To meet the real-time requirements of automotive control and ensure the real-time nature of publish-subscribe transactions, the real-time distributed architecture implements QoS policies within DDS. Each policy defines a structure described by IDL (Interface Description Language). IDL stands for Interface Description Language, which enables communication between objects running on different platforms and programs written in different languages. QoS policies act on publishers and subscribers, filtering data to meet requirements. Subscriptions can specify time and content filters to retrieve only a subset of the data published on a topic.

[0039] The DCPS layer translates user resource demands and resource availability into Quality of Service (QoS). QoS includes multiple policy forms, each of which describes service behavior by associating it with a valued name. In DDS, QoS parameters are used throughout the entire DDS communication process. Application developers simply need to specify the desired QoS and configure it to program server behavior.

[0040] The intelligent driving system architecture mainly includes the perception layer, decision layer and control layer. The perception layer is mainly realized through various sensors and high-precision maps, including vehicle positioning and object recognition. Perception refers to the ability of the intelligent driving system to collect information from the environment and extract relevant knowledge from it. The intelligent driving system collects relevant information from the environment and transmits it to the sensors in the hardware facilities. The sensors receive relevant data and transmit it to the perception layer. The perception layer here includes environmental perception and positioning functions. Understanding and grasping the environment is a very important step in the intelligent driving process. The environmental perception part of the intelligent driving system needs to obtain a large amount of surrounding information and extract the following information of the car into multiple topics, such as Figure 2 As shown, topics include obstacle location, current vehicle speed, possible behavior, drivable areas, and traffic regulations. Smart cars typically obtain this information by fusing data from multiple sensors, such as lidar, cameras, and millimeter-wave radar. After acquiring this information, they determine the data type based on the corresponding topic in the global data space and then transmit the information to subscribers. The perception layer predefines the information input by the vehicle's sensors into several topics. The decision layer receives topics from the perception layer. Specific subscribers subscribe only to specific topics, ensuring that specific sensor information is correctly received. The control layer receives information published by the decision layer, and subscribers at the control layer also receive only specific topics.

[0041] To ensure normal communication between nodes in an intelligent driving system, we must consider the real-time and reliability requirements of data transmission, as well as how to manage multiple publishers of the same data at the same time. If an existing publisher is abnormal, how to recover from the failure to ensure normal operation of the system.

[0042] To meet the real-time requirements of automotive control and ensure the real-time nature of publish-subscribe transactions, the real-time distributed architecture implements QoS policies within DDS. Each policy defines a structure described in IDL. QoS policies apply to publishers and subscribers, filtering data to ensure it meets the requirements. Subscriptions can specify time and content filters, retrieving only a subset of the data published on a topic.

[0043] Define QoS strategy: LIVELINESS strategy is used for topics, data readers and data writers. Setting LIVELINESS on the topic means that all data readers and data writers of this topic are valid.

[0044] The HISTORY strategy determines how samples are persisted in the data writer and data reader for a particular instance. For data writers, the values ​​are persisted until the publisher retrieves and successfully sends them to all connected subscribers. For data readers, the values ​​are persisted until the application "takes the data away."

[0045] DEADLINE QoS allows applications to detect whether data is written or read within a specified time. The data writer monitors the data changes made by the application. When the data writer detects that the data state has not changed before the deadline, it calls the on_offered_deadline_missed() listener callback. When the data reader detects that the data state has not changed before the deadline, it triggers the on_requested_deadline_missed() listener callback.

[0046] Real-time transmission service is a requirement for sensors when distributing data. This is achieved by optimizing DDS QoS settings on both the publisher and subscriber sides to improve the real-time nature of data distribution. The following QoS policies are used to implement real-time transmission service: deadline, latency_budget, reliability, and transport_priority.

[0047] 2. Plan the various behaviors of the smart car when performing tasks

[0048] Vehicle positioning primarily relies on integrating information from lidar, GPS, inertial sensors, high-precision maps, and other sensors to determine the vehicle's precise location. Object recognition is primarily achieved using lidar and binocular cameras. The vehicle needs to know its exact position relative to the environment, and the positioning error cannot exceed 10cm. Currently, the most widely used vehicle positioning method is the fusion of a global positioning system and an inertial navigation system. GPS positioning accuracy ranges from tens of meters to centimeters, making high-precision GPS sensors relatively expensive. The fusion of GPS / IMU positioning methods cannot achieve high-precision positioning when the GPS signal is weak or missing, making it suitable only for certain scenarios. Map-assisted positioning algorithms are required for positioning.

[0049] The decision layer's inputs include information from the perception layer, path planning, and feedback from the control layer. Decision instructions are issued by adding a learning algorithm (the learning algorithm refers to the multi-threaded worker data distribution method described in Section 3). These issued decision instructions act as senders in the DDS architecture. These instructions include following, overtaking, accelerating, braking, decelerating, steering, and U-turns. These instructions are receivers, and vehicle operations are determined based on matching topic types. Based on the instructions issued by the decision layer, the control layer implements specific vehicle controls, including throttle, brake, steering, and gear control.

[0050] The role of behavior planning is to plan the various behaviors of the smart car when performing tasks. Behavior planning is generally planned using a finite state machine. The state machine divides the smart car into several states, each corresponding to a different behavior, such as parking waiting state, normal driving state, out of state state, etc. The state machine also includes the conditions for switching between different states. The finite state machine for driving behavior planning is as follows: Figure 3 shown.

[0051] After the decision-making layer completes its task, it will pass the target behavior to the control layer, which will pass the command to the brake for use in the environment. Longitudinal control is used to control the speed of the smart car. The input is the desired speed, actual speed, road slope, vehicle status and other information. The control amount of the accelerator pedal and brake pedal is calculated and output so that the speed of the smart car can follow the desired speed. Lateral control is used to control the steering of the smart car. The steering angle of the steering wheel is calculated based on the deviation between the vehicle and the desired trajectory, so that the smart car can follow the desired trajectory. Other controls are used to control the gear position, parking brake, lights, horns and other components of the smart car. The above are the main tasks that need to be completed by the perception layer, decision layer and control layer of the intelligent driving system. The workflow diagram of each layer of intelligent driving is as follows. Figure 4 As shown, the sensors and environmental perception and localization services of the hardware environment (Hardware) are in a publish-subscribe relationship, monitoring each other's data. V2V communication is responsible for transmitting data in perception and planning, where planning includes task planning, behavior planning, and motion planning. The actuators and path tracking and trajectory tracking are in a publish-subscribe relationship, monitoring each other's data. As the bottom layer of the intelligent vehicle system, the control layer's task is to implement the planned actions, so the evaluation index of the control layer is the control accuracy. There will be measurements inside the control system, and the controller of the control layer outputs control actions by comparing the vehicle's measurements with the expected state. The controller operation flow chart is as follows Figure 5 As shown, the vehicle controller sends control signals to the transmission control, generator control, motor control, and brake control respectively, thereby respectively realizing engine control, motor control, and brake system control, thereby controlling the entire vehicle's driving.

[0052] In this embodiment, DDS (Data Distribution Service) uses the vehicle's sensors as nodes. When a node needs to send data, it first creates a domain participant (participant, including publishers and subscribers). The participant then creates a topic representing the data type, a publisher, and a data writer, DataWriter.

[0053] Data publishers use Deadline QoS (Quality of Service) to indicate the sensor data update cycle, while data subscribers use Deadline QoS to indicate the expected data update cycle. For road condition sensors, Deadline QoS corresponds to the data refresh rate specified in the publish and subscribe conditions. Deadline parameters must be passed to the image topic, the camera's Datawriter, and the road condition sensor's Datareader. During registration matching in the registration server, if the camera's publish condition's topic_qos.deadline (Quality of Service deadline) is less than the road condition sensor's subscription condition's topic_qos.deadline, the publisher's camera deadline meets the subscriber's deadline. The camera publishes its detected image data according to the dw_qos.deadline (Quality of Service deadline for the data writer). The road condition sensor verifies the validity of the received data based on the Datareader's dr_qos.deadline (Quality of Service deadline for the data reader). If the time interval between arrivals of the camera's image data exceeds the deadline, the data packet is deemed invalid and discarded. When the deadline policy is set to a finite value, the image data writer fed back from the traffic sensor monitors data changes made by the application, sets corresponding status conditions, and when the trigger conditions are met, calls back the on_offered_deadline_missed() listener (the callback function for when the offer time exceeds the deadline). If the data reader detects that the data status has not changed before the deadline, it triggers the on_requested_deadline_missed() listener callback (the callback function for when the request time exceeds the deadline).

[0054] Latency_budget (latency estimate) is a latency budget, a key step in ensuring system real-time performance. In an intelligent driving system, the vehicle's driving status sensor, on the subscription side, assigns the latency value in the subscription conditions to the Latency_budgetQos (latency estimate quality of service). The Latency_budgetQos is then passed to the acceleration topic's Datareader, setting the real-time performance requirements for the vehicle's driving status sensor data. The inertial sensor, as the publisher, is unaware of the vehicle's driving status sensor's real-time performance requirements and therefore sets this Qos to a higher value to ensure a match with the Datareader. When the Datareader's subscription conditions match the inertial sensor Datawriter's publishing conditions, the registrar returns the subscription conditions to the inertial sensor Datawriter. The Datareader's Latency_budgetQos in the subscription conditions is then assigned to the Datawriter. This approach is based on the changeable nature of the Latency_budget, meaning that the Qos can be updated even after the associated entity is created. When data is transmitted to the vehicle's driving status sensor, the Datareader compares the data latency with its Latency_budgetQos. If the latency is less than its Latency_budget, it meets the real-time requirements and is uploaded to the perception layer of the smart driving vehicle. The perception layer then passes the data to the decision layer of the system. As the publisher, the decision layer also needs to determine the type of data to be transmitted based on the vehicle's driving status. After receiving the relevant data from the sensor, the decision layer will make a series of decisions based on the road conditions, such as slowing down or discarding the data.

[0055] Latency_budget works in conjunction with transport_priority. A smaller latency_budget maps to a higher transport_priority. Transport_priority prioritizes data sent by Datawriters, improving real-time performance by prioritizing access to communication resources. For Datawriters with a smaller dw_qos.latency_budget (the estimated latency of the data writer's quality of service), setting a higher transport_priority prioritizes communication resources, reducing local data latency and shortening the overall distribution process.

[0056] Reliability refers to reliability, but reliability and real-time performance are mutually exclusive. To achieve high real-time performance, reliability must be sacrificed. Therefore, a low-reliability transport protocol is selected to achieve higher real-time performance. The Reliability QoS kind parameter is set to Best Effort to ensure system real-time performance.

[0057] The decision layer, as a subscriber of the perception layer, receives a series of topic data such as images, obstacle locations, geographic coordinates, and acceleration from the perception layer, parses the data, and then transmits the data as parameters to the decision algorithm. The decision algorithm publishes the decision results to the control layer. That is, the decision layer is the publisher and the control layer is the subscriber of the decision layer.

[0058] The control layer needs to understand the current vehicle operation requirements. Subscription services are another core function of the publish / subscribe middleware layer. The subscription service uses the registration service to send data requests to the network, then waits for subscribed data and delivers the subscribed data to upper-layer applications. Similar to publishers, subscribers' data subscriptions are handled by DataReaders. The vehicle's control layer needs to receive data from the decision layer to perform vehicle operations. The control layer uses the DDS standard's subscription with listener model to receive data. For each DataReader, a DataReaderListener is created to monitor the DataReader for incoming data. If data arrives, the DataReaderListener's on_data_available() operation (data arrival callback) is triggered to notify the application of the data arrival. The DataReader's read() operation (data read function) is then called to read the subscribed data from the DataReader cache. The subscribed data at this layer is the decision instruction. For example, if the vehicle needs to brake, the subscriber needs to provide the braking data, including the braking force and duration. In this way, real-time message transmission is achieved in the vehicle's perception layer, decision layer, and control layer through data publishing / subscription.

[0059] 3. Multi-threaded worker data distribution method solves slow subscriber and slow connection problems

[0060] (1) Slow Subscriber Problem

[0061] When a connection is established between the publisher and the subscriber, the main cause of packet loss is that the subscriber's message receiving speed cannot keep up with the publisher's message sending speed. After the subscriber receives the message, the message needs to be processed by the business flow. When the business flow thread has a large time overhead, it will cause the recv thread (receiving thread) in the subscriber process to be blocked. After the blockage ends, it will listen to the publisher's message again. ZMQ (zero message queue) is an implementation of DDS. Its underlying layer uses an edge triggering mechanism. When multiple messages arrive after the subscription business (SUB, subscription subscription) flow thread is blocked, the trigger will only be triggered once. Only the messages that are actually triggered will be read, and the untriggered messages will be discarded. Figure 6 This is a schematic diagram of the SUB process. In the figure, message represents message data.

[0062] There are two ways to solve the problem of slow subscribers. One is to store messages in the publisher buffer, but in high-throughput application scenarios, the publisher buffer will cause memory overflow due to accumulated data, especially when facing multiple subscribers, the disk buffer is more likely to overflow. The other is to store messages in the subscriber buffer, which is also the default behavior of ZMQ. This method is very suitable for application scenarios with a large amount of instantaneous messages. The subscriber may only be temporarily unable to keep up with the publisher's speed and will eventually catch up. However, in the scenario of continuously receiving a large amount of messages, the subscriber will crash due to buffer overflow. This method does not solve the problem of slow subscribers itself.

[0063] (2) Slow connection problem

[0064] Data transmission between subscribers and publishers requires establishing a TCP connection. Subscribers establish a connection to publishers using a hard-coded endpoint string, a process involving a TCP three-way handshake. This handshake typically takes several milliseconds for subscribers. For publishers, upon receiving the start command, they begin encapsulating raw data into messages and sending them. The PUB (publish) process and the SUB (subscribe) process execute in parallel, with ZeroMQ performing asynchronous I / O in the background. Because the publisher sends messages before the subscriber establishes a connection, the publisher may not have any subscribers to receive messages. To reduce system load, zmq.PUB sockets automatically discard all messages when there are no receivers. Due to this mechanism, any messages sent by the publisher while a subscriber is connected will be lost. Even if you attempt to start the publisher after starting the subscriber, the several milliseconds required for the TCP three-way handshake cannot be avoided; the publisher will ultimately discard any messages sent before the connection is established. Figure 7This is a diagram illustrating a slow connection problem. In the figure, publisher.bind serializes the original message, reads the message to be published from memory, and converts it into a string. addr:port represents the address and port to which the message will be sent. Send(msgN+1) publishes the N+1th message. Subscriber.connect establishes a connection between the subscriber and the publisher, and Subscriber.recv receives the message.

[0065] There are two solutions to the slow connection problem. The first method is for the publisher to delay publishing and block its PUB process after the publisher is started. The SUB process establishes a connection during the blocking period of the PUB process to achieve process synchronization. The disadvantage of this method is that it is impossible to accurately set the length of time the PUB process is blocked. If the blocking time is set too long, it will cause unnecessary time overhead. If the blocking time is set too short, the sending and receiving processes cannot be fully synchronized, and the slow connection problem cannot be completely solved. The second method regards the publisher's data stream as an infinite stream with no so-called starting point and end point. The first message received after the subscriber establishes a connection is used as the starting point, and the messages discarded when the connection is established are ignored. This method does not solve the slow connection problem and is applicable to fewer scenarios.

[0066] like Figure 8 As shown, to address the slow subscriber problem, the present invention implements the subscriber through multi-threaded workers, with multiple workers executing message receiving threads and business flow threads in parallel. The shared memory inproc protocol serves as a bridge for communication between workers and subscribers, improving model execution efficiency. The multi-threaded worker subscriber logically divides the traditional model subscriber buffer into several partitions, and uses multiple worker threads to read each logically divided partition separately, achieving an effect superior to a single thread processing the entire buffer, improving subscriber message reception capabilities.

[0067] On this basis, in order to achieve load balancing among workers, a polling pool is newly added and implemented in the subscriber of the present invention as a worker status polling interface, which is responsible for orderly managing multiple buffer partitions, and realizes worker load balancing through bidirectional asynchronous listening and polling algorithms, thereby preventing the same worker from processing too many messages and improving the concurrency of the publish-subscribe model; in addition, based on the traditional subscriber, a subscriber collector for message aggregation is designed and implemented, and different workers asynchronously aggregate the message results processed by each of them, and further improve the subscriber's message receiving capability through a lock-free algorithm such as a fair queue.

[0068] In the traditional publish-subscribe model, publishers and subscribers can only communicate in one direction. Subscribers are invisible to publishers, which causes slow connection problems. This invention achieves two-way communication between publishers and subscribers by establishing a two-way channel. Based on this two-way communication, the publisher and subscriber states are synchronized, and the publisher and subscriber processes are controlled to execute synchronously, preventing packet loss caused by slow connection problems.

[0069] Continue reading Figure 8 ,The multi-threaded worker data distribution method mainly includes three parts: the design of the ,relationship structure between subscribers and workers, the message ,distribution design between workers, and the control of the ,timing of worker resources.

[0070] (1) The design of the relationship structure between subscribers and workers. This design involves whether subscribers and workers are decoupled and whether different workers can communicate with each other. It is very critical for the concurrency of the model. When the subscriber starts, multiple worker instances are created through its context subscriber_context. The created workers serve the subscribers, thus forming a subscriber that can process messages in parallel with multiple threads. A worker with a well-designed message distribution mechanism can orderly manage different partitions of the subscriber buffer, which is of great significance for fully utilizing the concurrency of the model and reducing the coupling between workers.

[0071] (2) Design of message distribution between workers. Since workers are message-driven, multiple workers serving the same subscriber are decoupled from each other and do not communicate directly. Instead, subscribers create polling pools to distribute messages to workers. In addition, the rationality of message distribution determines the efficiency of worker task distribution. The load balancing distribution method based on the polling pool is consistent with the data distribution service business scenario. The polling pool that uses the polling algorithm for message distribution is conducive to improving the utilization rate of worker received messages and fully utilizing the concurrency efficiency of multi-threaded programming.

[0072] (3) Control the timing of worker resource release. In the multi-threaded worker data distribution service model, subscribers can dynamically control the timing of worker resource release, which is crucial for the data distribution system. If data is distributed to a worker for reception, the system must wait for the message to be received. The waiting process includes the reception, processing, and aggregation of the message. When the worker is no longer needed to receive messages, the subscriber can release the worker resources in a timely manner to save system resources. When releasing worker resources, the classic resource recycling mechanism of the Python language is used. When the worker reference count is zero, it will be treated as garbage and the memory space occupied by the worker will be recycled. Since this operation involves greater risks, the automatic garbage collection mechanism provided by the Cpython interpreter is used to prevent possible memory overflow problems caused by memory recycling.

[0073] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A real-time distributed system for intelligent driving, characterized by: It includes a perception layer, a decision layer, and a control layer. The perception layer includes a collection unit. The external information collected by the collection unit is encapsulated into a DataWriter and then written to a predefined Topic. Subscribers subscribe to Topics and obtain external messages from them. Publishers publish external messages to the decision layer. The decision layer generates and publishes decision instructions based on the subscribed external messages and external input instructions. The control layer subscribes to the decision instructions and generates and publishes corresponding control instructions. The car adjusts its throttle, steering wheel, brakes, and gear according to the control instructions. The perception layer publishes external messages to the decision layer, the decision layer publishes decision instructions, and the control layer publishes control instructions, all of which are achieved through a multi-threaded worker data distribution method. The multi-threaded worker data distribution method includes: The publisher adds a timestamp and corresponding index to the message data and sends the message to the polling pool. Multiple worker partitions receive messages from the polling pool, obtain the timestamp and index after unpacking, and then calculate the delay and determine whether there is packet loss. The delay and packet loss information are then encapsulated together with the message data and sent to the subscriber collector. The subscriber collector writes the delay and packet loss information into the log, counts the total number of received messages, and determines whether to continue receiving. If the delay and packet loss of the total received messages exceed the preset threshold, a termination signal is sent. The corresponding worker stops receiving messages after receiving the termination signal.

2. A real-time distributed system for intelligent driving according to claim 1, characterized in that: The message data is partitioned according to the preset index segments. Each worker sets the corresponding index segment, and each worker receives the message data of the corresponding index segment. When all the indexes of the message data received by the worker are missing compared with the indexes corresponding to its index segment, it indicates that packet loss has occurred.

3. A real-time distributed system for intelligent driving according to claim 1, characterized in that: The worker sets a corresponding deadline for receiving each message data. If the message data is not received within the deadline, it means that the data is delayed. The delay is calculated by subtracting the timestamp from the deadline for receiving the message data.

4. A real-time distributed system for intelligent driving according to claim 1, characterized in that: When the worker is no longer needed to receive messages, the automatic garbage collection mechanism provided by the CPython interpreter is used to release the worker resources and reclaim the memory space occupied by the worker.

5. The real-time distributed system for intelligent driving according to claim 1, characterized in that: In the absence of external input instructions, the starting address and destination address are input from the high-precision map, and the path is planned using the path planning algorithm. The planned path is input to the decision layer and the control layer. The decision layer derives decision instructions based on the planned path and issues them to the control layer. The control layer generates corresponding control instructions based on the decision instructions.

6. The real-time distributed system for intelligent driving according to claim 1, characterized in that: The acquisition unit includes a camera, a light radar, a GPS and an inertial sensor.

7. The real-time distributed system for intelligent driving according to claim 1, characterized in that: The decision instructions include following, overtaking, changing lanes, braking and steering.

8. The real-time distributed system for intelligent driving according to claim 1, characterized in that: The control instructions generated by the control layer include throttle size, steering wheel angle, braking force, and forward and reverse movement.

Images