Intelligent driving application communication framework visualization system design method considering performance

Abstract

The application relates to a performance-considered intelligent driving application communication framework visual system design method, and belongs to the technical field of intelligent driving system integration. In the application, a message definition module realizes visual definition of message transmission between modules based on XML, supports message addition, deletion, modification and query, and XML file import, mode verification and saving; and a module design module realizes visual design of intelligent driving application algorithm module communication under multiple machines based on XML, supports module interface addition, deletion, modification and query, and corresponding XML file operation. When requirements or design change, message definition and module design can be more flexibly adjusted, tedious modification is reduced, and redundant code accumulation is avoided.

Description

Technical Field

[0001] This invention belongs to the field of intelligent driving system integration technology, specifically relating to a performance-considered visual system design method for intelligent driving application communication framework. Background Technology

[0002] The development of existing intelligent driving application communication frameworks generally follows these steps: First, clearly define the overall module division of the intelligent driving application; second, based on the integrated development environment (IDE), develop the communication framework at the C / C++ code level for each module, including module communication protocol message definitions, module publish / subscribe interfaces, etc.; then, compile the developed code, correct any compilation errors based on the error messages, until compilation is successful; finally, start the application communication framework and use specific tools to verify whether the communication link transmission is normal.

[0003] The above-mentioned intelligent driving application communication framework development process has the following problems:

[0004] Question 1: In existing technical workflows, module division is typically based on requirements specifications, which are then used to create design documents for subsequent development. This approach leads to a series of tedious modifications to subsequent processes should requirements or designs change. In the "module communication protocol message definition" stage, message files must be rewritten according to the interface definition language specifications. Similarly, code modifications are unavoidable in the "module publish / subscribe interface" stage, which not only easily leads to repeated fixing of the same compilation errors but also results in the accumulation of redundant code.

[0005] Question 2: In existing technology processes, verifying the integrity and performance of communication links (including link connectivity and latency) typically relies on testing each link individually using specific tools. However, due to the extreme complexity of communication links in intelligent driving applications, this step-by-step inspection method is not only inefficient but also carries the risk of missing some links.

[0006] Question 3: Current technical workflows often overlook the system resource consumption of the overall application communication framework. Performance optimization is typically considered only after the application algorithm logic has been developed. At this point, it's difficult to distinguish whether the communication framework or the application algorithm is consuming more system resources, thus affecting the effectiveness of performance optimization. Summary of the Invention

[0007] (a) Technical problems to be solved

[0008] The technical problem to be solved by this invention is to realize the visual design of the communication framework for intelligent driving applications. When requirements or design change, message definitions and module designs can be adjusted more flexibly, reducing tedious modifications and avoiding the accumulation of redundant code. At the same time, the status of the communication link can be displayed intuitively, and performance problems can be identified and optimized in advance during the design process, thereby improving the performance optimization effect.

[0009] (II) Technical Solution

[0010] To address the aforementioned technical problems, this invention provides a performance-considered visual system design method for a communication framework for intelligent driving applications. The intelligent driving application system includes an intelligent driving application communication framework and intelligent driving application algorithm modules. The intelligent driving application algorithm modules implement intelligent driving application algorithms, and the results calculated by each intelligent driving application algorithm module are transmitted through a communication middleware. The information transmission framework constructed during transmission is the intelligent driving application communication framework. The visual system design includes a message definition module, a module design module, an application integration module, and a performance testing module.

[0011] The message definition module is used to visualize the message protocol between intelligent driving application algorithm modules based on XML, and generate XML files for reference in subsequent application integration module design; the message definition module supports adding, deleting, modifying and querying message protocols, as well as importing XML files, pattern verification and saving;

[0012] The module design module is used to visualize the communication between intelligent driving application algorithm modules on multiple machines based on XML, and generate XML files. Specifically, it describes the message passing interfaces of intelligent driving application algorithm modules on different machines, which are used for subsequent code generation, compilation, deployment and operation of application integration modules. The module design module supports adding, deleting, modifying and querying the message passing interfaces of intelligent driving application algorithm modules, as well as importing, validating and saving XML files. The machine refers to the domain controller where the intelligent driving application communication framework runs.

[0013] The application integration module is used to generate intelligent driving application communication framework code based on the XML file designed by the message definition module and the module design module, perform automated compilation on different machines, perform automated deployment on different machines, and realize one-click remote start of intelligent driving application communication framework.

[0014] The performance testing module is used to monitor and record the system resources and communication links occupied by the intelligent driving application communication framework using system tools and communication tools, in order to confirm whether the corresponding indicators exceed the predetermined thresholds, thereby determining whether the intelligent driving application communication framework needs to be redesigned.

[0015] The present invention also provides a visualization system for a communication framework for intelligent driving applications designed and implemented based on the method described above.

[0016] This invention also provides a visual design method for an intelligent driving application communication framework implemented using the system, comprising the following steps:

[0017] The first step is to define the message protocol based on the message definition interface, perform pattern verification until the pattern verification is correct, and generate an XML file.

[0018] The second step is to design the publish / subscribe interface for each module on different machines. The message protocol used for publish / subscribe is the same as the message protocol designed in the first step, and the schema is validated until the schema is validated correctly, and then an XML file is generated.

[0019] The third step involves generating code based on the XML files designed in the first and second steps, selecting a platform, and then automating the compilation, deployment, and operation of the intelligent driving application algorithm modules according to the selected platform.

[0020] The fourth step is to set a communication resource usage threshold and remotely monitor and display the intelligent driving application algorithm module started in the third step. If the threshold is exceeded, the message protocol and module design in the first and second steps will be re-executed until the communication resource usage conditions are met.

[0021] (III) Beneficial Effects

[0022] 1. Adaptability to changes in requirements

[0023] Existing technology: Module division is based on the requirement specification document and the design document is written. When the requirements or design change, the subsequent process needs to be modified in a cumbersome way. The module communication protocol message definition needs to be rewritten according to the interface definition language specification. Modification of the module publish and subscribe interface code is also inevitable. It is easy to repeatedly modify the same compilation error and accumulate redundant code.

[0024] This invention comprises two modules: a message definition module based on XML for visually defining messages passed between modules, supporting CRUD operations on messages, as well as importing, validating, and saving XML files; and a module design module based on XML for visually designing communication between intelligent driving application algorithm modules across multiple machines, supporting CRUD operations on module interfaces and corresponding XML file operations. When requirements or designs change, message definitions and module designs can be adjusted more flexibly, reducing tedious modifications and avoiding the accumulation of redundant code.

[0025] 2. Communication link verification efficiency and accuracy

[0026] Existing technology: Verifying the integrity and performance of communication links relies on specific tools to test each link individually. Intelligent driving applications have complex communication links, making this method inefficient and risky of missing detections.

[0027] This invention uses a script to acquire all communication links at the bottom layer, performs connectivity and latency tests on each link one by one, and directly displays the connectivity and latency of the communication links through the interface. This can effectively improve the efficiency and accuracy of communication link verification, and at the same time, it can more intuitively display the status of the communication links.

[0028] 3. Performance optimization effect

[0029] Existing technology: The resource consumption of the overall application communication framework is usually ignored. Performance optimization is only considered after the application algorithm logic is developed. It is difficult to distinguish the resource consumption of the communication framework and the application algorithm, which affects the performance optimization effect.

[0030] This invention: The performance testing module uses system tools and communication tools to monitor and record the system resources and communication links occupied by the intelligent driving application communication framework, confirm whether they exceed the predetermined threshold, and thus determine whether the intelligent driving application communication framework needs to be redesigned. It can identify performance problems in advance during the design process and optimize and adjust them, thereby improving the performance optimization effect. Attached Figure Description

[0031] Figure 1 This is an example of a message definition interface for the present invention;

[0032] Figure 2 This is an example of the module design interface for the present invention;

[0033] Figure 3 This is an example of the application integration interface of the present invention;

[0034] Figure 4 This is an example of the performance testing interface of the present invention;

[0035] Figure 5 This is a flowchart illustrating the overall process of the method of the present invention. Detailed Implementation

[0036] To make the objectives, contents, and advantages of the present invention clearer, the specific embodiments of the present invention will be described in further detail below with reference to the accompanying drawings and examples.

[0037] A single functional module in autonomous driving includes a series of intelligent driving application algorithm modules such as perception, decision-making, planning, and control. These modules implement their respective intelligent driving application algorithms and need to transmit the results calculated by each module through communication middleware. The information transmission framework constructed when the modules transmit information through the communication middleware is called the intelligent driving application communication framework. An intelligent driving application system includes the intelligent driving application communication framework and the intelligent driving application algorithm modules. This invention primarily generates a communication framework for algorithm developers in advance based on the preset message transmission format of each intelligent driving application algorithm module. Thus, algorithm developers only need to focus on the development and debugging of the algorithms themselves.

[0038] The present invention provides a performance-oriented intelligent driving application communication framework visualization system, mainly based on QT design and implementation. It achieves the visual design of the intelligent driving application communication framework through a QT interface, involving four main modules: message definition, module design, application integration, and performance testing. The functions of each module are as follows:

[0039] The message definition module is based on XML to visualize the message protocol design between intelligent driving application algorithm modules (including perception, decision-making, planning, and control modules), generating an XML file (containing the message protocol) for reference in subsequent application integration module design. It supports adding, deleting, modifying, and querying message protocols, as well as importing XML files, schema validation, and saving.

[0040] The module design module is a visual design based on XML for communication between intelligent driving application algorithm modules across multiple machines. It generates XML files that describe the message passing interfaces of the intelligent driving application algorithm modules on different machines. These files are used for subsequent application integration module code generation, compilation, deployment, and execution. It supports adding, deleting, modifying, and querying the message passing interfaces of the intelligent driving application algorithm modules, as well as importing, validating, and saving XML files. "Machine" refers to the domain controller where the intelligent driving application communication framework runs.

[0041] The application integration module generates intelligent driving application communication framework code from XML files designed based on the message definition module and module design module. It performs automated compilation on different machines, automated deployment on different machines, and enables one-click remote startup of the intelligent driving application communication framework.

[0042] The performance testing module uses system and communication tools to monitor and record the system resources and communication links used by the intelligent driving application's communication framework. This confirms whether predetermined thresholds are exceeded, thus determining whether the intelligent driving application's communication framework needs to be redesigned. System tools include the `top` tool, used to monitor the CPU usage of each module, and the `free` tool, used to monitor the memory usage of each module. Communication tools refer to ROS2 tools, including the `ros2 topic` tool, used to view the list of message passing channels, frequency, latency, and publish / subscribe relationships.

[0043] The specific design method is as follows:

[0044] The message definition module is a visual design tool for the message protocol between intelligent driving application algorithm modules, implemented using QT and XML technologies. It primarily edits and verifies the message protocol used for communication between these modules. It supports various basic data types (referring to data types that may be included in the message protocol), such as bool, uint8_t, char, float, double, int8_t, uint16_t, int32_t, int64_t, uint16_t, uint32_t, uint64_t, std::string, etc., as well as complex data types, including std::array.<T,N> It also supports composite structure types such as std::vector, std::vector, etc.

[0045] The visual design and implementation of a message protocol based on QT and XML technologies includes XML file format definition, visual editing, and schema validation. One possible interface design implementation is as follows: Figure 1 As shown, it provides operation buttons and a display interface for XML files.

[0046] The XML file generated by the message definition module is designed with the following format: Messages is the root element, containing all message definitions. Message represents each message type, with a name attribute to identify the message name. Field represents the fields in the message, with name and type attributes to identify the field name and data type. For complex types, such as std::vector and std::array, the subtype and size attributes are used to further describe their internal structure.

[0047] The module design module is based on QT and uses XML to visually design the publish / subscribe interface of the intelligent driving application algorithm module. It is used to describe the message communication links between different intelligent driving application algorithm modules on different machines. In the process of visually designing the publish / subscribe interface, it is necessary to first define the publish / subscribe interface. The message protocol used by the publish / subscribe interface must refer to the message protocol in the XML file generated by the message definition module.

[0048] Visual design implementation based on QT and XML technologies includes XML file format definition, visual editing, and schema validation. One possible interface design implementation is as follows: Figure 2 As shown, it provides operation buttons and a display interface for XML files.

[0049] The XML file generated by the module design module has the following design format: `Machines` is the root element, representing the entire intelligent driving application system and containing multiple machines. The `Machine` element includes an `ID` attribute, an `IP` element, a `Port` element, and a `Modules` element. The `ID` attribute uniquely identifies each machine, the `IP` element identifies the IP address of each machine, the `Port` element identifies the port number that can be remotely connected to the machine, and the `Modules` child element contains all the modules within that machine. Each module is identified by its `name` attribute and further subdivided into `Publishers` and `Subscribers` child elements. The `Publishers` child element contains multiple `Publisher` elements, each containing `Topic` and `Message` elements, used to specify the topic name and message type to be published, respectively. The `Subscribers` child element contains multiple `Subscriber` elements, each containing `Topic` and `Message` elements, used to define the topic name and message type to be subscribed to.

[0050] The visual editing functions of the XML files generated by the message protocol module and the XML files generated by the module design module, as well as the subsequent pattern verification functions, are implemented similarly and are described uniformly here.

[0051] The message protocol module and module design module also implement visual editing functionality for their respective generated XML files based on QT. Editing functions include reading, saving, adding, deleting, modifying, and querying. The specific implementation steps are as follows:

[0052] 1. Read the XML file and display it in a QTreeView.

[0053] Parsing XML files: First, use QDomDocument to read and parse XML files.

[0054] Create the data model: Use QStandardItemModel as the data model for QTreeView. Convert the structure of the XML file into a tree structure and populate the data model.

[0055] Setting the QTreeView model: Set the created QStandardItemModel as the model of the QTreeView.

[0056] 2. Implement CRUD (Create, Read, Update, Delete) functions

[0057] Adding a node: Select a node in the QTreeView, and then add a new child node using the QStandardItemModel data model.

[0058] Deleting a node: Delete a node by selecting it and calling the removeRow method of the QStandardItemModel data model.

[0059] Modify a node: Trigger the modification operation by double-clicking a node or using the right-click menu, and then update the corresponding node in the QStandardItemModel data model.

[0060] Query node: Enables the query function for a specific node through an input box or other controls.

[0061] 3. Save the modified XML file

[0062] Update QDomDocument: When a user modifies a node in a QTreeView, the structure of the XML file in the QDomDocument is updated synchronously.

[0063] Save to file: Use the QDomDocument::save() method to save the updated XML file.

[0064] The following section performs schema verification to check the correctness of the modified XML file. Schema verification includes two parts: XSD file definition and XML schema verification. XML schema verification refers to performing XSD file schema verification on the XML files generated by the message definition module and the module design module of the communication framework visual design system. If the schema is correct, subsequent template code generation proceeds; if the schema is incorrect, the XML file is modified according to the error message until the schema verification is correct.

[0065] The XSD file definition, specifically, defines the communication message protocol between intelligent driving application algorithm modules and the communication structure of the intelligent driving application system. It describes the publish / subscribe relationships between machines, intelligent driving application algorithm modules, and between these modules, as well as the message transmission protocol between them. The XSD file definition includes basic data type definitions, complex data type definitions, composite structure types, machine type definitions, publish / subscribe type definitions, and a root element definition.

[0066] Basic data type definition: using <xs:simpletype>Define basic data types, such as boolType, uint8Type, etc. Each basic type is defined through... <xs:restriction>Specify its underlying type (e.g., xs:boolean, xs:unsignedByte, etc.). These types provide basic data type constraints for elements in an XML document, ensuring data accuracy and consistency.

[0067] Complex data type definition: using <xs:complextype>Define complex data types, such as arrayType and vectorType. These types are defined through... <xs:sequence>and <xs:element>Defines its internal structure, allowing it to contain multiple child elements. maxOccurs="unbounded" indicates that it can have multiple identical child elements, which is suitable for data structures such as arrays and vectors.

[0068] Composite structure type: using <xs:complextype>Define a composite struct type containing multiple fields (child elements), each of which can be a basic or complex type. This struct type is used to define more complex object structures and is suitable for scenarios that require combining multiple data fields.

[0069] Machine Type Definition: `MachineType` defines a machine type and contains a `Modules` element, which can contain multiple `Module` elements. Each `Module` element contains `Publishers` and `Subscribers` child elements, used to define publishers and subscribers, respectively. <xs:extension>Extending PublisherType and SubscriberType allows for the addition of more attributes or elements on top of these types. This information specifies which topics the intelligent driving application algorithm module will publish to and which topics it will subscribe to.

[0070] Publish and Subscribe Type Definitions: PublisherType and SubscriberType define the types of publications and subscriptions, containing Topic and Message elements. These types describe two key pieces of information that each publish or subscribe operation must include: the topic name and the message type. Topic specifies the message topic, and Message specifies the message type (using xs:QName to represent the qualified name).

[0071] Root element definition: Define "Machines" as the root element, which contains multiple "Machine" elements, each representing a machine instance. This structure allows multiple machines, their intelligent driving application algorithm modules, and message publish-subscribe relationships to be defined within an XML document.

[0072] The XML schema validation function is implemented using QT-based XSD-driven XML file schema validation technology. The specific steps are as follows:

[0073] 1. Load the XSD file

[0074] Place the XSD file created above in a fixed path and load it using the QXmlSchema class. Check if the XSD file was loaded successfully; if it fails, display a message indicating the loading error (e.g., file not found or incorrect format).

[0075] 2. Verify the validity of the XSD file

[0076] After loading the XSD file, verify its validity and ensure it conforms to the XML Schema specification. If the XSD file is invalid, prompt the user to repair it or obtain a valid XSD file again.

[0077] 3. Create a validator

[0078] Create a QXmlSchemaValidator instance using the loaded QXmlSchema object. This is used to validate whether the XML file conforms to the structure and constraints defined by the XSD.

[0079] 4. Validate the XML file

[0080] Use the QXmlSchemaValidator validation methods to validate the user-loaded XML file. Check the validation results to determine whether the XML file conforms to the structure and data type constraints defined by XSD.

[0081] 5. Process the verification results

[0082] If validation passes, the XML file is valid and conforms to the requirements defined by XSD. If validation fails, the user is prompted with a validation error message, allowing them to analyze the cause of the error and modify or adjust the XML file.

[0083] The application integration module generates application communication framework code from XML files designed based on the message definition module and module design module. It performs automated compilation for different platforms, automated deployment on different machines, and one-click remote startup of the application communication framework. A possible interface for the application integration module is shown below. Figure 3 As shown, it provides operation buttons and a log display interface.

[0084] The intelligent driving application communication framework code generation method involves using Python to populate a code template file with XML files generated from the message definition module and module design module (as designed in the previous steps), thus generating the code file. This invention uses ROS2 as the communication library for code file generation, but other communication libraries can also be used. The specific steps for generating the intelligent driving application communication framework code are as follows:

[0085] 1. Parse two types of XML files (XML files generated by the message definition module and XML files generated by the module design module).

[0086] Use Python's XML parsing library (such as xml.etree.ElementTree or lxml) to read and parse XML files, extracting relevant information for generating code, such as message definition structure fields, intelligent driving application algorithm module names, publish / subscribe interface information, etc.

[0087] 2. Create a C++ code template file

[0088] Create a C++ code template file, typically in text file format (such as .cpp or .h). The C++ code template file contains placeholders for inserting specific information parsed from the various XML files, specifically the names of the intelligent driving application algorithm modules and publish / subscribe interfaces extracted from the XML files in the previous step. For example, generate a class definition template for each node, including necessary header files, node initialization code, and callback functions for message publishing and subscription.

[0089] 3. Fill in the placeholders in the C++ code template file.

[0090] Use Python's string formatting methods (such as str.format() or f-strings) to populate placeholder positions in the C++ code template file with information parsed from the XML files.

[0091] 4. Generate ROS2 message definition file

[0092] Based on the message definition structure fields extracted from each XML file, ROS2 message definition files are generated with the .msg extension. These ROS2 message definition files define the message format, including the message field names and types.

[0093] 5. Generate the CMakeLists.txt file

[0094] Based on the generated C++ code template file and project dependencies, a CMakeLists.txt file is produced. The CMakeLists.txt file should contain the necessary compilation instructions and dependencies for compilation in the ROS2 environment.

[0095] 6. Generate the package.xml file

[0096] Generate a package.xml file based on project information, containing metadata such as package name, version, and dependencies. This content is dynamically generated using a Python script.

[0097] 7. Save the generated file

[0098] Save the generated C++ code template file, CMakeLists.txt file, and package.xml file to the specified directory for subsequent code compilation, deployment, and execution.

[0099] Code compilation involves automating the compilation of the generated code module by module. Depending on the platform selected by the user through the interface, it is divided into x86 architecture compilation and cross-compilation. Python's XML parsing library is used to read the names of the modules to be compiled from the XML configuration file, and the corresponding compilation script is called to compile the extracted modules according to the target machine's architecture.

[0100] Deployment and execution involve uploading the compiled code output to a designated machine based on the XML file obtained from the module design module. First, Python's XML parsing library reads the XML file from the module design module and generates the corresponding program startup script based on the module name on different machines. Second, Python's paramiko library creates an SSH client object on the local machine, automatically adding the remote host's SSH key to avoid manual intervention. It parses the machine IP address and port number specified in the XML file from the module design module, establishes an SSH connection with the remote machine, and transfers the compiled output and program startup script to a specified path on the target machine via SSH. Execution also involves creating an SSH client object to remotely connect to the machine and execute the execution script.

[0101] The performance testing module monitors and displays the processes of the intelligent driving application communication framework launched by the application integration module. First, it sets communication resource consumption thresholds, including the CPU and memory resources used by each process of the intelligent driving application communication framework, as well as the communication latency of each communication link. It primarily uses Python scripts to extract and display the CPU and memory usage of each intelligent application algorithm module and the latency of each communication link. A possible performance testing interface is shown below. Figure 4 As shown, there is a threshold setting button and a display interface.

[0102] The specific steps for the performance testing module to monitor and display the process of the intelligent driving application communication framework started by the application integration module are as follows:

[0103] 1. Establish SSH connection

[0104] Use Python's paramiko library to create an SSH client object on the local machine, automatically add the remote host's SSH key to avoid manual intervention, parse the machine IP address and port number specified in the XML file, and establish an SSH connection with the remote machine.

[0105] 2. Execute communication resource monitoring commands

[0106] This section describes a method using Python to invoke system commands (such as the `ps` command) on a remote machine to obtain CPU and memory usage information for a specified process. The `grep` command is used to filter out lines containing the target process name and exclude lines containing the `grep` command itself, ensuring accurate process information. It also describes using Python to invoke a command on the remote machine's ROS2 system to obtain all communication links. The ROS2 communication latency command is then used to test each link individually, and the `grep` command is used to filter and extract specific latency data, including average latency, minimum latency, and maximum latency.

[0107] 3. Analyze and display data

[0108] The acquired text data is parsed on the local machine to extract key information such as process CPU and memory usage, as well as latency data for each communication link. The parsed data is then displayed in an easy-to-understand format through a graphical interface. Data exceeding a set threshold is highlighted.

[0109] Users determine whether to adjust the communication framework of the intelligent driving application based on the communication resource usage results of the interface, until the performance requirements are met.

[0110] Based on the above-described QT-based intelligent driving application communication framework development interface, the overall flow of the performance-considered intelligent driving application communication framework visual design method of the present invention is as follows: Figure 5 As shown. The first step is to define the message protocol based on the message definition interface, perform pattern verification until the pattern verification is correct, and generate an XML file. The second step is to design the publish / subscribe interface for each module on different machines. The message protocol used for publish / subscribe references the message protocol designed in the first step, and performs pattern verification until the pattern verification is correct, generating an XML file. The third step is to generate code based on the XML files designed in the first and second steps, select the platform, and perform automated compilation, deployment, and operation according to the selected platform. The fourth step is to set a communication resource usage threshold, remotely monitor and display the intelligent driving application algorithm module started in the third step. If the threshold is exceeded, the message protocol and module design in the first and second steps are re-executed until the communication resource usage conditions are met.

[0111] It can be seen that the present invention has the following characteristics:

[0112] 1. Visual design

[0113] By developing a QT-based interface, we have enabled visual operations for message definition, module design, and application integration in the intelligent driving application communication framework. This makes the design process more intuitive and convenient, reduces development difficulty, and improves development efficiency.

[0114] 2. Flexibility in message definition and module design

[0115] The message definition module uses XML to visually define messages passed between modules. It supports CRUD operations on various basic data types, complex data types, and combined structure types, as well as importing, validating, and saving XML files. The module design module also uses XML to visually design communication between intelligent driving application modules across multiple machines. It can describe the message passing interfaces of modules on different machines and supports corresponding CRUD functions. This design approach allows for more flexible adjustments to message definitions and module designs when requirements or designs change, reducing tedious modification work and avoiding the accumulation of redundant code.

[0116] 3. Automated integration and deployment

[0117] The application integration module can generate intelligent driving application communication framework code based on XML files designed by the message definition module and module design module. It also performs automated compilation for different platforms, automated deployment on different machines, and enables remote launch of the application communication framework via a button. This significantly improves the efficiency and accuracy of application integration, reducing the tediousness and error probability of manual operations.

[0118] 4. Performance Testing and Optimization

[0119] The performance testing module uses system and communication tools to monitor and record the system resources and communication links used by the application's communication framework. This allows it to confirm whether predetermined thresholds are exceeded, providing a basis for deciding whether to redesign the intelligent driving application's communication framework. This approach, considering performance during the design process and conducting testing accordingly, helps to identify performance issues early and make timely optimizations. It avoids the difficulty in distinguishing between the communication framework and application algorithm resource usage that often occurs after the application algorithm logic has been developed, thus improving the effectiveness of performance optimization.

[0120] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the technical principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.< / xs:extension> < / xs:complextype> < / xs:element> < / xs:sequence> < / xs:complextype> < / xs:restriction> < / xs:simpletype>

Claims

1. A performance-considered visual system design method for intelligent driving application communication frameworks, characterized in that, The intelligent driving application system includes an intelligent driving application communication framework and an intelligent driving application algorithm module. The intelligent driving application algorithm module implements the intelligent driving application algorithm, and the results calculated by each intelligent driving application algorithm module are transmitted through a communication middleware. The information transmission framework constructed during transmission is the intelligent driving application communication framework. The visualization system is designed to include a message definition module, a module design module, an application integration module, and a performance testing module. The message definition module is used to visualize the message protocol between intelligent driving application algorithm modules based on XML, and generate XML files for reference in subsequent module design and application integration module design. The message definition module supports adding, deleting, modifying and querying message protocols, as well as importing XML files, pattern verification and saving. The module design module is used to visualize the communication between intelligent driving application algorithm modules across multiple machines based on XML, and generates XML files that describe the message passing interfaces of intelligent driving application algorithm modules on different machines. These files are used for subsequent code generation, compilation, deployment, and operation of the application integration module. The module design module references the message protocol in the XML file generated by the message definition module. The module design module supports adding, deleting, modifying, and querying the message passing interfaces of intelligent driving application algorithm modules, as well as importing, validating, and saving XML files. The machine refers to the domain controller where the intelligent driving application communication framework runs. The application integration module is used to generate intelligent driving application communication framework code based on XML files designed by the message definition module and module design module. Specifically, it extracts message definition structure fields from the XML file generated by the message definition module, extracts intelligent driving application algorithm module names and publish / subscribe interface information from the XML file generated by the module design module, creates a C++ code template file containing placeholders, fills the placeholder positions in the C++ code template file with the information extracted from the XML file, generates CMakeLists.txt and package.xml files, and saves them to a specified directory; it performs automated compilation on different machines, automated deployment according to different machines, and realizes one-click remote startup of the intelligent driving application communication framework. Specifically, it automatically compiles the generated code for each intelligent driving application algorithm module; it uses Python's XML parsing library to read the module names to be compiled from the XML configuration file, and calls the corresponding compilation script to compile the extracted intelligent driving application algorithm modules according to the architecture of the target machine. The code compilation artifacts are uploaded to the specified machine based on the XML file obtained from the module design module; Python's XML parsing library is used to read the XML file obtained from the module design module, and the corresponding program startup script is generated according to the module name on different machines; Python's paramiko library is used to create an SSH client object on the local machine, automatically add the SSH key of the remote host, parse the machine IP address and port number specified in the XML file obtained from the module design module, establish an SSH connection with the remote machine, and transfer the compilation artifacts and program startup script to the specified path on the target machine via SSH; the SSH client object is then used to remotely connect to the machine and execute the running script. The performance testing module is used to monitor and record the system resources and communication links occupied by the intelligent driving application communication framework using system tools and SSH connection, in order to confirm whether CPU resources, memory size, and communication latency exceed the predetermined thresholds, thereby determining whether the intelligent driving application communication framework needs to be redesigned.

2. The method as described in claim 1, characterized in that, The message definition module uses QT and XML technologies to implement the visual design of message protocols between intelligent driving application algorithm modules; the visual design of message protocols based on QT and XML technologies includes XML file format definition, visual editing, and pattern verification; The XML file generated by the message definition module has the following design format: It includes: Messages: as the root element, containing all message definitions. Message has a name attribute to identify the message name; Field: the fields in the message, with name and type attributes to identify the field name and data type.

3. The method as described in claim 1, characterized in that, The module design module is based on QT and uses XML to visually design the publish-subscribe interface of the intelligent driving application algorithm module. It is used to describe the message communication links between different intelligent driving application algorithm modules on different machines. In the process of visually designing the publish-subscribe interface, the publish-subscribe interface is first defined, and the message protocol used by the publish-subscribe interface is defined by referencing the message protocol in the XML file generated by the message definition module. Visual design implementation based on QT and XML technologies includes XML file format definition, visual editing, and schema validation; The XML file generated by the module design module has the following design format: This includes: Machines: As the root element, representing the entire intelligent driving application system, containing multiple machines; Machine elements: including ID attribute, IP element, Port element, and Modules element. The ID attribute uniquely identifies each machine, the IP element identifies the IP address of each machine, the Port element identifies the port number that can be remotely connected to the machine, and the Modules element contains all the modules in that machine; Each Module is identified by its name attribute and is further subdivided into Publishers and Subscribers sub-elements; The Publishers sub-elements contain multiple Publisher elements, each containing two Topic and Message elements, used to specify the topic name and message type to be published, respectively; The Subscribers sub-elements contain multiple Subscriber elements, each also containing two Topic and Message elements, used to define the topic name and message type to be subscribed to.

4. The method as described in claim 1, characterized in that, Both the message protocol module and the module design module are based on QT to implement visual editing functionality for their respective generated XML files. This visual editing functionality includes reading, saving, adding, deleting, modifying, and querying. The specific implementation steps are as follows: 1) Read the XML file and display it in QTreeView Parsing XML files: First, use QDomDocument to read and parse the XML file; Create the data model: Use QStandardItemModel as the data model for QTreeView; convert the structure of the XML file into a tree structure and populate it into the data model; Set the QTreeView model: Set the created QStandardItemModel as the model of the QTreeView; 2) Implement CRUD (Create, Read, Update, Delete) functions Adding a node: Select a node in the QTreeView, and then add a new child node using the QStandardItemModel data model; Delete a node: Delete a node by selecting it and calling the removeRow method of the QStandardItemModel data model; Modify a node: Trigger the modification operation by double-clicking a node or using the right-click menu, and then update the corresponding node in the QStandardItemModel data model; Query Nodes: Enables the query function for nodes through input boxes or controls; 3) Save the modified XML file Update QDomDocument: When a user modifies a node in a QTreeView, the structure of the XML file in the QDomDocument is updated synchronously. Save to file: Use the QDomDocument::save() method to save the updated XML file.

5. The method as described in claim 1, characterized in that, Both the message protocol module and the module design module perform schema verification on the correctness of the modified XML file. Schema verification includes two parts: XSD file definition and XML schema verification. XML schema verification means: performing schema verification on the XML file based on the XSD file. If the schema is correct, the subsequent template code is generated. If the schema is incorrect, the XML file is modified according to the error message until the schema verification is correct. XSD file definitions are used to describe machines, intelligent driving application algorithm modules, publish and subscribe relationships between intelligent driving application algorithm modules, and message protocols between intelligent driving application algorithm modules; XSD file definitions include basic data type definitions, complex data type definitions, composite structure types, machine type definitions, publish and subscribe type definitions, and root element definitions; Basic data type definition: using <xs:simpletype>Define basic data types; each basic type is defined by... <xs:restriction> Specify its underlying types, which provide basic data type constraints for elements in the XML document;< / xs:restriction> < / xs:simpletype> Complex data type definition: using <xs:complextype>Define complex data types, which are obtained through... <xs:sequence>and <xs:element> Defines its internal structure, allowing it to contain multiple child elements; maxOccurs="unbounded" indicates that there are multiple identical child elements, which is suitable for array and vector data structures;< / xs:element> < / xs:sequence> < / xs:complextype> Composite structure type: using <xs:complextype> Define a composite struct type containing multiple fields, each of which is a basic or complex type; this composite struct type is used to define object structures that are more complex than complex data types.< / xs:complextype> Machine Type Definition: `MachineType` defines a machine type, containing a `Modules` element, which in turn contains multiple `Module` elements. Each `Module` element contains `Publishers` and `Subscribers` child elements, used to define publishers and subscribers, respectively. <xs:extension> Extend PublisherType and SubscriberType to allow adding more attributes or elements on top of these machine types, specifying which topics the intelligent driving application algorithm module will publish to and which topics it will subscribe to;< / xs:extension> Publish and Subscribe Type Definitions: Use PublisherType and SubscriberType to define the types of publications and subscriptions, including Topic and Message elements; These publish and subscribe types describe two key pieces of information contained in each publish or subscribe operation: the topic name and the message type; The Topic element is used to specify the message topic, and the Message element is used to specify the message type; Root element definition: Define Machines as the root element, which contains multiple Machine elements. Each Machine element is a machine instance. This structure allows multiple machines and their intelligent driving application algorithm modules to be defined in an XML document, as well as message publish-subscribe relationships. The XML schema validation function is implemented using QT-based XSD driver XML file schema validation technology. The specific steps are as follows: 1) Load the XSD file Place the created XSD file in a fixed path, and load the XSD file using the QXmlSchema class; check if the XSD file is loaded successfully, and if it fails, prompt for handling the loading error; 2) Verify the validity of the XSD file After loading the XSD file, verify its validity and ensure it conforms to the XML Schema specification. If the XSD file is invalid, prompt for repair or to obtain a valid XSD file again. 3) Create a validator Create a QXmlSchemaValidator instance using the loaded QXmlSchema object to validate whether the XML file conforms to the structure and constraints defined by XSD; 4) Validate XML files The QXmlSchemaValidator validation method is used to validate the XML file loaded by the user, check the validation results, and determine whether the XML file conforms to the structure and data type constraints defined by XSD. 5) Processing verification results If the validation passes, it means that the XML file is valid and meets the requirements of the XSD definition; if the validation fails, the user is prompted with a validation error message.

6. The method as described in claim 1, characterized in that, The application integration module generates the intelligent driving application communication framework code specifically by using an XML file generated by the message definition module and an XML file generated by the module design module in Python to populate the code template file, thus generating the code file. The specific steps for generating the intelligent driving application communication framework code are as follows: 1) Parsing two types of XML files Use Python's XML parsing library to read and parse XML files, extracting relevant information for generating code, including message definition structure fields, intelligent driving application algorithm module names, and publish / subscribe interface information; 2) Create a C++ code template file Create a C++ code template file using a text file format. The C++ code template file contains placeholders for inserting specific information parsed from each XML file, specifically the names of intelligent driving application algorithm modules and publish / subscribe interface information extracted from each XML file. 3) Fill in the placeholders in the C++ code template file. Use Python's string formatting methods to fill the placeholder positions in the C++ code template file with the information parsed from the various XML files; 4) Generate the CMakeLists.txt file Based on the generated C++ code template file and project dependencies, a CMakeLists.txt file is generated. The CMakeLists.txt file contains compilation instructions and dependencies for compilation in the ROS2 environment. 5) Generate the package.xml file Generate a package.xml file based on the project information, which contains metadata such as the package name, version, and dependencies; 6) Save the generated file Save the generated C++ code template file, CMakeLists.txt file, and package.xml file to the specified directory for subsequent compilation, deployment, and operation of the intelligent driving application algorithm module code.

7. The method as described in claim 1, characterized in that, The performance testing module monitors and displays the process of the intelligent driving application communication framework launched by the application integration module. First, it sets the communication resource usage threshold, including the CPU resources, memory resources, and communication latency of each process of the intelligent driving application communication framework. Then, it uses Python scripts to extract and display the CPU and memory usage of each intelligent application algorithm module and the latency of each communication link.

8. The method as described in claim 7, characterized in that, The specific steps by which the performance testing module monitors and displays the process of the intelligent driving application communication framework started by the application integration module are as follows: 1) Establish SSH connection Use Python's paramiko library to create an SSH client object on the local machine, automatically add the SSH key of the remote host, parse the machine IP address and port number specified in the XML file, and establish an SSH connection with the remote machine. 2) Execute communication resource monitoring commands This program uses Python to call system commands on a remote machine to obtain CPU and memory usage information for a specified process. The `grep` command is used to filter out lines containing the target process name and exclude lines containing the `grep` command itself. Python is also used to call the ROS2 command on the remote machine to obtain all communication links. Then, the ROS2 communication latency command is called to test each communication link individually. The `grep` command is used to filter and extract specific latency data, including average latency, minimum latency, and maximum latency. 3) Analyze and display data The acquired text data is parsed on the local machine to extract CPU and memory usage data of the process, as well as latency data of each communication link; the extracted data is then visualized through a graphical interface, and any data exceeding a set threshold is highlighted.

9. A visualization system for a communication framework for intelligent driving applications designed and implemented based on the method described in any one of claims 1 to 8.

10. A visual design method for an intelligent driving application communication framework implemented using the system described in claim 9, characterized in that, Includes the following steps: The first step is to define the message protocol based on the message definition interface, perform pattern verification until the pattern verification is correct, and generate an XML file. The second step is to design the publish / subscribe interface for each module on different machines. The message protocol used for publish / subscribe is the same as the message protocol designed in the first step, and the schema is validated until the schema is validated correctly, and then an XML file is generated. The third step involves generating code based on the XML files designed in the first and second steps, selecting a platform, and then automating the compilation, deployment, and operation of the intelligent driving application algorithm modules according to the selected platform. The fourth step is to set a communication resource usage threshold and remotely monitor and display the intelligent driving application algorithm module started in the third step. If the threshold is exceeded, the message protocol and module design in the first and second steps will be re-executed until the communication resource usage conditions are met.

Images