Performance-considered intelligent driving application communication framework visualization system design method

Abstract

The invention relates to an intelligent driving application communication framework visualization system design method considering performance, and belongs to the technical field of intelligent driving system integration. In the invention, the message definition module realizes visual definition of messages transmitted among the modules based on XML, and supports addition, deletion, modification and check of the messages and import, mode verification and storage of XML files; the module design module realizes visual design of communication of the intelligent driving application algorithm modules under a plurality of machines based on XML, and supports addition, deletion, modification and check of module interfaces and corresponding operation of XML files. When a demand or a design is changed, message definition and module design can be adjusted more flexibly, tedious modification is reduced, and redundant code accumulation is avoided.

Description

Technical Field

[0001] The present invention belongs to the technical field of intelligent driving system integration, and particularly relates to a design method for visualizing an intelligent driving application communication framework considering performance. Background Art

[0002] The development of the existing intelligent driving application communication framework generally follows the following steps: First, clarify the overall module division of the intelligent driving application; Second, based on the integrated development environment IDE, develop the communication framework at the C / C++ code level for the divided modules in sequence, including the definition of module communication protocol messages, the publish-subscribe interfaces of the modules, etc.; Then, compile the above-developed code, and correct the compilation errors according to the compilation error prompts until the compilation is successful; Finally, start the application communication framework and confirm whether the communication link transmission is normal through specific tools.

[0003] The above intelligent driving application communication framework development process has the following problems:

[0004] Problem 1: In the existing technical process, the module division usually writes the design document based on the requirements specification document and conducts subsequent development accordingly. Once this method faces changes in requirements or designs, it will lead to a series of cumbersome modifications in the subsequent processes. In the "module communication protocol message definition" link, the message file must be rewritten according to the specifications of the interface definition language. In the "module publish-subscribe interface" link, code modification is also inevitable, which not only easily leads to the repeated occurrence of the same compilation errors but also easily causes the accumulation of redundant code.

[0005] Problem 2: In the existing technical process, verifying the integrity and performance of the communication link (including link connectivity and latency) usually relies on using specific tools to test each link separately. However, due to the extremely complex communication links in intelligent driving applications, this method of checking one by one is not only inefficient but also has the risk of missed inspections.

[0006] Problem 3: The current technical process usually ignores the occupation of system resources by the overall application communication framework. Often, only after the development of the application algorithm logic is completed, performance optimization is considered. At this time, it is very difficult to distinguish whether the communication framework or the application algorithm occupies more system resources, thus affecting the effect of performance optimization. Summary of the Invention

[0007] (1) Technical Problems to be Solved

[0008] The technical problem to be solved by the present invention is to achieve the visual design of the intelligent driving application communication framework. When requirements or design changes occur, the message definition and module design can be adjusted more flexibly, reducing cumbersome modifications and avoiding the accumulation of redundant code. At the same time, the status of the communication link is visually displayed, and performance problems can be discovered and optimized in advance during the design process, improving the performance optimization effect.

[0009] (II) Technical Solution

[0010] To solve the above technical problem, the present invention provides a method for designing a visual system for an intelligent driving application communication framework considering performance. The intelligent driving application system includes an intelligent driving application communication framework and an intelligent driving application algorithm module; the intelligent driving application algorithm module implements the intelligent driving application algorithm, and the results calculated by each intelligent driving application algorithm module are transmitted through a communication middleware. The information transmission framework constructed during transmission is the intelligent driving application communication framework; the visual system is designed to include a message definition module, a module design module, an application integration module, and a performance testing module;

[0011] The message definition module is used to visually design the message protocol between intelligent driving application algorithm modules based on XML and generate an XML file for subsequent reference during the design of the application integration module; the message definition module supports the addition, deletion, modification, and query of the message protocol, as well as the import, schema validation, and saving of XML files;

[0012] The module design module is used to visually design the communication between intelligent driving application algorithm modules under multiple machines based on XML and generate an XML file, specifically describing the message transmission interfaces of intelligent driving application algorithm modules under different machines for subsequent code generation, compilation, deployment, and operation of the application integration module; the module design module supports the addition, deletion, modification, and query of the message transmission interfaces of intelligent driving application algorithm modules, as well as the import, schema validation, and saving of XML files; the machine refers to the domain controller where the intelligent driving application communication framework runs;

[0013] The application integration module is used to generate the intelligent driving application communication framework code based on the XML files designed by the message definition module and the module design module, perform automatic compilation for different machines, perform automatic deployment according to different machines, and implement one-key remote startup of the intelligent driving application communication framework;

[0014] The performance testing module is used to monitor and record the system resources and communication links occupied by the intelligent driving application communication framework using system tools and communication tools to confirm whether the corresponding indicators exceed the established thresholds, so as to determine whether to redesign the intelligent driving application communication framework.

[0015] The present invention also provides an intelligent driving application communication framework visual system designed and implemented based on the above method.

[0016] The present invention also provides a method for visual design of an intelligent driving application communication framework implemented by the system, comprising the following steps:

[0017] The first step is to define the message protocol based on the message definition interface, perform schema verification until the schema verification is correct, and generate an XML file;

[0018] The second step is to design the publish-subscribe interface for each module on different machines. The message protocol used for publish-subscribe refers to the message protocol designed in the first step, and the schema is verified until the schema is verified correctly and an XML file is generated.

[0019] The third step is to generate code based on the XML files designed in the first and second steps, select the platform, and automatically compile, deploy, and run the intelligent driving application algorithm module based on the selected platforms;

[0020] The fourth step is to set the communication resource occupancy threshold and remotely monitor and display the intelligent driving application algorithm module started in the third step. If the threshold is exceeded, the message protocol and module design of the first and second steps are repeated until the communication resource occupancy meets the conditions.

[0021] (III) Beneficial effects

[0022] 1. Adaptability to changes in demand

[0023] Existing technology: Module division is based on the writing of design documents based on the requirements specification documents. When the requirements or design changes, the subsequent processes need to be cumbersomely modified. The module communication protocol message definition needs to rewrite the message file according to the interface definition language specification. Modification of the module publishing and subscription interface code is also inevitable. It is easy to modify the same compilation errors repeatedly and accumulate redundant code.

[0024] The present invention: The message definition module implements the visual definition of message transmission between modules based on XML, supports the addition, deletion, modification and query of messages, and the import, model verification and preservation of XML files; the module design module implements the visual design of intelligent driving application algorithm module communication under multiple machines based on XML, supports the addition, deletion, modification and query of module interfaces and the corresponding operations of XML files. When the demand or design changes, the message definition and module design can be adjusted more flexibly, reducing tedious modifications and avoiding the accumulation of redundant code.

[0025] 2. Communication link verification efficiency and accuracy

[0026] Existing technology: Verifying the integrity and performance of communication links relies on specific tools to test each link individually. The communication links of intelligent driving applications are complex, so this method is inefficient and has the risk of missed detection.

[0027] The present invention: The underlying layer obtains all communication links through scripts, and conducts connectivity and latency tests on each communication link one by one, and directly displays the connectivity and latency of the communication links through the interface, which can effectively improve the verification efficiency and accuracy of the communication links, and at the same time can more intuitively display the status of the communication links.

[0028] 3. Performance optimization effect

[0029] The prior art: Usually ignores the occupation of system resources by the overall application communication framework, and only considers performance optimization after the application algorithm logic is developed. It is difficult to distinguish the resource occupation of the communication framework and the application algorithm, which affects the performance optimization effect.

[0030] The present invention: The performance test module uses system tools and communication tools to monitor and record the system resources and communication links occupied by the intelligent driving application communication framework, and confirms whether it exceeds the established threshold, so as to determine whether to redesign the intelligent driving application communication framework. It can discover performance problems in advance during the design process and optimize and adjust them to improve the performance optimization effect. Description of the drawings

[0031] Figure 1 It is an example of the message definition interface of the present invention;

[0032] Figure 2 It is an example of the module design interface of the present invention;

[0033] Figure 3 It is an example of the application integration interface of the present invention;

[0034] Figure 4 It is an example of the performance test interface of the present invention;

[0035] Figure 5 It is the overall flowchart of the method of the present invention. Detailed implementation manners

[0036] To make the objectives, contents, and advantages of the present invention clearer, the following further describes the detailed implementation manners of the present invention in conjunction with the drawings and embodiments.

[0037] A module for completing a function in driverless driving includes a series of intelligent driving application algorithm modules such as perception, decision-making, planning, and control. These modules implement the corresponding intelligent driving application algorithms, and the results calculated by each module also need to be transmitted through a communication middleware. The information transmission framework constructed when each module transmits through the communication middleware is called an intelligent driving application communication framework. The intelligent driving application system includes an intelligent driving application communication framework and intelligent driving application algorithm modules. The present invention mainly generates a set of communication frameworks for algorithm developers in advance according to the format of the transmitted messages preset by each intelligent driving application algorithm module. In this way, the algorithm developers only need to focus on the development and debugging of the algorithm itself.

[0038] A visualization system for an intelligent driving application communication framework considering performance provided by the present invention is mainly designed and implemented based on QT. The visualization design of the intelligent driving application communication framework is realized through the QT interface, which involves four modules: message definition, module design, application integration, and performance testing. The functions of each module are as follows:

[0039] The message definition module is used to visually design the message protocol between intelligent driving application algorithm modules (including perception, decision-making, planning, control, etc. modules) based on XML, generate an XML file (including the message protocol), and is used for reference during the subsequent application integration module design. It supports the addition, deletion, modification, and query of the message protocol, as well as the import, schema validation, and saving of XML files.

[0040] The module design module is used to visually design the communication between intelligent driving application algorithm modules under multiple machines based on XML, generate an XML file, specifically describe the message passing interfaces of intelligent driving application algorithm modules under different machines, and is used for code generation, compilation, deployment, and operation of the subsequent application integration module. It supports the addition, deletion, modification, and query of the message passing interfaces of intelligent driving application algorithm modules, as well as the import, schema validation, and saving of XML files. A machine refers to the domain controller where the intelligent driving application communication framework runs, and the intelligent driving application communication framework runs in this machine.

[0041] The application integration module generates the intelligent driving application communication framework code based on the XML files designed by the message definition module and the module design module, performs automatic compilation for different machines, performs automatic deployment according to different machines, and realizes one-key remote startup of the intelligent driving application communication framework.

[0042] The performance testing module uses system tools and communication tools to monitor and record the system resources and communication links occupied by the intelligent driving application communication framework to confirm whether they exceed the established thresholds, so as to determine whether to redesign the intelligent driving application communication framework. System tools include the top tool for monitoring the CPU occupancy of each module; the free tool for monitoring the memory occupancy of each module. Communication tools refer to ROS2 tools, including the ros2 topic tool for viewing the message passing channel list, frequency, and latency, as well as the publish-subscribe relationship.

[0043] The specific design method is as follows:

[0044] The message definition module realizes the visual design of the message protocol between intelligent driving application algorithm modules based on QT and XML technologies. It mainly edits and validates the message protocol used for communication between intelligent driving application algorithm modules. It supports a variety of basic data types (referring to the data types that may be included in the message protocol), such as bool, uint8_t, char, float, double, int8_t, uint16_t, int32_t, int64_t, uint16_t, uint32_t, uint64_t, std::string, etc., as well as complex data types, including std::array<T,N>, std::vector, etc., and also supports composite structure types composed of the above types.

[0045] The realization of the visual design of the message protocol based on QT and XML technologies includes XML file format definition, visual editing, and schema validation. A possible interface design implementation is as Figure 1 shown, providing operation buttons and display interfaces for XML files.

[0046] The design format of the XML file generated by the message definition module is as follows: Messages is used as the root element, containing all message definitions. Message represents each message type and has a name attribute to identify the message name. Field is a field in the message and has name and type attributes to identify the field name and data type. For complex types, such as std::vector and std::array, the subtype and size attributes are used to further describe their internal structures.

[0047] The module design module realizes the visual design of the publish-subscribe interfaces of intelligent driving application algorithm modules based on QT using XML, which is used to describe the message communication links between different intelligent driving application algorithm modules on different machines. During the process of visual design of the publish-subscribe interfaces, it is first necessary to define the publish-subscribe interfaces, and the message protocol used to define the publish-subscribe interfaces needs to reference the message protocol in the XML file generated by the message definition module.

[0048] The realization of the visual design based on QT and XML technologies includes XML file format definition, visual editing, and schema validation. A possible interface design implementation is as Figure 2 shown, providing operation buttons and display interfaces for XML files.

[0049] The XML file design format generated by the module design module is as follows: Machines, as the root element, represents the entire intelligent driving application system and contains multiple machines (Machine). The Machine element includes an ID attribute, an IP element, a Port element, and a Modules element. The ID attribute uniquely identifies each machine, the IP element identifies the IP of each machine, the Port element identifies the port number through which the machine can be remotely connected, and the Modules subelement contains all the modules (Module) in the machine. Each Module module is identified by its name attribute and is further divided into two subelements, Publishers and Subscribers. The Publishers subelement contains multiple Publisher elements, and each Publisher element contains two elements, Topic and Message, which are used to specify the name of the published topic and the message type respectively. The Subscribers subelement contains multiple Subscriber elements, and each Subscriber element also contains two elements, Topic and Message, which are used to define the name of the subscribed topic and the message type.

[0050] The visualization editing function of the XML file generated by the message protocol module and the XML file generated by the module design module, as well as the subsequent schema validation function, are implemented similarly and are described uniformly here.

[0051] The message protocol module and the module design module also respectively implement the function of visualizing and editing the XML files they generate based on QT. The editing functions include reading, saving, adding, deleting, modifying, querying, etc. The specific implementation steps are as follows:

[0052] 1. Read the XML file and display it in QTreeView

[0053] Parse the XML file: First, use QDomDocument to read and parse the XML file.

[0054] Create a data model: Use QStandardItemModel as the data model for QTreeView. Convert the structure of the XML file into a tree structure and fill it into the data model.

[0055] Set the QTreeView model: Set the created QStandardItemModel as the model of QTreeView.

[0056] 2. Implement the functions of adding, deleting, modifying, and querying

[0057] Add Node: Select a node in the QTreeView and then add a new child node through the QStandardItemModel data model.

[0058] Delete Node: Delete a node by selecting the node and calling the removeRow method of the QStandardItemModel data model.

[0059] Modify Node: Trigger the modification operation by double-clicking on the node or using the right-click menu, and then update the corresponding node in the QStandardItemModel data model.

[0060] Query Node: Implement the query function for specific nodes through an input box or other controls.

[0061] 3. Save the modified XML file

[0062] Update QDomDocument: When the user modifies a node in the QTreeView, synchronously update the structure of the XML file in the QDomDocument.

[0063] Save to File: Use the QDomDocument::save() method to save the updated XML file.

[0064] Next, perform schema validation on the correctness of the above modified xml file. Schema validation includes two parts: XSD file definition and XML schema validation. XML schema validation means: Perform schema validation based on the XSD file pattern for the XML files generated by the message definition module and the module design module of the communication framework visualization design system. If the schema is correct, proceed with subsequent template code generation. If the schema is incorrect, modify the XML file according to the error prompt until the schema validation is correct.

[0065] XSD File Definition. Specifically, it defines the intelligent driving application communication message protocol and the communication structure of the intelligent driving application system among intelligent driving application algorithm modules, which is used to describe the publishing and subscribing relationships among machines, intelligent driving application algorithm modules, and the message transmission protocol among intelligent driving application algorithm modules. The XSD file definition includes basic data type definition, complex data type definition, combined structure type, machine type definition, publishing and subscribing type definition, and root element definition.

[0066] Basic Data Type Definition: Use <xs:simpletype>Define basic data types, such as boolType, uint8Type, etc. Each basic type is passed through <xs:restriction>Specify its base type (such as xs:boolean, xs:unsignedByte, etc.). These types provide basic data type constraints for elements in the XML document, ensuring data accuracy and consistency.

[0067] Complex data type definition: Use <xs:complextype>Define complex data types such as arrayType and vectorType. These types are passed through <xs:sequence>And <xs:element>Define its internal structure, allowing it to contain multiple child elements. "maxOccurs = \"unbounded\"" means that there can be multiple identical child elements, which is applicable to data structures such as arrays and vectors.

[0068] Composite structure type: Use <xs:complextype>Define a composite structure type that contains multiple fields (sub-elements), where each field can be a basic type or a complex type. This structure type is used to define more complex object structures and is suitable for scenarios that require combining multiple data fields.

[0069] Machine type definition: MachineType defines a machine type that contains a Modules element, which can contain multiple Module elements. Each Module element contains Publishers and Subscribers sub-elements, which are used to define publishers and subscribers respectively. Use <xs:extension>Extend PublisherType and SubscriberType to allow adding more attributes or elements on the basis of these types. This information specifies which topics the intelligent driving application algorithm module will publish to and which topics it will subscribe to.

[0070] Publication and subscription type definitions: Use PublisherType and SubscriberType to define the types of publication and subscription, including Topic and Message elements. These types describe two key pieces of information that must be included in each publish or subscribe operation: the topic name and the message type. Topic is used to specify the message topic, and Message is used to specify the message type (represented by xs:QName for qualified name).

[0071] Root element definition: Define Machines as the root element, which contains multiple Machine elements, and each Machine element is a machine instance. This structure allows defining multiple machines and their intelligent driving application algorithm modules and message publish-subscribe relationships in an XML document.

[0072] The implementation of the XML schema validation function adopts the QT-based XSD-driven XML file schema validation technology. The specific steps are as follows:

[0073] 1. Load the XSD file

[0074] Place the above-created XSD file in a fixed path and use the QXmlSchema class to load the XSD file. Check whether the XSD file is loaded successfully. If it fails, prompt to handle the loading error (such as file not found or incorrect format).

[0075] 2. Verify the validity of the XSD file

[0076] After loading the XSD file, verify its own validity to ensure it complies with the XML Schema specification. If the XSD file is invalid, prompt to repair or obtain a valid XSD file again.

[0077] 3. Create a validator

[0078] Create a QXmlSchemaValidator instance using the loaded QXmlSchema object. It is used to verify whether an XML file complies with the structure and constraints defined by the XSD.

[0079] 4. Verify the XML file

[0080] Use the validation method of QXmlSchemaValidator to validate the XML file loaded by the user. Check the validation result to determine whether the XML file complies with the structure and data type constraints defined by the XSD.

[0081] 5. Process the verification result

[0082] If the verification passes, it indicates that the XML file is valid and meets the requirements defined by the XSD. If the verification fails, prompt the user with the verification error message for the user to analyze the cause of the error and modify or adjust the XML file.

[0083] The application integration module generates the application communication framework code based on the XML files designed by the message definition module and the module design module, performs automated compilation for different platforms, performs automated deployment according to different machines, and realizes one-key remote startup of the application communication framework. A possible interface of the application integration module is as Figure 3 shown, providing operation buttons and a log display interface.

[0084] For the generation of the intelligent driving application communication framework code, specifically, it fills the code template file with the XML files generated by the message definition module and the XML files generated by the module design module designed in the above steps using Python, and generates a code file. The present invention generates a code file based on ROS2 as the communication library, and other communication libraries can also be used to generate code files. The specific steps for generating the intelligent driving application communication framework code are as follows:

[0085] 1. Parse two types of XML files (XML files generated by the message definition module and XML files generated by the module design module)

[0086] Use a Python XML parsing library (such as xml.etree.ElementTree or lxml) to read and parse the XML files, and extract relevant information required for code generation, such as message definition structure fields, intelligent driving application algorithm module names, publish-subscribe interface information, etc.

[0087] 2. Create a C++ code template file

[0088] Create a C++ code template file, usually in the format of a text file (such as.cpp or.h). The C++ code template file contains placeholders for inserting specific information parsed from each XML file, specifically the intelligent driving application algorithm module names and publish-subscribe interfaces and other information extracted from each XML file in the previous step. For example, generate a class definition template for each node, including necessary header files, node initialization code, callback functions for message publishing and subscribing, etc.

[0089] 3. Fill the placeholders in the C++ code template file

[0090] Use Python's string formatting methods (such as str.format() or f-strings) to fill the placeholder positions in the C++ code template file with the information parsed from each XML file.

[0091] 4. Generate ROS2 message definition files

[0092] Generate ROS2 message definition files with the extension.msg based on the message definition structure fields extracted from each XML file. These ROS2 message definition files define the format of the messages, including the field names and types of the messages.

[0093] 5. Generate CMakeLists.txt file

[0094] Generate the CMakeLists.txt file based on the generated C++ code template file and project dependencies. The CMakeLists.txt file should contain the necessary compilation instructions and dependencies for compilation in the ROS2 environment.

[0095] 6. Generate package.xml file

[0096] Generate the package.xml file based on the project information, including meta-information such as the name, version, and dependencies of the package. Use a Python script to generate the above content dynamically.

[0097] 7. Save the generated files

[0098] Save the generated C++ code template file, CMakeLists.txt file, and package.xml file to the specified directory for subsequent code compilation, deployment, and running.

[0099] Code compilation is to automate the compilation of the generated code module by module. Depending on the platform selected by the user through the interface, it is divided into x86 architecture compilation and cross-compilation. Use Python's XML parsing library to read the names of the modules to be compiled in the XML configuration file, and call the corresponding compilation script according to the architecture of the target machine to compile the extracted modules.

[0100] Deployment and operation involve uploading the compilation product of the above code to a specified machine according to the XML file obtained from the module design module. First, use the XML parsing library in Python to read the XML file obtained from the module design module, and generate the program startup script required for the corresponding machine according to the module name under different machines. Secondly, use the paramiko library in Python to create an SSH client object on the local machine, automatically add the SSH key of the remote host to avoid manual intervention, parse the IP address and port number of the machine specified in the XML file obtained from the module design module, establish an SSH connection with the remote machine, and transfer the compilation product and the program startup script to the specified path of the target machine through SSH. Operation is also achieved by creating an SSH client object to remotely connect to the above machine and execute the operation script.

[0101] The performance testing module monitors and displays the processes of the intelligent driving application communication framework started by the application integration module. First, set the communication resource occupancy thresholds, including the CPU resources, memory resources occupied by each process of the intelligent driving application communication framework during operation, and the communication delay of each communication link, etc. Mainly use Python scripts to extract and display the CPU and memory sizes occupied by each intelligent application algorithm module and the delay time of each communication link. A possible performance testing interface is as Figure 4 shown, with a threshold setting button and a display interface.

[0102] The specific steps for the performance testing module to monitor and display the processes of the intelligent driving application communication framework started by the application integration module are as follows:

[0103] 1. Establish an SSH connection

[0104] Use the paramiko library in Python to create an SSH client object on the local machine, automatically add the SSH key of the remote host to avoid manual intervention, parse the IP address and port number of the machine specified in the XML file, and establish an SSH connection with the remote machine.

[0105] 2. Execute the communication resource monitoring command

[0106] Use Python to call the system command of the remote machine (such as the ps command) to obtain the CPU and memory occupancy information of the specified process. Use the grep command to filter out the lines containing the target process name and exclude the lines containing the grep command itself to ensure accurate process information. Use Python to call the ROS2 command of the remote machine to obtain all communication links, and then call the ROS2 communication delay command to test each communication link one by one. Use the grep command to filter and extract specific delay data, including average delay, minimum delay, and maximum delay, etc.

[0107] 3. Parse and display data

[0108] Parse the obtained text data on the local machine, extract key information such as the CPU occupancy rate and memory occupancy rate of the process, as well as the latency data of each communication link. Display the parsed data in an easy-to-understand format and visualize it through a graphical interface. If it exceeds the set threshold, highlight it.

[0109] The user determines whether to adjust the designed intelligent driving application communication framework based on the communication resource occupancy results of the interface until the performance requirements are met.

[0110] According to the interface developed based on QT for the intelligent driving application communication framework described above, the overall process of the visualization design method of the intelligent driving application communication framework considering performance of the present invention is as Figure 5 shown. The first step is to define the message protocol based on the message definition interface, perform schema validation until the schema validation is correct, and generate an XML file. The second step is to design the publish-subscribe interfaces for each module under different machines. The message protocol used for publish-subscribe references the message protocol designed in the first step and performs schema validation until the schema validation is correct, and generates an XML file. The third step is to generate code based on the XML files designed in the first and second steps, select a platform, and perform automated compilation, deployment, and operation according to the selected different platforms. The fourth step is to set the communication resource occupancy threshold, remotely monitor and display the intelligent driving application algorithm module started in the third step. If it exceeds the threshold, re-perform the message protocol and module design in the first and second steps until the communication resource occupancy meets the conditions.

[0111] It can be seen that the present invention has the following characteristics:

[0112] 1. Visual design

[0113] Through the interface development based on QT, the visualization operations of message definition, module design, application integration, etc. of the intelligent driving application communication framework are realized, making the design process more intuitive and convenient, reducing the development difficulty, and improving the development efficiency.

[0114] 2. Flexibility of message definition and module design

[0115] The message definition module realizes the visual definition of message transmission between modules based on XML, supports the addition, deletion, modification, and query of multiple basic data types, complex data types, and combined structure types, as well as the import, schema validation, and saving of XML files. The module design module also realizes the visual design of intelligent driving application module communication under multiple machines based on XML, can describe the message transmission interfaces of modules under different machines, and also supports functions such as corresponding addition, deletion, modification, and query. This design method enables more flexible adjustment of message definition and module design when requirements or designs change, reduces cumbersome modification work, and avoids the accumulation of redundant code.

[0116] 3. Automated Integration and Deployment

[0117] The application integration module can generate the communication framework code of the intelligent driving application based on the XML files designed by the message definition module and the module design module, perform automated compilation for different platforms, perform automated deployment according to different machines, and also realizes remote startup of the application communication framework by pressing a button. This greatly improves the efficiency and accuracy of application integration, reduces the cumbersome nature of manual operations and the probability of errors.

[0118] 4. Performance Testing and Optimization

[0119] The performance testing module uses system tools and communication tools to monitor and record the system resources and communication links occupied by the application communication framework, and can confirm whether the established threshold is exceeded, thereby providing a basis for whether to redesign the intelligent driving application communication framework. This way of considering performance and conducting tests during the design process helps to detect performance problems in advance, make timely optimization adjustments, avoids the problem of difficulty in distinguishing the resource occupation of the communication framework and the application algorithm after the development of the application algorithm logic is completed, and improves the effect of performance optimization.

[0120] The above are only the preferred embodiments of the present invention. It should be noted that for those of ordinary skill in the art of this technology, without departing from the technical principle of the present invention, several improvements and deformations can be made, and these improvements and deformations should also be regarded as the protection scope of the present invention.< / xs:extension> < / xs:complextype> < / xs:element> < / xs:sequence> < / xs:complextype> < / xs:restriction> < / xs:simpletype>

Claims

1. A performance-based intelligent driving application communication framework visualization system design method, characterized in that: The intelligent driving application system includes an intelligent driving application communication framework and an intelligent driving application algorithm module; the intelligent driving application algorithm module implements the intelligent driving application algorithm, and the results calculated by each intelligent driving application algorithm module are transmitted through the communication middleware. The information transmission framework constructed during the transmission is the intelligent driving application communication framework; the visualization system is designed to include a message definition module, a module design module, an application integration module and a performance testing module; The message definition module is used to implement the visual design of the message protocol between the intelligent driving application algorithm modules based on XML, and generate an XML file for reference in the subsequent application integration module design; the message definition module supports the addition, deletion, modification and query of the message protocol, as well as the import, pattern verification and storage of XML files; The module design module is used to implement the visual design of communication between intelligent driving application algorithm modules under multiple machines based on XML, and generate XML files, specifically describing the message passing interface of the intelligent driving application algorithm modules under different machines, which is used for code generation, compilation, deployment and operation of subsequent application integration modules; the module design module supports the addition, deletion, modification and query of the message passing interface of the intelligent driving application algorithm module, as well as the import, pattern verification and storage of XML files; the machine refers to the domain controller where the intelligent driving application communication framework runs; The application integration module is used to generate the intelligent driving application communication framework code based on the XML files designed by the message definition module and the module design module, perform automatic compilation on different machines, perform automatic deployment according to different machines, and realize one-click remote startup of the intelligent driving application communication framework; The performance testing module is used to use system tools and communication tools to monitor and record the system resources and communication links occupied by the intelligent driving application communication framework to confirm whether the corresponding indicators exceed the established thresholds, thereby determining whether to redesign the intelligent driving application communication framework.

2. The method according to claim 1, characterized in that The message definition module implements the visual design of the message protocol between the intelligent driving application algorithm modules based on QT and XML technology; the visual design of the message protocol based on QT and XML technology includes XML file format definition, visual editing and mode verification; The XML file design format generated by the message definition module is as follows: It contains: Messages: as the root element, it contains all message definitions. Message has a name attribute to identify the message name. Field: is the field in the message, it has name and type attributes to identify the field name and data type.

3. The method according to claim 1, characterized in that The module design module is based on QT to visually design the publish-subscribe interface of the intelligent driving application algorithm module using XML, which is used to describe the message communication link between different intelligent driving application algorithm modules of different machines. In the process of visual design of the publish-subscribe interface, the publish-subscribe interface is first defined, and the message protocol used by the publish-subscribe interface is defined to refer to the message protocol in the XML file generated by the message definition module; Visual design implementation based on QT and XML technology includes XML file format definition, visual editing and mode verification; The XML file design format generated by the module design module is as follows: It includes: Machines: as the root element, it represents the entire intelligent driving application system, which includes multiple machines; Machine elements: including ID attributes, IP elements, Port elements and Modules elements. The ID attribute uniquely identifies each machine, the IP element identifies the IP of each machine, the Port element identifies the port number that can remotely connect to the machine, and the Modules element contains all modules in the machine; each Module module is identified by its name attribute, and is further subdivided into two sub-elements, Publishers and Subscribers; the Publishers sub-element contains multiple Publisher elements, each of which contains two elements, Topic and Message, which are used to specify the published topic name and message type respectively; the Subscribers sub-element contains multiple Subscriber elements, each of which also contains two elements, Topic and Message, which are used to define the subscribed topic name and message type.

4. The method according to claim 1, characterized in that The message protocol module and the module design module both implement the visual editing function of the XML files generated by each based on QT. The visual editing functions include reading, saving, adding, deleting, modifying, and querying. The specific implementation steps are as follows: 1) Read the XML file and display it in QTreeView Parsing XML files: First use QDomDocument to read and parse the XML file; Create a data model: Use QStandardItemModel as the data model of QTreeView; convert the structure of the XML file into a tree structure and fill it into the data model; Set the QTreeView model: Set the created QStandardItemModel as the model of QTreeView; 2) Implement the functions of adding, deleting, modifying and checking Adding nodes: Select a node in QTreeView and then add a new child node through the QStandardItemModel data model; Delete a node: Delete the node by selecting the node and calling the removeRow method of the QStandardItemModel data model; Modify a node: trigger the modification operation by double-clicking the node or using the right-click menu, and then update the corresponding node in the QStandardItemModel data model; Query node: Use input boxes or other controls to query specific nodes; 3) Save the modified XML file Update QDomDocument: When the user modifies a node in QTreeView, the structure of the XML file in QDomDocument is updated synchronously; Save to file: Use the QDomDocument::save() method to save the updated XML file.

5. The method according to claim 1, characterized in that The message protocol module and the module design module both perform schema verification on the correctness of the modified XML file. Schema verification includes two parts: XSD file definition and XML schema verification. XML schema verification means: verifying the XML file based on the XSD file schema. If the schema is correct, then generate the subsequent template code. If the schema is wrong, modify the XML file according to the error prompt until the schema verification is correct. XSD file definitions are used to describe machines, intelligent driving application algorithm modules, the publishing and subscription relationships between intelligent driving application algorithm modules, and the message protocols between intelligent driving application algorithm modules; XSD file definitions include basic data type definitions, complex data type definitions, composite structure types, machine type definitions, publishing and subscription type definitions, and root element definitions; Basic data type definition: Use <xs:simpletype>Defines basic data types; each basic type is defined by <xs:restriction> Specify its base types, which provide basic data type constraints for elements in XML documents;< / xs:restriction> < / xs:simpletype> Complex data type definition: Use <xs:complextype>Define complex data types. These complex data types are defined by <xs:sequence>and <xs:element> Defines its internal structure, allowing multiple sub-elements; maxOccurs = "unbounded" means there are multiple identical sub-elements, applicable to array and vector data structures;< / xs:element> < / xs:sequence> < / xs:complextype> Combining structure types: Use <xs:complextype> Define a composite structure type, which contains multiple fields, each of which is a basic type or a complex type; this composite structure type is used to define an object structure that is more complex than a complex data type;< / xs:complextype> Machine type definition: MachineType defines a machine type, including the Modules element, which contains multiple Module elements; each Module element contains Publishers and Subscribers sub-elements, which are used to define publishers and subscribers respectively; use <xs:extension> Extending PublisherType and SubscriberType allows adding more attributes or elements based on these machine types. This information specifies which topics the intelligent driving application algorithm module will publish to and subscribe to.< / xs:extension> Publishing and subscription type definition: Use PublisherType and SubscriberType to define publishing and subscription types, including Topic and Message elements; These publish and subscribe types describe two key pieces of information that each publish or subscribe operation contains: the topic name and the message type; The Topic element is used to specify the message topic, and the Message element is used to specify the message type; Root element definition: Define Machines as the root element, which contains multiple Machine elements. Each Machine element is a machine instance. This structure allows multiple machines and their intelligent driving application algorithm modules to be defined in an XML document, as well as message publishing and subscription relationships. The XML schema validation function is implemented using QT-based XSD-driven XML file schema validation technology. The specific steps are as follows: 1) Load the XSD file Place the created XSD file in a fixed path and use the QXmlSchema class to load the XSD file; check whether the XSD file is loaded successfully. If it fails, prompt to handle the loading error; 2) Verify the validity of the XSD file After loading the XSD file, verify whether it is valid and ensure that it complies with the XML Schema specification. If the XSD file is invalid, prompt to repair or re-obtain a valid XSD file; 3) Create a validator Use the loaded QXmlSchema object to create a QXmlSchemaValidator instance to verify that the XML file conforms to the structure and constraints defined by the XSD; 4) Verify the XML file Use the validation method of QXmlSchemaValidator to validate the XML file loaded by the user, check the validation results, and determine whether the XML file conforms to the structure and data type constraints defined by XSD; 5) Process verification results If the validation passes, it means that the XML file is valid and meets the requirements defined by XSD; if the validation fails, the user is prompted with validation error information.

6. The method according to claim 1, characterized in that The application integration module generates the intelligent driving application communication framework code by filling the code template file with the XML file generated by the message definition module and the XML file generated by the module design module based on Python to generate the code file. The specific steps of generating the intelligent driving application communication framework code are as follows: 1) Parse two types of XML files Use Python's XML parsing library to read and parse XML files and extract relevant information that needs to be generated code, including message definition structure fields, intelligent driving application algorithm module name, and publish-subscribe interface information; 2) Create a C++ code template file Create a C++ code template file in text file format. The C++ code template file contains placeholders for inserting specific information parsed from each XML file, specifically the name of the intelligent driving application algorithm module and the publish-subscribe interface information extracted from each XML file. 3) Fill in the placeholders in the C++ code template file Use Python's string formatting method to fill the information parsed from each XML file into the placeholder position in the C++ code template file; 4) Generate CMakeLists.txt file Generate a CMakeLists.txt file based on the generated C++ code template file and project dependencies. The CMakeLists.txt file contains the necessary compilation instructions and dependencies for compilation in the ROS2 environment. 5) Generate package.xml file Generate a package.xml file based on the project information, including meta-information such as package name, version, and dependencies; 6) Save the generated file Save the generated C++ code template file, CMakeLists.txt file, and package.xml file to the specified directory to facilitate the subsequent code compilation, deployment, and operation of the intelligent driving application algorithm module; Code compilation is to automatically compile the generated code for each intelligent driving application algorithm module one by one; use Python's XML parsing library to read the module name to be compiled in the XML configuration file, and call the corresponding compilation script according to the architecture of the target machine to compile the extracted intelligent driving application algorithm module; Deployment and operation are to upload the code compilation product to the specified machine according to the XML file obtained by the module design module; first, use Python's XML parsing library to read the XML file obtained by the module design module, and generate the program startup script required by the corresponding machine according to the module name under different machines; secondly, use Python's paramiko library to create an SSH client object on the local machine, automatically add the SSH key of the remote host, parse the machine IP address and port number specified in the XML file obtained by the module design module, establish an SSH connection with the remote machine, and transfer the compiled product and the program startup script to the specified path of the target machine through SSH; operation is to remotely connect to the machine by creating an SSH client object and execute the run script.

7. The method according to claim 1, characterized in that The performance testing module monitors and displays the process of the intelligent driving application communication framework started by the application integration module; first, the communication resource occupancy threshold is set, including the CPU resources, memory resources and communication delay of each communication link occupied by each process of the intelligent driving application communication framework during operation; the Python script is used to extract and display the CPU and memory size occupied by each intelligent application algorithm module and the delay time of each communication link.

8. The method according to claim 1, characterized in that The specific steps for the performance testing module to monitor and display the process of the intelligent driving application communication framework started by the application integration module are as follows: 1) Establish an SSH connection Use Python's paramiko library to create an SSH client object on the local machine, automatically add the remote host's SSH key, parse the machine IP address and port number specified in the XML file, and establish an SSH connection with the remote machine; 2) Execute the communication resource monitoring command Use Python to call the system command of the remote machine to obtain the CPU and memory usage information of the specified process, use the grep command to filter out the line containing the target process name, and exclude the line containing the grep command itself; use Python to call the ROS2 command of the remote machine to obtain all communication links, get all communication links, and then call the ROS2 communication delay command to test the communication links one by one, and use the grep command to filter and extract specific delay data, including average delay, minimum delay and maximum delay; 3) Analyze and display data Parse the acquired text data on the local machine to extract the CPU usage and memory usage data of the process, as well as the delay data of each communication link; display the extracted data in a certain format and visualize it through a graphical interface. If it exceeds the set threshold, it will be highlighted.

9. A smart driving application communication framework visualization system designed and implemented based on the method described in any one of claims 1 to 8.

10. A method for visual design of intelligent driving application communication framework implemented by the system as claimed in claim 9, characterized in that: The following steps are involved: The first step is to define the message protocol based on the message definition interface, perform schema verification until the schema verification is correct, and generate an XML file; The second step is to design the publish-subscribe interface for each module on different machines. The message protocol used for publish-subscribe refers to the message protocol designed in the first step, and the schema is verified until the schema is verified correctly and an XML file is generated. The third step is to generate code based on the XML files designed in the first and second steps, select the platform, and automatically compile, deploy, and run the intelligent driving application algorithm module based on the selected platforms; The fourth step is to set the communication resource occupancy threshold and remotely monitor and display the intelligent driving application algorithm module started in the third step. If the threshold is exceeded, the message protocol and module design of the first and second steps are repeated until the communication resource occupancy meets the conditions.

Images