Sensor manual automatic parsing and ros2 driving generation system and method

Abstract

The application discloses a kind of sensor manual automatic analysis and ROS2 generation system and method, to sensor PDF manual as input, by AI multimodal analysis and integrity evaluation, extract three elements and complete credibility score;Low credibility manual is carried out to multi-source information completion;Generation standardized format structured protocol intermediate representation IR, after checking, automatically generate ROS2 drive code package, complete compilation verification and AI code repair;Start hardware-independent simulation verification closed loop, carry out multidimensional automatic verification and output structured report;For manual version iteration, by IR difference comparison realizes incremental update and incremental verification.The application can shorten the development cycle of drive, adapt to incomplete manual scene, guarantee the quality of drive, reduce version iteration cost, suitable for laser radar, camera, IMU and other sensor ROS2 drive automation development.

Description

Technical Field

[0001] This invention relates to ROS2 driver development technology, and more particularly to a system and method for automatic parsing of sensor manuals and generation of ROS2 drivers. Background Technology

[0002] Sensors are core sensing components in intelligent systems such as autonomous driving and robotics. Common types include LiDAR, cameras, IMU (Inertial Measurement Unit), millimeter-wave radar, and GNSS. In ROS2 (Robot Operating System 2)-based system development, all of these sensors need to interact with upper-layer applications through dedicated drivers.

[0003] However, the existing development model for ROS2-driven sensors has the following problems: Manually reading manuals is inefficient and time-consuming. Traditional processes require engineers to manually read PDF technical manuals provided by sensor manufacturers, extracting communication protocol information from hundreds of pages of unstructured content and translating it into programmable interface specifications. For large-scale autonomous driving platforms involving dozens of sensors, the development cycle often lasts for weeks, severely limiting system integration efficiency.

[0004] Protocol extraction is highly dependent on engineer experience, and there is a lack of effective means to handle incomplete information. Some sensor manuals have issues such as ambiguous field semantics, missing communication parameters, and incomplete data frame definitions. Engineers can only rely on personal experience to make subjective inferences, and there is no systematic method to complete the information, which makes it difficult to guarantee the quality of the drive.

[0005] The lack of automated verification capabilities after code generation necessitates manual integration and testing. After the driver code is written, engineers need to connect the sensor hardware to the ROS2 system for manual integration and verification, which is time-consuming and highly dependent on the hardware environment. Code defects cannot be detected in advance if the hardware is not available, resulting in delayed problem exposure and high repair costs.

[0006] When the manual version is updated, a complete redevelopment is required, lacking incremental update capabilities. When the sensor firmware is upgraded, the manufacturer releases a new version of the manual, and engineers must reread the manual and completely modify and re-verify the driver code, making it impossible to reuse verified modules, resulting in duplicated work and quality risks. Summary of the Invention

[0007] To address the shortcomings of existing technologies, the present invention aims to provide a system and method for automatic parsing of sensor manuals and generation of ROS2 drivers.

[0008] To achieve the objectives of this invention, the technical solution adopted is as follows: A method for automatically parsing sensor manuals and generating ROS2 drivers includes the following steps: (1) Perform AI multimodal analysis and integrity assessment on the sensor PDF manual, extract the three major elements of interface type, data frame format and data semantics, and score the credibility, and output the credibility level judgment result; (2) Perform multi-source information completion on medium / low confidence manuals. Complete missing elements through three channels: searching the same manufacturer's manual library, referencing the open-source ROS2 driver library, and precise follow-up questions from users. The completion results are provided to users for review after being weighted by confidence. (3) Generate a standardized structured protocol intermediate representation (IR), and output the IR file after syntax and semantic verification; (4) Automatically generate ROS2 driver code package using IR as input; (5) Start a hardware-independent simulation verification closed loop, output a structured verification report and support automatic repair and re-verification; (6) Perform incremental updates for each version of the manual, identify changes by comparing the differences between the old and new IRs, and perform incremental generation and verification of the change-related code and test cases.

[0009] Furthermore, in step (1), the three main elements specifically include: Interface type: including physical communication interface and its configuration parameters; Data frame format: including frame header identifier, data field definition, field order, data length, and verification method; Data semantics: including the physical meaning of the fields, engineering units, numerical range, coordinate system definition, and data update frequency.

[0010] Furthermore, in step (1), the credibility score divides the manual into: ≥85 points is high credibility, and it directly enters IR generation; 50~84 points is medium credibility, and multi-source information completion is triggered; <50 points is low credibility, and generation is paused and core information is missing.

[0011] Furthermore, in step (3), the IR is stored in YAML / JSON format and contains the following core field groups: Communication interface type and parameter configuration; complete definition of data frame, frame header, field offset / length / type / byte order, and verification algorithm; field semantics, physical meaning, engineering units, numerical precision, and effective range; sensor data release frequency; recommended ROS2 QoS configuration, reliability strategy, and historical depth; data coordinate system definition and default transformation relationship.

[0012] Further, in step (4), the complete ROS2 driver package generation steps are as follows: The first step is to select the most suitable ROS2 standard message type based on the semantic description of each field in the IR. The second step involves the system generating standard ROS2 node core code, including node class definition, communication interface initialization code, data frame parsing logic, ROS2 publisher creation, and ROS2 parameter declaration. The third step is for the system to generate a ROS2 startup file in Python format, which includes node startup configuration, default parameter value settings, and parameter file loading logic. The fourth step is for the system to generate a build file that conforms to the specifications, containing correct dependency declarations, compilation target definitions, and installation rules; The fifth step is to compile and verify the generated code package using colcon build. If the compilation fails, the AI ​​code repairer will automatically analyze the errors and repair and recompile until the compilation is successful or the maximum number of retries is reached, at which point a failure report will be output.

[0013] Furthermore, in step (5), the verification process includes the following steps: Automatic test node generation: The system automatically generates ROS2 test nodes based on the frame format and data frequency definition in IR to simulate the actual data transmission behavior of the sensor; ROS2 simulation environment startup: Start the test node and the driver node under test simultaneously without relying on real hardware to build a complete end-to-end data link. Multi-dimensional verification execution: topic posting frequency verification, message format verification, timestamp correctness verification, and data value range verification; Generate a structured validation report: Summarize the validation results and generate a report in JSON / HTML format.

[0014] Furthermore, in step (6), the incremental update will be classified into four categories: interface parameters, frame format, semantics, and frequency / QoS, and only the modules affected by the regeneration and reverification will be regenerated.

[0015] A sensor manual automatic parsing and ROS2 driver generation system, comprising: The AI ​​multimodal parsing and integrity assessment module is used to extract and parse multimodal data from sensor PDF manuals, detect the three main elements, and output the confidence level. The multi-source information completion module is used to perform three-way completion, confidence fusion, and user verification on medium / low confidence manuals. The structured protocol intermediate representation IR generation module is used to generate and verify standardized IR files, and supports manual editing; The ROS2 driver code automatic generation module is used to generate a complete driver package based on IR and complete compilation verification and code repair. The simulation verification closed-loop module is used for hardware-free simulation testing, multi-dimensional automatic verification, and report generation. The incremental update module is used for IR difference comparison, change identification, incremental code generation, and incremental verification.

[0016] The beneficial effects of the present invention are as follows: Compared with the prior art, the present invention has the following advantages: This invention effectively handles incomplete manual scenarios, with a multi-source completion mechanism covering common information loss situations. Through a three-pronged parallel completion strategy—searching from the same vendor's knowledge base, referencing open-source drivers, and responding to precise user inquiries—the system expands its processing range from complete manuals to those containing partial information, significantly improving engineering applicability.

[0017] The driver code of this invention is of guaranteed quality, and simulation verification automatically covers core functionalities. It performs automated testing on core indicators such as topic frequency, message format, timestamp correctness, and data value range, generating structured verification reports. This makes driver quality quantifiable and traceable, reducing the problem discovery rate during the hardware integration phase from the source.

[0018] This invention offers low version iteration costs, with incremental update mechanisms modifying only the affected modules. For version upgrade scenarios involving only minor field changes, the update workload can be reduced while preserving the stability of verified modules.

[0019] This invention introduces a structured intermediate representation, improving system maintainability and scalability. The IR, as a standard interface, allows for independent evolution of the manual parsing and code generation phases. Engineers can manually review and correct the IR when necessary, providing a means of human intervention while maintaining automation efficiency.

[0020] This invention establishes system reliability boundaries through confidence scoring, avoiding the generation of low-quality driver code under conditions of insufficient information. Instead of silently outputting questionable results, the system proactively informs the user when confidence is insufficient, effectively preventing hidden quality risks arising from incomplete manual interpretation.

[0021] This invention significantly improves driver development efficiency, reducing the time from manual input to generating runnable driver code from several days / weeks to minutes. For large-scale autonomous driving projects that require the integration of multiple sensors, the overall development cycle compression effect is particularly significant. Attached Figure Description

[0022] Figure 1 This is a schematic diagram of a sensor manual automatic parsing and ROS2 driver generation method described in this invention. Detailed Implementation

[0023] The technical solution of the present invention will be further described below with reference to the accompanying drawings and embodiments. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and should not be used to limit the scope of protection of this application.

[0024] like Figure 1 As shown, the sensor manual automatic parsing and ROS2 driver generation method of the present invention takes a sensor PDF manual containing communication protocol description as input, and processes it through core modules such as AI multimodal parsing and integrity assessment, multi-source information completion, structured protocol intermediate representation (IR) generation, ROS2 driver code automatic generation, simulation verification closed loop and incremental update, and finally outputs a driver code package and structured verification report that can be compiled and run in the ROS2 environment.

[0025] Specifically, the following steps are included: (1) Perform AI multimodal analysis and integrity assessment on the sensor PDF manual; After receiving the sensor's PDF manual, the system calls the AI ​​multimodal parsing engine to extract the full text content, supporting the recognition and parsing of various content formats such as text paragraphs, data tables, and protocol frame format diagrams. The AI ​​multimodal parsing engine is based on a large language model (LLM) and has the ability to understand natural language protocol descriptions, extracting structured communication parameter information from narrative paragraphs.

[0026] After content extraction is completed, the system performs item-by-item checks on the three essential elements for driver generation: ① Interface Type: including physical communication interface (USB, UART / serial port, UDP / Ethernet, CAN bus, etc.) and its configuration parameters; ② Frame Format: including frame header identifier, data field definition, field order, data length, verification method, etc.; ③ Data Semantics: including the physical meaning of fields, engineering units, numerical range, coordinate system definition, data update frequency, etc.

[0027] The system quantifies and scores the three major elements, and outputs a comprehensive credibility score after weighted summation. The manual is divided into three levels: high credibility (≥85 points, directly enters IR generation); medium credibility (50-84 points, triggers multi-source information completion); low credibility (<50 points, informs the user that core information is seriously missing, and suspends automatic generation).

[0028] (2) Automatically perform multi-source information completion on medium-confidence manuals, or perform multi-source information completion on low-confidence manuals after user confirmation; systematically obtain missing protocol elements from multiple external information sources; After the integrity assessment module outputs a list of missing elements, the multi-source information completion module initiates three completion paths in parallel for each missing item: Path 1: Searching the same manufacturer's manual database and inferring protocol patterns. The system searches for similar protocol descriptions in the manuals of other models from the same manufacturer. Leveraging the characteristic that products from the same manufacturer usually follow similar communication design specifications, it infers the missing information of the current model from the protocols of known models and marks the confidence level.

[0029] Path Two: Reference to Open-Source ROS2 Driver Libraries. The system integrates the ability to search open-source code repositories (GitHub, ROS Index, etc.), searching for existing implementations for similar or model sensors in the open-source ROS2 driver code repositories. By parsing the frame format definitions, message type mappings, and interface configurations in the open-source driver code, it extracts protocol information that can be used for reference.

[0030] Path 3: Targeted follow-up questions to users regarding missing items. For missing elements that cannot be automatically filled in through the first two paths, the system generates precise user prompts, focusing only on the truly missing information items. These prompts are presented in a structured question format, clearly explaining the function and expected format of each piece of information.

[0031] After the completion results from the three paths are weighted and fused based on confidence levels, each completion information carries an inference source label (vendor knowledge base / open source driver / user input). The system provides a user review interface, where users can confirm or correct each completion result before proceeding to the IR generation stage.

[0032] (3) Generate a structured protocol intermediate representation (IR), a standardized sensor communication protocol intermediate representation (IR) format, as a decoupling layer between manual parsing and driver code generation; IR uses YAML / JSON format for storage and includes the following core field groups: ① interface_type (communication interface type and parameter configuration); ② frame_format (complete definition of data frame: frame header, field offset / length / type / byte order, verification algorithm); ③ field_semantics (field semantics: physical meaning, engineering units, numerical precision, and effective range); ④ data_frequency (sensor data transmission frequency, Hz); ⑤ qos_profile (recommended ROS2 QoS configuration: reliability policy, historical depth, etc.); ⑥ coordinate_frame (data coordinate system definition and default transformation relationship).

[0033] After IR generation, the IR validator checks the results for syntax correctness and semantic consistency. If the validation passes, an IR file is output; if validation fails, a detailed error description is returned. The IR file allows engineers to manually edit and correct it (such as adjusting field types or adding missing fields). Corrections directly trigger the code generation process without requiring re-parsing of the manual, improving system flexibility and project usability.

[0034] (4) Automatic generation of ROS2 driver code: Based on structured IR, standard ROS2 driver code is generated automatically, and a complete driver package conforming to the ROS2 ecosystem specification is output. The ROS2 driver code generation module takes the IR file as its sole input and generates a complete ROS2 driver package by following these steps: The first step is automatic message type matching. Based on the semantic descriptions of each field in the IR (Information Relationship), the system automatically selects the most suitable ROS2 standard message type: Point cloud / distance data → sensor_msgs / PointCloud2 (sensor message / point cloud data); Image → sensor_msgs / Image (sensor message / image data); IMU data → sensor_msgs / Imu (sensor message / inertial measurement data); GNSS (Global Navigation Satellite System) positioning → sensor_msgs / NavSatFix (sensor message / satellite positioning data); Custom data is automatically generated into a .msg definition file.

[0035] The second step is the generation of ROS2 node core code. The system generates a standard rclcpp (C++) or rclpy (Python) node implementation, including node class definition, communication interface initialization code, data frame parsing logic, ROS2 publisher creation (topic name / message type / QoS are all from IR), and ROS2 parameter declaration.

[0036] The third step is startup file generation. The system generates a ROS2 startup file in Python format, which includes node startup configuration, default parameter value settings, and parameter file loading logic.

[0037] The fourth step is build file generation. The system generates CMakeLists.txt (CMake build configuration file) and package.xml (package description file) that conform to the aament_cmake or aament_python specification, containing correct dependency declarations, build target definitions, and installation rules.

[0038] Step 5: Compilation and Verification. The generated code package is compiled and verified using colcon build; if compilation fails, the AI ​​code repairer automatically analyzes the errors and repairs and recompiles until compilation succeeds or the maximum number of retries is reached, at which point a failure report is output.

[0039] (5) Simulation verification closed loop: After the driver code is generated, an automated simulation verification closed loop is introduced, which can verify the functional correctness of the driver code without relying on physical sensors. The verification process includes the following steps: ① Automatic test node generation: Based on the frame format and data frequency definition in the IR, the system automatically generates ROS2 test nodes (Test Publisher Nodes) to simulate the actual data publishing behavior of the sensor, covering various test scenarios such as normal data, boundary value data, and abnormal data. ② ROS2 simulation environment startup: Without relying on real hardware, the test nodes (data publishers) and the tested driver nodes (data receivers and processors) are started simultaneously to build a complete end-to-end data link. ③ Multi-dimensional verification execution: Verification of topic publishing frequency (actual Hz vs. expected Hz), message format verification (field integrity / type matching), timestamp correctness verification (header.stamp continuity and monotonically increasing), and data value range verification (range / precision / unit). ④ Generation of structured verification report: The verification results are summarized and a JSON / HTML format report is generated, including the PASS / FAIL / WARN status of each item and repair suggestions.

[0040] If any verification fails, the system will report the failure to the code generation module, triggering an automatic repair process and re-verification until all core verification items pass or the final problem list is output.

[0041] (6) Design an incremental update mechanism based on IR difference comparison for the sensor manual version iteration scenario; When a sensor manufacturer releases a new version of the manual, the system parses the new manual and generates an IR file to obtain the new version of the IR file. It then performs a field-by-field structured difference comparison (diff) with the old version of the IR file, identifies all changed fields and categorizes them as follows: interface parameter changes (affecting communication initialization code), frame format changes (affecting data frame parsing logic), semantic changes (affecting message type filling logic), and frequency / QoS changes (affecting publisher configuration).

[0042] Based on the difference comparison results, the code generation module only regenerates the code modules containing the changed fields, while the remaining modules that have passed simulation verification remain unchanged. The simulation verification module also adopts an incremental verification strategy, only rerunning the test cases related to the changes.

[0043] This invention also provides an automatic sensor manual parsing and ROS2 driver generation system, including an AI multimodal parsing and integrity assessment module, a multi-source information completion module, a structured protocol intermediate representation (IR) generation module, a ROS2 driver code automatic generation module, a simulation verification closed-loop module, and an incremental update module. The modules work together to complete the fully automated processing from inputting the sensor PDF manual to outputting the ROS2 driver code package and verification report.

[0044] The AI ​​multimodal parsing and integrity assessment module receives sensor PDF manuals, performs multimodal extraction and parsing on text, tables, and protocol frame diagrams, detects three core elements—communication interface, data frame format, and data semantics—and performs credibility scoring, outputting high / medium / low-level judgment results. The multi-source information completion module completes missing elements in medium / low confidence manuals through three channels: searching the same manufacturer's manual library, referencing the open-source ROS2 driver library, and precise follow-up questions from users. It weights and merges the confidence of the completed information and marks the source, supporting user review and correction. The structured protocol intermediate representation (IR) generation module generates standardized IRs in YAML / JSON format based on complete protocol information, performs syntax correctness and semantic consistency checks on the IRs, and supports manual editing and correction. The ROS2 driver code automatic generation module takes IR as input and automatically completes ROS2 standard message matching, driver node code generation, Launch file generation, build file generation, and performs ament compilation verification and AI code repair, outputting a driver package that conforms to the ROS2 specification. The simulation verification closed-loop module automatically generates test nodes for simulated sensor data, builds a hardware-free ROS2 simulation environment, performs multi-dimensional automatic verification of topic frequency, message format, timestamp, and data value range, generates a structured verification report, and supports automatic repair and re-verification. The incremental update module generates a new IR for the new version of the manual and compares the differences with the old version of the IR. It identifies changes in interface, frame format, semantics, and frequency / QoS, and performs incremental generation and incremental verification only on the code and test cases related to the changes.

[0045] The applicant of this invention has provided a detailed description of the embodiments of the invention in conjunction with the accompanying drawings. However, those skilled in the art should understand that the above embodiments are merely preferred embodiments of the invention. The detailed description is only intended to help readers better understand the spirit of the invention and is not intended to limit the scope of protection of the invention. On the contrary, any improvements or modifications made based on the inventive spirit of the invention should fall within the scope of protection of the invention.

Claims

1. A method for automatic parsing of sensor manuals and generation of ROS2 drivers, characterized in that, Includes the following steps: (1) Perform AI multimodal analysis and integrity assessment on the sensor PDF manual, extract the three major elements of interface type, data frame format and data semantics, and score the credibility, and output the credibility level judgment result; (2) Perform multi-source information completion on medium / low confidence manuals. Complete missing elements through three channels: searching the same manufacturer's manual library, referencing the open-source ROS2 driver library, and precise follow-up questions from users. The completion results are provided to users for review after being weighted by confidence. (3) Generate a standardized structured protocol intermediate representation (IR), and output the IR file after syntax and semantic verification; (4) Automatically generate ROS2 driver code package using IR file as input; Complete steps for generating a ROS2 driver package: The first step is to select the most suitable ROS2 standard message type based on the semantic description of each field in the IR. The second step involves the system generating standard ROS2 node core code, including node class definition, communication interface initialization code, data frame parsing logic, ROS2 publisher creation, and ROS2 parameter declaration. The third step is for the system to generate a ROS2 startup file in Python format, which includes node startup configuration, default parameter value settings, and parameter file loading logic. The fourth step is for the system to generate a build file that conforms to the specifications, containing correct dependency declarations, compilation target definitions, and installation rules; The fifth step is to compile and verify the generated code package using colcon build; if the compilation fails, the AI ​​code repairer will automatically analyze the errors and repair and recompile until the compilation is successful or the maximum number of retries is reached, and then output a failure report. (5) Start a hardware-independent simulation verification closed loop, output a structured verification report and support automatic repair and re-verification; (6) Perform incremental updates for each version of the manual, identify changes by comparing the differences between the old and new IRs, and perform incremental generation and verification of the change-related code and test cases.

2. The method for automatic parsing of sensor manuals and generation of ROS2 drivers according to claim 1, characterized in that, In step (1), the three main elements specifically include: Interface type: including physical communication interface and its configuration parameters; Data frame format: including frame header identifier, data field definition, field order, data length, and verification method; Data semantics: including the physical meaning of the fields, engineering units, numerical range, coordinate system definition, and data update frequency.

3. The method for automatic parsing of sensor manuals and generation of ROS2 drivers according to claim 1, characterized in that, In step (1), the credibility score divides the manual into: ≥85 points is high credibility, and it directly enters IR generation; 50~84 points is medium credibility, and multi-source information completion is triggered; <50 points is low credibility, and generation is paused and core information is missing.

4. The method for automatic parsing of sensor manuals and generation of ROS2 drivers according to claim 1, characterized in that, In step (3), the IR is stored in YAML / JSON format and contains the following core field groups: Communication interface type and parameter configuration; complete definition of data frame, frame header, field offset / length / type / byte order, verification algorithm; field semantics, physical meaning, engineering units, numerical precision and effective range; sensor data transmission frequency; Recommended ROS2 QoS configuration, reliability policy, historical depth; data coordinate system definition and default transformation relationship.

5. The method for automatic parsing of sensor manuals and generation of ROS2 drivers according to claim 1, characterized in that, In step (5), the verification process includes the following steps: Automatic test node generation: The system automatically generates ROS2 test nodes based on the frame format and data frequency definition in IR to simulate the actual data transmission behavior of the sensor; ROS2 simulation environment startup: Start the test node and the driver node under test simultaneously without relying on real hardware to build a complete end-to-end data link. Multi-dimensional verification execution: topic posting frequency verification, message format verification, timestamp correctness verification, and data value range verification; Generate a structured validation report: Summarize the validation results and generate a report in JSON / HTML format.

6. The method for automatic parsing of sensor manuals and generation of ROS2 drivers according to claim 1, characterized in that, In step (6), the incremental update will be classified into four categories: interface parameters, frame format, semantics, and frequency / QoS. Only the modules that are regenerated and re-verified will be affected.

Images