Intelligent networked automobile multi-mode perception and simulation engineering integrated practical training equipment

Abstract

The invention discloses intelligent networked automobile multi-mode perception and simulation engineering integrated practical training equipment, and aims to solve the defects of an existing practical training platform in the aspects of expansibility, operation convenience, multi-sensor fusion and the like. Based on Linux and ROS2, efficient acquisition, transmission and visual display of sensor data are realized. Based on Linux open source ecology and ROS2 open architecture, the system has high expansibility and compatibility, and facilitates third-party software and algorithm access and sensor module upgrading. A simple operation interface is developed based on ROS2, and usability is improved. The STM32 single-chip microcomputer and the motor controller are utilized to accurately control the movement of the wheels of the 3D vehicle model. And a multi-sensor fusion algorithm is adopted, so that the environment perception capability is enhanced. The practical training rack effectively overcomes the defects of existing equipment and provides powerful support for intelligent network connection automobile teaching.

Description

Technical field

[0001] The present invention relates to the field of intelligent networked environment perception sensor assembly and adjustment and educational equipment. Background Art

[0002] Most existing training platforms for intelligent connected environmental perception sensors focus on a single sensor (such as ultrasonic radar or lidar) in their functional design, resulting in insufficient functional integration. This makes it difficult to fully simulate the collaborative working process of multi-sensor fusion and complex environmental perception in real autonomous driving scenarios.

[0003] Intelligent connected vehicle technology is constantly developing, and new sensors and technologies are constantly emerging. Existing training platforms are often difficult to upgrade and expand, and cannot adapt to new sensors and software systems.

[0004] The operating interface design of existing training platforms lacks systematic and user-friendly considerations. The operational process is complex, and the lack of a simple and easy-to-use interface increases the difficulty for students to get started, affecting teaching efficiency and learning experience. These issues restrict the effectiveness of training platforms in the practical application of intelligent connected vehicle teaching. Summary of the Invention

[0005] In view of the many shortcomings of the existing intelligent networked environmental perception sensor training platform, the present invention aims to invent an intelligent networked environmental perception sensor training platform based on Linux and ROS2 systems with more comprehensive functions and superior performance to solve related technical problems.

[0006] Application of multi-sensor fusion algorithms: Integrates multiple environmental perception sensors, including lidar, millimeter-wave radar, cameras, ultrasonic radar, and inertial measurement unit systems. Advanced multi-sensor fusion algorithms are used to fuse and process data collected by multiple sensors, including lidar, millimeter-wave radar, and cameras.

[0007] Convenient operation process: Based on ROS2 development, the concise and clear operation interface is designed in accordance with ergonomic principles, with reasonable functional layout and clear operation instructions.

[0008] Data processing and diversified display: With the help of ROS2's rich command line tools and graphical interface, complex sensor data can be presented in an intuitive and easy-to-understand way.

[0009] Vehicle model motion control: The STM32 microcontroller is introduced as the control core, combined with a motor controller to achieve precise control of the wheel motion of the 3D vehicle model.

[0010] Compared with the prior art, the present invention has the following beneficial effects:

[0011] 1. Advantages of multi-sensor fusion: Through multi-sensor fusion, the advantages of each sensor can be fully utilized to improve the accuracy and reliability of environmental perception, provide students with more comprehensive and accurate environmental perception data, and help students deeply learn the practical application of multi-sensor fusion in intelligent connected vehicles.

[0012] 2. High-performance data processing and diverse presentations based on Linux and ROS2: Traditional training platforms present data in a single format, making it difficult for students to understand. This new method leverages ROS2's rich tools and interfaces to present data in a variety of formats, including real-time graphics and visual models. This allows students to more intuitively and deeply understand the complex relationship between sensor data and environmental perception, effectively improving learning outcomes and cultivating their data interpretation and analysis skills.

[0013] 3. High scalability and compatibility: Based on the Linux open source ecosystem and ROS2 open architecture, it supports third-party software and algorithm integration. Functional upgrades can be achieved by simply replacing or adding corresponding sensor modules. This allows the training platform to keep pace with the development of intelligent connected vehicle technology and reduce equipment upgrade costs.

[0014] 4. Convenient Operation Process: The ROS2-based operation interface is concise and clear, allowing students to quickly familiarize themselves with the operation process and easily start practical training. It provides a high-quality practical operation platform for intelligent connected vehicle teaching.

[0015] 5. This invention uses an STM32 microcontroller and motor controller to precisely control the wheel motion of a 3D vehicle model. Students can visually observe sensor data changes and operating principles during vehicle motion, deepening their understanding of intelligent connected vehicle systems and improving their practical skills and overall understanding of intelligent connected vehicle technology. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 This is the design layout of the intelligent network environment perception sensor training platform of the present invention;

[0017] Figure 2 Design layout for the interior of the 3D vehicle model;

[0018] Figure 1: 1. Intelligent connected environment perception sensor training rig body; 2. 3D vehicle model metal support column; 3. Display terminal; 4. 3D vehicle model; 5. Millimeter wave radar; 6. Ultrasonic radar 1; 7. Ultrasonic radar 2; 8. Ultrasonic radar 3; 9. Ultrasonic radar 4; 10. LiDAR; 11. Camera; 12. CNSS antenna 1; 13. GNSS antenna 2; 14. Power supply adapter board; 15. Inertial measurement unit; 16. STM32 microcontroller; 17. Motor controller 1; 18. Host and debugging software; 19. Signal converter; 20. Ultrasonic radar control box; 21. Motor controller 2. DETAILED DESCRIPTION

[0019] The following will be combined with the accompanying drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0020] 1. Data processing and display system construction

[0021] (1) Hardware level: Please refer to Figure 1-2 Ultrasonic radars 6-9 are connected to the ultrasonic radar control box 20 via connecting cables. The power adapter board 14 supplies power to the ultrasonic radar control box 20 and the millimeter-wave radar 5 via power cables. The ultrasonic radar control box 20 and the millimeter-wave radar 5 are each connected to the signal converter 19 via connecting cables; the signal converter 19 is connected to the host computer 18 via a network cable. The CNSS antenna 12 is connected to the inertial measurement unit 15 via connecting cables, and the inertial measurement unit 15 is connected to the host computer 18 via connecting cables. The power adapter board 14 supplies power to the lidar 19 via power cables. The lidar 19 and the camera are each connected to the host computer 18 via connecting cables. The host computer 18 is connected to the STM32 microcontroller 16 via connecting cables; the STM32 microcontroller 16 is connected to the motor controller 117 and the motor controller 221 via connecting cables; the motor controller 117 and the motor controller 221 are each connected to the four wheel motors via connecting cables.

[0022] (2) Software level: Build the ROS2 framework on the Linux system and develop corresponding ROS2 nodes for different types of sensors. Utilize the ROS2 topic communication mechanism to implement data exchange between sensor nodes. At the same time, develop data processing and display software based on ROS2, using advanced data processing algorithms to fuse and analyze sensor data, and display the processing results in the form of 3D visualization, real-time charts, etc. through command line tools or graphical interfaces.

[0023] 2. Data processing and display system construction

[0024] Software architecture: Based on the Linux open source ecosystem and leveraging the open architecture of ROS2, third-party software and algorithm developers can encapsulate their own software or algorithms into ROS2 nodes for easy access to the training bench system.

[0025] 3. Build a convenient operation process

[0026] The user interface, developed based on ROS2, features an intuitive graphical design. The main interface displays various functional modules using simple icons and menus. Detailed operational instructions and prompts are provided for each module, allowing students to complete complex practical training tasks with simple mouse clicks or keyboard operations. Shortcut keys are also provided for quick operation and improved training efficiency.

[0027] 4. Implementation of wheel motion control based on STM32 and motor controller

[0028] A control program is written on the STM32 microcontroller, which receives sensor data from the data processing unit through the ROS2 communication mechanism, calculates the control parameters of the motor, and then sends the corresponding PWM control signal to the motor controller to accurately control the motor speed and direction, realizing the simulation of various motion states of the wheels of the 3D vehicle model.

[0029] 5. Specific implementation of multi-sensor fusion algorithm:

[0030] The pre-processed sensor data is directly fused, such as merging the distance information of the lidar and the visual information of the camera, to obtain more accurate environmental perception results.

[0031] To operate the intelligent connected environmental perception sensor training platform, connect the external power supply and turn on the host computer. Run the debugging software to visualize environmental perception sensor signals, integrate multiple sensors, and control the operation and stopping of the wheel motors.

[0032] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to these embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the appended claims and their equivalents.

Claims

1. Intelligent connected vehicle multimodal perception and simulation engineering integrated training equipment, characterized by: (1) Data processing and transmission module: Based on the stability and open source characteristics of the Linux system, combined with the powerful distributed computing and communication capabilities of ROS2, the ROS2 framework has corresponding ROS2 nodes developed for different sensors, and each node exchanges data through a topic communication mechanism. (2) Data display module: With the help of ROS2's rich command line tools and graphical interface, the processed data is presented in intuitive forms such as real-time graphics and visual models. (3) Expansion and compatibility modules: Built on the Linux open source ecosystem and ROS2 open architecture, it supports access to third-party software and algorithms; the training platform functions can be upgraded by replacing or adding corresponding sensor modules. (4) 3D vehicle model motion control module: The STM32 microcontroller sends control instructions to the motor controller after processing the sensor data through a preset control algorithm, thereby simulating the movement states of the 3D vehicle model wheels, such as forward, backward, steering, acceleration, and deceleration. (5) Multi-sensor fusion module: Fusion processing of data collected by multiple sensors such as lidar, millimeter-wave radar, and cameras.

2. The intelligent networked environment perception sensor training platform according to claim 1 is characterized by: (1) The data preprocessing is performed according to the characteristics of different sensor data, denoising and filtering are performed on the lidar point cloud data, grayscale and normalization are performed on the camera image data, and data calibration and format conversion are performed on the millimeter wave radar data. (2) The intelligent networked environment perception sensor training platform according to claim 1 is characterized in that the data fusion adopts a combination of data layer fusion and feature layer fusion, and the data layer directly fuses the pre-processed sensor data to obtain more accurate environment perception.

Images