Perception practical training platform and method based on intelligent edge calculation and heterogeneous data fusion

Abstract

The invention discloses a perception practical training platform and method based on intelligent edge calculation and heterogeneous data fusion. The platform comprises a sensor module, an interaction and display module, a calculation module, a movable base and a supporting rack. The sensing practical training platform adopts a modular design and a standardized interface; the sensor module, the interaction and display module and the calculation module can be conveniently disassembled, assembled and replaced, and convenient indoor and outdoor movement can be realized through a movable base with a stopper; through a data fusion technology of a sensor module, in combination with a GPU hardware acceleration strategy and a time synchronization technology, targets around the practical training platform are accurately detected in real time based on multi-source data, and a sensing result of the intelligent networked automobile to the targets in the surrounding environment is displayed. According to the invention, edge calculation provides a more convenient extensible path, and powerful support is provided for intelligent traffic through data acquisition expansion calculation and communication capability of road end data acquisition equipment and cloud end data acquisition.

Description

Technical Field

[0001] The present invention belongs to the technical field of edge computing, and in particular relates to a perception training platform and method based on intelligent edge computing and heterogeneous data fusion. Background Art

[0002] With the development of smart transportation and intelligent connected vehicles, the current edge computing technology will be widely used in target perception in road traffic and intelligent traffic signal passage, providing important technical support for the adjustment of signal lights and guidance of vehicles on the road in smart road traffic. In view of the task requirements of road vehicle control, road condition monitoring, vehicle guidance, etc., combined with the Internet of Everything technology, by deploying edge computing models for smart road traffic, comprehensively utilizing technologies such as fusion learning, multimodal perception, and deep learning, the parallel processing and analysis of various vehicle and pedestrian data in smart transportation can be realized on the road-side perception unit side, providing large-scale, fast-response distributed information computing and data provision functions to meet the rapid provision of road-side information in smart transportation, and provide support for tasks such as intelligent scheduling, vehicle guidance, and road condition monitoring.

[0003] At the same time, due to the vigorous development of smart transportation and intelligent connected vehicle technologies, and their increasing share in terminal applications, these have brought convenience to people's travel, but also brought the urgency of talent training. The application of the Internet of Things, edge computing devices, and heterogeneous data fusion algorithms has brought smart transportation and corresponding talent training into a new stage. In fact, the driving and road control of vehicles in smart transportation require road operation data from the real world, based on the collected road information. At present, the image data of lidar and cameras is very large, and it needs to be processed first at the roadside collection end to provide semantic and other refined data to the cloud. The use of this new biological component does not have corresponding teaching aids to provide practical guidance in traditional talent training. It is necessary to develop new teaching aids and multimodal heterogeneous data fusion algorithms to collect and process roadside information.

[0004] Edge computing is a new computing model that deploys computing and storage units at roadside sensing units to obtain higher real-time computing capabilities and information provision efficiency; huge road unit data information is processed at the edge and then uploaded to the cloud, greatly reducing network load and cloud computing resource pressure. Edge devices provide edge computing capabilities based on heterogeneous data fusion and deep learning technologies. Business programs running on edge devices actively obtain collection results from terminal devices and perform computing and extraction tasks on the edge to improve the efficiency of information transmission.

[0005] Existing teaching equipment mostly focuses on single-vehicle intelligence, and the focus on smart road terminals is not enough. In terms of student training, the lack of teaching equipment has led to a shortage of talents required for the smart transportation system, aggravating the contradiction of structural talent shortage. Summary of the invention

[0006] Purpose of the invention: The purpose of the present invention is to provide a mobile perception training platform that can perform heterogeneous data fusion and intelligent edge computing after multi-sensor integration, aiming to solve the shortcomings of existing perception training platforms in terms of complex deployment, lack of data fusion algorithms, and low training efficiency.

[0007] Technical solution: The present invention provides a perception training platform based on intelligent edge computing and heterogeneous data fusion, including a sensor module, an interaction and display module, a computing module, a movable base and a supporting frame; the perception training platform adopts a modular design and a standardized interface; the sensor module, the interaction and display module, and the computing module can be conveniently disassembled and replaced, and can be conveniently moved indoors and outdoors through a movable base with an inhibitor; through the data fusion technology of the sensor module, combined with the GPU hardware acceleration strategy and time synchronization technology, the targets around the training platform are detected in real time and accurately based on multi-source data, and the perception results of the intelligent connected vehicle on the surrounding environment targets are displayed.

[0008] Furthermore, the standardized interfaces include Ethernet, USB, RS232 and time-sensitive network interfaces. The Ethernet interface is used for high-speed data transmission and remote management, and is connected to the laser radar. The USB interface is used for camera data transmission and local device debugging. The RS232 interface is used for low-latency data communication and multi-node network topology. The standard interface is compatible with future communication technology standards, supports mobile communication protocols, and has the ability to expand networked practical training and teaching functions.

[0009] Furthermore, the data transmission between the sensor module and the interaction and display module is based on the selective application of multiple data transmission protocols, including UDP protocol, TCP / IP protocol, and a customized low-latency data transmission protocol; under a multi-node network topology, through adaptive network routing and dynamic bandwidth allocation strategies, the data transmission path and speed are optimized to ensure the real-time transmission and accuracy of multi-source data.

[0010] Furthermore, the multi-sensor data fusion technology includes the combined application of multi-level data preprocessing, a target detection algorithm based on point cloud data, a target detection algorithm based on image data, and a weighted fusion processing algorithm for detection results, so as to realize real-time fusion of multi-source data; the hardware acceleration strategy is implemented through integer calculation of GPU, which accelerates feature extraction, matrix operation and filtering calculation respectively, and optimizes the target detection processing rate.

[0011] Furthermore, time synchronization uses a timestamp module and NTP (Network Time Protocol) to calibrate the time of the entire network, ensuring the time consistency of multi-sensor output data and automatically starting emergency processing procedures when data anomalies or failures are detected.

[0012] Furthermore, a touch screen is used to implement gesture-based human-computer interaction, enabling convenient algorithm operation and real-time display of target detection algorithms. A large mirrored screen is used to display the fusion processing effect in real time, allowing the device to conduct training for multiple people at a time, achieving higher training efficiency.

[0013] The present invention also discloses a training method for a perception training platform based on intelligent edge computing and heterogeneous data fusion, comprising the following steps:

[0014] Step 1: Connection and configuration between modules. The connection between modules is realized through standardized quick connectors (such as Ethernet, USB, RS232, etc.). The sensor modules (C16 LiDAR, millimeter wave radar and surround view camera) are connected through USB and Ethernet interfaces and can be quickly installed and removed. The industrial host can be tightly connected or removed from the training bench through fixing bolts.

[0015] Step 2: Rapid movement and deployment of the training platform, using a mobile base with a self-locking device, with a high-strength lightweight material training platform and a metal platform body made of stainless steel metal material on the base. The metal platform body and the movable base with a self-locking device can ensure the rapid movement and deployment of the training platform and its stability during training and testing. The high-strength lightweight material training platform can provide a larger training platform and installation position, which can balance the support's own quality and the overall stability of the training platform.

[0016] Step 3: Quick disassembly and assembly of the sensor unit. The laser radar, camera, millimeter-wave radar and other sensor modules are connected to the industrial computer using standard interfaces such as Ethernet and USB, and fixed on the high-strength lightweight material table with universal m4 and m3 screws for easy disassembly and assembly. After installation, tightening and locking, the workbench will remain in a stable state and accurately detect external environmental targets.

[0017] Step 4: Time synchronization and fault recovery mechanism. In order to ensure the accuracy of multi-sensor data perception and fusion, the system uses NTP or GPS protocol to achieve time synchronization of data frames. The industrial computer monitors the status of the sensor in real time. When abnormal data is detected or a system component fails, the fault recovery mechanism is triggered to restart the sensor and collect new data to ensure the correct and stable operation of the system.

[0018] Step 5: Data preprocessing. Before multimodal data perception and fusion, data preprocessing is performed first. Use adaptive low-pass filters and Gaussian smoothing methods to process image data and point cloud data to eliminate noise interference signals in the image. Segment and filter the three-dimensional point cloud data to reduce the total amount of data.

[0019] Step 6: Multi-sensor data fusion, using the decision-level fusion method, first jointly calibrate the camera and lidar to obtain the correspondence between the lidar and pixel coordinates, establish a connection between the lidar point cloud data and the image data, and then calculate the IOU value of the detection box to obtain the fusion detection result of target recognition.

[0020] Step 7: Hardware acceleration strategy, use the GPU hardware on the industrial computer to accelerate the integer calculation content in machine learning and deep learning. The acceleration strategy is mainly used in filtering calculation, feature extraction and fusion judgment, which can greatly improve the data processing speed compared to CPU calculation.

[0021] Step 8: Touch interaction and large-screen result display. Use the touch screen to complete the data collection and start of individual sensors such as lidar, camera, millimeter wave, and ultrasonic radar, and control the computer through the touch screen to complete time synchronization detection and target output under multi-sensor data fusion. The calculation results and calculation process are displayed on the large screen above the training platform, which can be used for training multiple people at a time to improve training efficiency.

[0022] Furthermore, step 6 is specifically as follows: using the post-fusion technology in the fusion process, first calibrate to remove the camera distortion, then synchronize the output results of the camera and radar to ensure the matching between frames; then use the image recognition and point cloud recognition algorithms to extract the target information, obtain the corresponding detection frame, and then project the three-dimensional detection frame into the two-dimensional image for IOU calculation, and finally output the target recognition result under fusion detection. In the case of a good external environment, the detection effect is better than that of the laser radar because the image carries rich object texture information, and the weighted average method is used to correct the confidence of the successfully matched detection frame. The correction formula is:

[0023]

[0024] Among them, L represents the weighting coefficient of the lidar, C represents the weighting coefficient of the camera, and γ L represents the confidence of the lidar, γ C represents the confidence of the camera, γ t represents the corrected confidence level;

[0025] In harsh environments, the confidence of radar detection is improved because the camera loses texture information. The corrected confidence formula is:

[0026]

[0027] By integrating data from multiple on-board sensors and utilizing the rich texture and depth information collected, the accuracy and reliability of target detection around the vehicle can be improved.

[0028] Beneficial effects: Compared with the prior art, the present invention has the following significant advantages:

[0029] (1) The present invention realizes the rapid replacement and upgrading of various functional modules (including edge computing devices and standardized interfaces) through modular structural design and standardized interfaces, significantly improving the deployment efficiency and teaching convenience of the training platform, while enhancing the reliability and robustness of heterogeneous data sources such as cameras, millimeter-wave radars, and lidars, and effectively improving the reliability of data transmission and perception. It can be easily moved and deployed in multiple different scenarios, such as classrooms and road scenarios. The modular design reduces installation and debugging time and enhances the flexibility of the system.

[0030] (2) The present invention integrates a variety of advanced sensors (such as C16 laser radar, millimeter wave radar, camera), combined with extended Kalman filter and edge computing unit, to achieve efficient heterogeneous data fusion and implementation perception capabilities. This method not only improves the accuracy and response speed of the workbench's monitoring of the surrounding environment, adapts to the vehicle's needs for complex environmental perception, but also provides an advanced training platform and technical support for students' practical teaching.

[0031] (3) The present invention adopts a desktop movable platform, combined with modular design and quick connectors, which can realize the rapid and reliable installation, disassembly and configuration of related units on the work platform. This can improve the efficiency of device disassembly and replacement, speed up the replacement of different devices, and improve the use effect of students.

[0032] (4) Through the self-learning algorithm module and real-time feedback mechanism, the system can continuously adjust and optimize the perception and extraction strategy and parameter settings of the target according to the use of teaching and training, continuously improve the recognition ability of different targets, and ensure that the device can accurately identify different targets in the environment.

[0033] (5) The present invention enhances the system's ability to perceive targets in the surrounding environment through heterogeneous data fusion algorithms, edge computing units and self-learning mechanisms, enabling the system to operate and teach efficiently in a variety of application scenarios (inside training rooms and beside traffic roads).

[0034] (6) The present invention realizes information extraction and interaction through gesture recognition combined with a display screen. Through gesture recognition on the touch screen, the teaching object can quickly interact with the control unit of the perception training platform, and can quickly switch different fusion perception algorithms. By modifying the function parameters in the pre-processing and fusion of heterogeneous data, the content of heterogeneous data perception and edge computing can be intuitively and deeply understood. Through the display of the front-end display screen, it can be expanded to a single group of training, so that multiple people can follow and learn while one person is operating, thereby improving the efficiency of teaching and training.

[0035] (7) The present invention provides a high-efficiency and high-reliability training platform for intelligent perception training through multi-sensor inheritance, efficient heterogeneous data fusion method, and flexible deployment, effectively improving the training effect of professionals and practitioners related to intelligent networking. The present invention not only provides an efficient and high-reliability heterogeneous data perception training platform, but also enhances the mobility and adaptability of this device in different environments, and can be flexibly deployed in different application scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 This is a schematic diagram of the overall structure of the perception training platform based on intelligent edge computing and heterogeneous data fusion; in the figure, there are a screen with a touch panel 1, a medium-range millimeter-wave radar 2, a large screen 3, a C16 laser radar 4, an ultrasonic radar 5, an SCX-1400 embedded workstation 6, a 360-degree surround view camera 7, and a BSD millimeter-wave radar 8.

[0037] Figure 2 It is the algorithm flow chart of multi-sensor data fusion;

[0038] Figure 3 Schematic diagram of time synchronization and fault detection mechanism. DETAILED DESCRIPTION

[0039] The technical solution of the present invention is further described below in conjunction with the accompanying drawings.

[0040] like Figure 1 As shown, the present invention provides a mobile perception training platform for edge computing after multi-sensor integrated mobile multi-sensor heterogeneous data fusion. The training platform includes a sensor module, a movable stand, an edge computing module and a power supply module, which can be implemented through modular design, heterogeneous data fusion, efficient deployment and interactive methods.

[0041] On the basis of the above technical solutions, the training platform adopts a modular structure design, which makes it highly scalable, flexible, and efficient in assembly and maintenance. The modular design uses standardized interfaces and plug-and-play components to achieve rapid disassembly, installation, replacement and upgrading of each functional module. This design concept can simplify the disassembly, assembly and maintenance process of the training platform, and is also in line with the components used in the actual production and maintenance process of intelligent connected vehicles, supporting the upgrading and maintenance of subsequent sub-components, which can significantly improve the operational reliability and training efficiency of the training platform.

[0042] Based on the above technical solution, the training platform is equipped with a C16 laser radar ( Figure 1 Bid No. 4), a medium-range millimeter-wave radar ( Figure 1 Winning bid number 2), four BSD millimeter wave radars ( Figure 1 Winning bid number 8), 8 ultrasonic radars ( Figure 1 No. 5), 4 360-degree panoramic cameras ( Figure 1 The middle number 7) is used to extract multi-angle environmental information about the platform's surroundings. The surround view camera is calibrated and the graphics stitching algorithm is used to provide real-time texture information of the surrounding environment, which can be used by the semantic segmentation algorithm to extract the target object and improve the confidence in the recognition of the surrounding environment. The ultrasonic radar can monitor objects within 2m around the platform in real time, provide the latest warning information, and use automotive-grade components, which is conducive to practical training. The medium-range millimeter-wave radar can track and identify targets beyond 150m, and can maintain high-resolution monitoring of multiple targets in bad weather and low light conditions. The BSD millimeter-wave radar can supplement the monitoring of the vehicle's surroundings, track and warn fast-approaching targets through Doppler technology, and supplement the lack of information in depth of the image sensor unit. The C16 laser radar provides high-precision three-dimensional point cloud data, and completes the construction of the surrounding environment model and obstacle detection by supplementing the depth information. By installing the real vehicle-mounted sensor unit on the perception training teaching platform, the environmental information extraction around the vehicle is completed.

[0043] Based on the above technical solution, the edge computing module adopts SCX-1400 embedded workstation ( Figure 1 No. 6), configure multi-core CPU (such as / Core TMi7) and high-performance GPUs (such as NVIDIA RTX 3060) to achieve high-speed heterogeneous data fusion processing and AI algorithm implementation. The environmental information collected by multiple sensors is quickly processed and analyzed in the edge computing module, which can provide real-time support for vehicle decision-making. This process can provide real observable and tangible objects for training subjects. The modular components in edge computing have a variety of expansion interfaces (such as PCIe, M.2), support flexible configuration of storage devices and RAID storage protocols, provide real data scenarios, and use redundant storage technology (RAID 0,1,5,10) to provide data protection. . The edge computing module carries an active heat dissipation system and overheating protection function to ensure that the equipment can work normally and stably under high load and protect the safety of the equipment under extreme working conditions.

[0044] Based on the above technical solution, a linked dual screen is used to realize the input of interactive information and the display of execution results. A screen with a touch panel ( Figure 1 The training personnel can complete the input of instructions through touch and gesture operation, avoiding the tedious keyboard operation and simplifying the operation. The operation process can be displayed in real time on the touch screen, and the processing results of heterogeneous data fusion can be displayed on the touch screen. The operation and display on the touch screen can be mirrored in real time on the large screen above the training table ( Figure 1 Winning bid number 3) helps to realize batch training of personnel and improve training efficiency.

[0045] On the basis of the above technical solution, the convenient disassembly and replacement realizes the rapid switching of training scenarios and use debugging of the edge computing perception training platform for heterogeneous data processing through the combination of modularly designed components and a movable platform; modular components can ensure the rapid installation and disassembly of each functional module, and the dedicated terminals also ensure the reliability and durability of the components, and can ensure the reliability of the port after multiple uses; the movable base allows the training platform to be used and recycled in different outdoor environments, thereby improving the training effect of personnel.

[0046] Based on the above technical solutions, a decision-level fusion method is adopted. In the data fusion process, the camera is calibrated first to eliminate image distortion; then, the camera and lidar detection results are synchronized; after that, the image and point cloud are respectively sent to the image target recognition algorithm and the lidar point cloud target recognition algorithm to obtain the corresponding detection frame, and then the projection of the detection frame is calculated by IOU, and finally the fusion detection result is output.

[0047] The present invention provides a perception training platform device carrying an edge computing unit for multi-sensor heterogeneous data fusion, comprising the following steps:

[0048] Step 1: Connection and configuration between modules. The connection between modules is realized through standardized quick connectors (such as Ethernet, USB, RS232, etc.). The sensor modules (C16 LiDAR, millimeter wave radar and surround view camera) are connected through USB and Ethernet interfaces and can be quickly installed and removed. The industrial host can be tightly connected or removed from the training bench through fixing bolts.

[0049] Step 2: Rapid movement and deployment of the training platform, using a mobile base with a self-locking device, with a high-strength lightweight material training platform and a metal platform body made of stainless steel metal material on the base. The metal platform body and the movable base with a self-locking device can ensure the rapid movement and deployment of the training platform and its stability during training and testing. The high-strength lightweight material training platform can provide a larger training platform and installation position, which can balance the support's own quality and the overall stability of the training platform.

[0050] Step 3: Quick disassembly and assembly of the sensor unit. The laser radar, camera, millimeter-wave radar and other sensor modules are connected to the industrial computer using standard interfaces such as Ethernet and USB, and fixed on the high-strength lightweight material table with universal m4 and m3 screws for easy disassembly and assembly. After installation, tightening and locking, the workbench will remain in a stable state and accurately detect external environmental targets.

[0051] Step 4: Time synchronization and fault recovery mechanism. In order to ensure the accuracy of multi-sensor data perception and fusion, the system uses NTP or GPS protocol to achieve time synchronization of data frames. The industrial computer monitors the status of the sensor in real time. When abnormal data is detected or a system component fails, the fault recovery mechanism is triggered to restart the sensor and collect new data to ensure the correct and stable operation of the system.

[0052] Step 5: Data preprocessing. Before multimodal data perception and fusion, data preprocessing is performed first. Use adaptive low-pass filters and Gaussian smoothing methods to process image data and point cloud data to eliminate noise interference signals in the image. Segment and filter the three-dimensional point cloud data to reduce the total amount of data.

[0053] Step 6: Multi-sensor data fusion, using the decision-level fusion method, first jointly calibrate the camera and lidar to obtain the correspondence between the lidar and pixel coordinates, establish a connection between the lidar point cloud data and the image data, and then calculate the IOU value of the detection box to obtain the fusion detection result of target recognition.

[0054] Step 7: Hardware acceleration strategy, use the GPU hardware on the industrial computer to accelerate the integer calculation content in machine learning and deep learning. The acceleration strategy is mainly used in filtering calculation, feature extraction and fusion judgment, which can greatly improve the data processing speed compared to CPU calculation.

[0055] Step 8: Touch interaction and large-screen result display. Use the touch screen to complete the data collection and start of individual sensors such as lidar, camera, millimeter wave, and ultrasonic radar, and control the computer through the touch screen to complete time synchronization detection and target output under multi-sensor data fusion. The calculation results and calculation process are displayed on the large screen above the training platform, which can be used for training multiple people at a time to improve training efficiency.

[0056] In order to ensure the accuracy of data fusion, the computing module configures a timestamp module on the data acquisition node and uses NTP (Network Time Protocol) or GPS time synchronization protocol to complete the full network time synchronization detection and correction of multi-sensor data. When the system detects time anomalies or sensor failures, the computing module will automatically trigger the fault alarm program and quickly switch to the redundant module through fault detection and automatic recovery mechanisms to avoid erroneous data affecting the training effect.

[0057] A highly reliable data fusion technology is used in the present invention, and the post-fusion technology is used to achieve the unity of high reliability and high accuracy. In the fusion process, calibration is first performed to remove the distortion of the camera. Then the output results of the camera and the radar are synchronized in time to ensure the matching between the frames; then the target information is extracted using image recognition and point cloud recognition algorithms respectively, and the corresponding detection frame is obtained. After that, the three-dimensional detection frame is projected into the two-dimensional image for IOU calculation, and finally the target recognition result under fusion detection is output. In the case of a good external environment, the detection effect is better than that of the laser radar because the image carries rich object texture information, and the weighted average method is used to correct the confidence of the successfully matched detection frame. The correction formula is:

[0058]

[0059] In harsh environments, the confidence of radar detection is improved because the camera loses texture information. The corrected confidence formula is:

[0060]

[0061] By integrating the data from multiple sensors on the vehicle, the rich texture and depth information collected can be fully utilized to improve the accuracy and reliability of the detection of objects around the vehicle. Then, the interactive information and calculation results of the touch screen are mirrored on the large screen of the training platform, which improves the training effect and efficiency of the training platform.

[0062] Through the above steps and technical means, the present invention realizes a perception training platform with efficient fusion and processing of multi-source data, ensuring the real-time monitoring and target recognition and detection capabilities of the intelligent connected vehicle to the surrounding environment. At the same time, a touch screen is used to achieve convenient human-computer interaction, and a mirrored large screen is used to clearly display the detection and calculation results of the training platform, thereby improving the training efficiency and effectiveness.

Claims

1. A perception training platform based on intelligent edge computing and heterogeneous data fusion, characterized in that: It includes a sensor module, an interaction and display module, a computing module, a movable base and a supporting frame; the perception training platform adopts a modular design and a standardized interface; the sensor module, the interaction and display module, and the computing module can be easily disassembled and replaced, and can be conveniently moved indoors and outdoors through a movable base with an inhibitor; through the data fusion technology of the sensor module, combined with the GPU hardware acceleration strategy and time synchronization technology, the targets around the training platform are detected in real time and accurately based on multi-source data, and the perception results of the intelligent connected vehicle on the surrounding environment targets are displayed.

2. According to claim 1, a perception training platform based on intelligent edge computing and heterogeneous data fusion is characterized in that: The standardized interfaces include Ethernet interface, USB interface, RS232 interface and time-sensitive network interface; among them, the Ethernet interface is used for high-speed data transmission and remote management, and is connected to the laser radar, the USB interface is used for camera data transmission and local device debugging, and the RS232 interface is used for low-latency data communication and multi-node network topology; the standard interface is compatible with future communication technology standards, can support mobile communication protocols, and has the ability to expand networked practical training and teaching functions.

3. According to claim 1, a perception training platform based on intelligent edge computing and heterogeneous data fusion is characterized in that: The data transmission between the sensor module and the interaction and display module is based on the selective application of multiple data transmission protocols, including UDP protocol, TCP / IP protocol, and a customized low-latency data transmission protocol; under a multi-node network topology, the data transmission path and speed are optimized through adaptive network routing and dynamic bandwidth allocation strategies.

4. According to claim 1, a perception training platform based on intelligent edge computing and heterogeneous data fusion is characterized in that: The data fusion technology includes the combined application of multi-level data preprocessing, a target detection algorithm based on point cloud data, a target detection algorithm based on image data and a weighted fusion processing algorithm of detection results to achieve real-time fusion of multi-source data; the GPU hardware acceleration strategy is implemented through integer calculations of the GPU, which accelerates feature extraction, matrix operations and filtering calculations respectively, and optimizes the target detection processing rate.

5. According to claim 1, a perception training platform based on intelligent edge computing and heterogeneous data fusion is characterized in that: The time synchronization technology performs network-wide time calibration through a timestamp module and the network time protocol NTP to ensure the time consistency of multi-sensor output data and automatically initiate an emergency processing procedure when data anomalies or failures are detected.

6. The perception training platform based on intelligent edge computing and heterogeneous data fusion according to claim 1 is characterized in that: The interaction and display module uses a touch screen, and the touch screen is based on human-computer interaction of gestures.

7. A training method based on the perception training platform of claim 1, characterized in that: The steps include: Step 1: Connection and configuration between modules. The modules are connected through standardized quick connectors. The sensor modules are connected through USB and Ethernet interfaces. The industrial host can be tightly connected or removed from the training bench through fixing bolts. Step 2: Rapid movement and deployment of the training platform, using a mobile base with a self-locking device, with a training table top made of high-strength lightweight material and a metal platform body made of stainless steel metal material on top of the base; Step 3: Quick disassembly and assembly of the sensor unit. The sensor module is connected to the industrial computer using a standard interface and fixed to the high-strength lightweight material table with screws. After installation, tightening and locking, the table will remain in a stable state and accurately detect external environmental targets; Step 4: Time synchronization and fault recovery mechanism. The system uses NTP or GPS protocol to achieve time synchronization of data frames. The industrial computer monitors the status of the sensor in real time. When data anomalies are detected or system components are found to be faulty, the fault recovery mechanism is triggered to restart the sensor and collect new data to ensure the correct and stable operation of the system. Step 5: Data preprocessing: Use adaptive low-pass filter and Gaussian smoothing method to process image data and point cloud data, eliminate noise interference signals in the image, segment and filter the 3D point cloud data, and reduce the total amount of data; Step 6: Multi-sensor data fusion, using the decision-level fusion method, first jointly calibrate the camera and lidar to obtain the correspondence between the lidar and pixel coordinates, establish a connection between the lidar point cloud data and the image data, and then calculate the IOU value of the detection box to obtain the fusion detection result of target recognition; Step 7: Hardware acceleration strategy, using the GPU hardware on the industrial computer to accelerate the integer calculation content in machine learning and deep learning. The acceleration strategy is applied to filter calculation, feature extraction and fusion judgment; Step 8: Touch interaction and large-screen result display. Use the touch screen to complete the data collection and start of individual sensors, and control the computer through the touch screen to complete time synchronization detection and target output under multi-sensor data fusion. The calculation results and calculation process are displayed on the large screen above the training platform. It can be used to train multiple people at a time to improve training efficiency.

8. The training method according to claim 7, characterized in that: In step 1, the standardized quick connector includes Ethernet, USB or RS232; the sensor module includes a C16 laser radar, a millimeter wave radar or a surround view camera.

9. The training method according to claim 7, characterized in that: Step 6 is as follows: using the post-fusion technology, first calibrate to remove camera distortion during the fusion process, then synchronize the output results of the camera and radar to ensure matching between frames; then use image recognition and point cloud recognition algorithms to extract target information and obtain the corresponding detection frame, then project the 3D detection frame into the 2D image for IOU calculation, and finally output the target recognition result under fusion detection. In the case of a good external environment, the detection effect is better than that of the laser radar because the image carries rich object texture information. The weighted average method is used to correct the confidence of the successfully matched detection frame. The correction formula is: Among them, L represents the weighting coefficient of the lidar, C represents the weighting coefficient of the camera, and γ L represents the confidence of the lidar, γ C represents the confidence of the camera, γ t represents the corrected confidence level; In harsh environments, the confidence of radar detection is improved because the camera loses texture information. The corrected confidence formula is: By integrating data from multiple on-board sensors and utilizing the rich texture and depth information collected, the accuracy and reliability of target detection around the vehicle can be improved.

Images