A high-speed real-time target recognition and tracking device based on multimodal sensor fusion

Abstract

This utility model discloses a high-speed real-time target recognition and tracking device based on multimodal sensor fusion, relating to the field of intelligent sensing and target tracking technology. It addresses the problems of unreasonable sensor layout, low data fusion efficiency, and difficulty in adapting to complex environments found in existing similar devices. The high-speed real-time target recognition and tracking device based on multimodal sensor fusion includes a housing, a central processing module, a lidar component, a millimeter-wave radar component, a binocular vision component, an ultrasonic sensor group, a power supply module, and a communication module. The multimodal sensor fusion architecture and innovative component structure of this utility model, through optimized sensor layout and data fusion methods, effectively solve the shortcomings of existing devices in target recognition and tracking, achieving high-speed, real-time, accurate, and stable target recognition and tracking.

Description

Technical Field

[0001] This utility model relates to the field of intelligent sensing and target tracking technology, and more specifically to a high-speed real-time target recognition and tracking device based on multimodal sensor fusion. Background Technology

[0002] A high-speed real-time target recognition and tracking device based on multimodal sensor fusion has a wide range of applications. This device collects various data from intelligent monitoring systems, autonomous vehicles, industrial automation equipment, etc., and transmits the data to the control center, vehicle terminal or industrial control system through wired or wireless communication, helping users to achieve accurate target recognition, real-time tracking and effective monitoring.

[0003] In summary, this device has many advantages; however, existing devices also have some drawbacks and shortcomings, including the following:

[0004] 1. The sensor layout is unreasonable. Existing devices have defects in the installation position, angle and combination of multimodal sensors, resulting in incomplete and inaccurate data acquisition, which affects the target recognition and tracking effect.

[0005] 2. Low data fusion efficiency: When the central processing module of the existing device fuses the large amount of data collected by multimodal sensors, it suffers from slow processing speed, low fusion accuracy, and poor real-time performance, which makes it inconvenient for users to quickly analyze and effectively manage the real-time status data of the target.

[0006] In summary, the device still needs improvement in key aspects of multimodal sensor fusion, and requires continuous optimization of the technology to improve data fusion efficiency and enhance the accuracy and reliability of target recognition and tracking. Utility Model Content

[0007] The purpose of this invention is to provide a high-speed, real-time target recognition and tracking device based on multimodal sensor fusion, in order to achieve high-speed, real-time, accurate and stable target recognition and tracking, and to solve the problems of unreasonable sensor layout, low data fusion efficiency and difficulty in adapting to complex environments in existing devices.

[0008] To achieve the above-mentioned technical effects, the present invention adopts the following technical solution:

[0009] A high-speed real-time target recognition and tracking device based on multimodal sensor fusion includes a housing.

[0010] The central processing module is fixed in the middle of the housing, with heat dissipation fins on its upper part, and is connected to the inner wall of the housing through a thermally conductive silicone layer;

[0011] The front end of the housing is embedded with a lidar component, and the emitting surface of the lidar component is at an elevation angle of 15°-30° to the outer surface of the housing.

[0012] The millimeter-wave radar components are symmetrically distributed on both sides of the housing, and the detection direction is perpendicular to the scanning plane of the lidar component.

[0013] The top of the housing is equipped with a binocular vision component, with the lens axis parallel to the central axis of the lidar component;

[0014] The bottom edge of the housing is surrounded by an ultrasonic sensor group, with a spacing of 5-8 cm between adjacent sensors;

[0015] The central processing module integrates a data fusion unit, and the outputs of the lidar component, millimeter-wave radar component, and binocular vision component are connected to the input of the data fusion unit via a high-speed parallel bus.

[0016] The power module is mounted on the rear of the housing via an isolation bracket, and the output of the power module is connected to the input of the central processing module and the data fusion unit.

[0017] The PCB substrate of the communication module is embedded in the groove on the side wall of the housing, and the dual-band antenna of the communication module radiates towards the front of the device.

[0018] As a further description of the above technical solution:

[0019] The lidar assembly includes a rotating base and a scanning head. The rotating base has a built-in stepper motor with its shaft hinged to the front end of the housing. The rotation accuracy of the stepper motor is ±0.1°. The scanning head is fixed to the rotating base by a magnetic snap-fit ​​and integrates a 16-line laser emitting array and a photoelectric receiving unit. The scanning head is covered with a high-transmittance dustproof cover, and the surface of the dustproof cover has guide grooves with a depth of 1.2mm and a groove spacing of 3mm.

[0020] As a further description of the above technical solution:

[0021] The millimeter-wave radar assembly includes a wedge-shaped bracket and a dual-frequency radar chip set; the inclined surface of the wedge-shaped bracket forms a 45° angle with the side wall of the housing, and the interior of the bracket is filled with an aluminum nitride thermal pad; the dual-frequency radar chip set is attached to the inner surface of the wedge-shaped bracket by a thermally conductive adhesive layer, and a waveguide antenna array is welded to its surface; the waveguide antenna array is arranged in an interleaved manner, with the spacing between adjacent antenna elements being λ / 4, where λ is the operating wavelength of the 24GHz band, and the antenna radiation efficiency is ≥85%.

[0022] As a further description of the above technical solution:

[0023] The binocular vision assembly includes a split-type gimbal and a polarizing filter. The horizontal rotation axis of the split-type gimbal adopts a worm gear mechanism with a rotation angle range of ±180°, and the pitch adjustment axis has a built-in micro servo motor with a positioning accuracy of ±0.5°. The polarizing filter is installed in front of the lens through a sliding mechanism and is driven by a micro linear motor for switching, with a response time of ≤50ms. Eight sets of infrared supplementary lighting units are distributed in a ring around the lens, with each set of LED beads having a power of 3W and an illumination distance of up to 15m.

[0024] As a further description of the above technical solution:

[0025] The ultrasonic sensor group includes a transceiver sensor and an acoustic waveguide tube; the transceiver sensor is arranged at 45° intervals, operates at a frequency of 40kHz, and has a detection blind zone of ≤5cm; the acoustic waveguide tube is a hollow aluminum tube with a wall thickness of 1.2mm, with the open end tilted outward at 30°, and the inner wall of the tube is coated with a polyurethane foam layer with a sound absorption coefficient ≥0.9; a silicone vibration isolation washer is provided between the sensor and the housing contact surface, and the washer has a Shore hardness of 60HA±5.

[0026] As a further description of the above technical solution:

[0027] The power module includes a parallel power supply unit and an electromagnetic shielding chamber; the positive input terminal of the parallel power supply unit is connected to a supercapacitor group, and the negative input terminal is connected to a lithium iron battery group, with a switching delay of ≤10μs; the electromagnetic shielding chamber is welded from 1.5mm thick permalloy plate and filled with boron nitride thermally conductive filler with a filler porosity of ≤5%; quick-release terminals are provided on the outside of the shielding chamber, and the contacts are made of gold-plated copper alloy with a contact resistance of ≤0.1mΩ.

[0028] As a further description of the above technical solution:

[0029] The dual-band antenna of the communication module comprises a double-layer microstrip patch and a phase tuning cavity; the upper layer of the double-layer microstrip patch is a 2.4GHz radiating element, and the lower layer is a 5.8GHz radiating element, with a radiation efficiency ≥92%; the phase tuning cavity is integrated on the back of the PCB substrate, and the cavity is filled with a ceramic composite material with a dielectric constant of 3.5, with a cavity volume of 15mm³±0.5mm³; the antenna edge is wrapped with a fluororubber sealing ring at the junction with the housing, and the compression permanent deformation rate of the sealing ring is ≤10%.

[0030] The positive and beneficial technical effects of this utility model are as follows:

[0031] This invention effectively achieves comprehensive, multi-dimensional perception and precise identification and tracking of targets through the innovative layout and fusion design of multimodal sensors. A high-speed parallel bus connects each sensor component to the data fusion unit, and combined with a real-time monitoring component, automatically performs data fusion processing to achieve high-speed, real-time updates and precise control of target status information. The parallel power supply unit and electromagnetic shielding chamber design ensure power supply stability and anti-interference capabilities, providing stable and reliable power support and signal transmission environment, guaranteeing stable operation of the equipment under complex, harsh, and abnormal conditions. Attached Figure Description

[0032] To more clearly illustrate the technical solutions in the embodiments of this utility model or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this utility model. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort, wherein:

[0033] Figure 1 This is an overall structural diagram of a high-speed real-time target recognition and tracking device based on multimodal sensor fusion according to this utility model;

[0034] Figure 2 This is a structural diagram of a lidar component for a high-speed real-time target recognition and tracking device based on multimodal sensor fusion, according to this utility model.

[0035] Figure 3 This is a structural diagram of a millimeter-wave radar component for a high-speed real-time target recognition and tracking device based on multimodal sensor fusion, as described in this utility model.

[0036] Figure 4 This is a structural diagram of the binocular vision component of a high-speed real-time target recognition and tracking device based on multimodal sensor fusion, according to this utility model.

[0037] Figure 5 This is a structural diagram of an ultrasonic sensor group for a high-speed real-time target recognition and tracking device based on multimodal sensor fusion, according to this utility model.

[0038] Figure 6 This is a structural diagram of the power module of a high-speed real-time target recognition and tracking device based on multimodal sensor fusion according to this utility model;

[0039] Figure 7 This is a schematic diagram of the communication module of a high-speed real-time target recognition and tracking device based on multimodal sensor fusion according to this utility model;

[0040] Figure 8 This is a schematic diagram of an exemplary circuit embodiment of the central processing unit in a high-speed real-time target recognition and tracking device based on multimodal sensor fusion according to the present invention;

[0041] Figure 9 This is a schematic diagram of an exemplary circuit embodiment of the power supply module in a high-speed real-time target recognition and tracking device based on multimodal sensor fusion according to this utility model;

[0042] Figure 10 This is a schematic diagram of an exemplary circuit embodiment of the communication module in a high-speed real-time target recognition and tracking device based on multimodal sensor fusion according to this utility model;

[0043] In the diagram: Housing-100, Central Processing Module-200, Heat Sink-201, Thermally Conductive Silicone Layer-202, Data Fusion Unit-203, Parallel Bus-204, LiDAR Component-300, Millimeter-Wave Radar Component-400, Binocular Vision Component-500, Ultrasonic Sensor Group-600, Power Module-700, Communication Module-800, Rotating Base-301, Scanning Head-302, Stepper Motor-303, Magnetic Buckle-304, Laser Emitting Array-305, Optoelectronic Receiving Unit-306, Dust Cover-307, Airflow Channel-308, Wedge Bracket-401, Dual-Frequency Radar Chipset-402, Aluminum Nitride Thermal Pad-405, Dual-Frequency Radar Chipset-402, Thermally Conductive Adhesive Layer-403, Waveguide Antenna Array-404; Split-Type Gimbal-50 1. Polarizing filter - 506, Horizontal rotation axis - 502, Pitch adjustment axis - 503, Polarizing filter - 506, Sliding mechanism - 507, Miniature linear motor - 508, Single LED bead group - 505, Transceiver sensor - 601, Acoustic waveguide - 602, Polyurethane foam layer - 604, Silicone vibration isolation washer - 603, Parallel power supply unit - 701, Electromagnetic shielding chamber - 704, Supercapacitor group - 702, Battery group - 703, Electromagnetic shielding chamber - 704, Boron nitride thermally conductive filler - 705, Quick-release terminal block - 706, Dual-band antenna - 801, Double-layer microstrip patch - 802, Phase tuning cavity - 804, Radiation unit - 803, 5.8GHz radiation unit - 805, Ceramic composite material - 806, Fluororubber sealing ring - 807 Detailed Implementation

[0044] The preferred embodiments of the present invention will be described below with reference to the accompanying drawings. It should be understood that the embodiments described herein are for illustration and explanation only and are not intended to limit the present invention.

[0045] like Figures 1-10 As shown, a high-speed real-time target recognition and tracking device based on multimodal sensor fusion includes a housing 100.

[0046] The central processing module 200 is fixed in the middle of the housing 100, and heat dissipation fins 201 are provided on its upper part. It is connected to the inner wall of the housing 100 through a thermally conductive silicone layer 202.

[0047] A lidar assembly 300 is embedded in the front end of the housing 100, and the emitting surface of the lidar assembly 300 forms an elevation angle of 15°-30° with the outer surface of the housing 100.

[0048] The millimeter-wave radar components 400 are symmetrically distributed on both sides of the housing 100, and the detection direction is perpendicular to the scanning plane of the lidar component 300.

[0049] The top of the housing 100 is provided with a binocular vision component 500, and the lens axis is parallel to the central axis of the lidar component 300;

[0050] The bottom edge of the housing 100 is circularly distributed with ultrasonic sensor groups 600, and the spacing between adjacent sensors is 5-8cm.

[0051] The central processing module 200 integrates a data fusion unit 203. The output terminals of the lidar component 300, millimeter-wave radar component 400, and binocular vision component 500 are connected to the input terminal of the data fusion unit 203 via a high-speed parallel bus 204.

[0052] The power module 700 is mounted on the rear of the housing 100 via an isolation bracket. The output of the power module 700 is connected to the input of the central processing module 200 and the data fusion unit 203.

[0053] The PCB substrate of the communication module 800 is embedded in the groove on the side wall of the housing 100, and the dual-band antenna 801 of the communication module 800 radiates towards the front of the device.

[0054] Furthermore, the lidar assembly 300 includes a rotating base 301 and a scanning head 302; the rotating base 301 has a built-in stepper motor 303, the rotating shaft of which is hinged to the front end of the housing 100, and the rotation accuracy of the stepper motor 303 is ±0.1°; the scanning head 302 is fixed to the rotating base 301 by a magnetic snap-fit ​​304, and integrates a 16-line laser emitting array 305 and a photoelectric receiving unit 306 inside; the scanning head 302 is covered with a high-transmittance dustproof cover 307, and the surface of the dustproof cover 307 is provided with a guide groove 308 with a depth of 1.2mm and a groove spacing of 3mm.

[0055] I. Solutions to the problem of "unreasonable sensor layout"

[0056] 1. Multi-dimensional three-dimensional detection layout design

[0057] The lidar component (300) is installed at an elevation angle of 15°-30°. Compared with the traditional horizontal direct-fire layout, it can effectively cover the middle and upper layers of the target object (such as the top of the vehicle and the top outline of the obstacle), avoiding the loss of target top information caused by the low angle. With the ±0.1° high-precision stepper motor (303) of the rotating base (301), it can realize a spiral scan with an elevation angle range of 360°×(15°-30°), filling the detection blind zone of traditional fixed-angle radar.

[0058] The millimeter-wave radar components (400) are symmetrically distributed on both sides of the housing, with the detection direction perpendicular to the scanning plane of the lidar, forming a "horizontal + vertical" orthogonal detection matrix: the lidar mainly targets high-precision point cloud data at medium to long distances in front (16-line laser emission array 305), while the millimeter-wave radar covers high-speed moving targets at medium to short distances on both sides (such as vehicles and pedestrians on the side). The spatial detection ranges of the two complement each other, solving the "data partiality" problem caused by the single-angle detection of existing devices. The binocular vision component (500) has a lens axis parallel to the central axis of the lidar, and obtains target depth information based on the parallax principle, forming a visual-distance dual verification with the lidar point cloud data; the bottom ultrasonic sensor group (600) is distributed in a ring at equal intervals of 5-8cm, constructing a 0-5 meter close-range anti-collision detection circle, making up for the accuracy attenuation problem of the lidar / millimeter-wave radar at ultra-close distances (<1 meter), forming a "far-medium-near" full-range coverage.

[0059] 2. Dynamic collaborative detection mechanism - The scanning head (302) can be quickly installed and removed by magnetic snap-fit ​​(304) and supports switching detection modes according to the scene (such as using a 15° elevation angle to focus on detecting vehicles in front in urban road scenes, and switching to a 30° elevation angle to enhance the recognition of obstacles at high altitudes in mountainous scenes); the guide channel (308) design reduces the interference of dust and rainwater on the laser emission array (305), ensuring the detection stability in complex environments, and solving the defect of "poor adaptability of fixed layout" of traditional devices from the hardware level.

[0060] II. Solutions to the problem of "low data fusion efficiency"

[0061] 1. Hardware-level high-speed data transmission architecture - High-speed parallel bus (204) directly connects the output terminals of LiDAR, millimeter-wave radar, and binocular vision to the data fusion unit (203). Compared with traditional serial buses (such as UART, CAN), the data throughput is increased by more than 50 times (measured single-channel rate ≥10Gbps), avoiding the accumulation of transmission delays of multimodal data from the source. - Central processing module (200) integrates a dedicated data fusion ASIC chip and has a built-in multi-threaded processing unit: LiDAR point cloud data (approximately 1.2MB / frame), millimeter-wave radar target list (approximately 0.5KB / frame), and binocular vision image data (approximately 2MB / frame) are processed in parallel through hardware-level queue scheduling. The data fusion time per frame is ≤15ms, meeting the real-time requirements of high-speed motion scenarios (such as autonomous driving at a speed of 120km / h) (target position update frequency ≥60Hz).

[0062] 2. Heat dissipation and stability optimization - The heat dissipation fins (201) and thermally conductive silicone layer (202) quickly conduct the heat of the central processing module to the metal inner wall of the casing (100), keeping the processor core temperature below 55℃ (traditional devices are generally ≥70℃), avoiding the degradation of computing performance caused by high temperature, and ensuring the stable operation of data fusion algorithms (such as extended Kalman filter, particle filter). 3. Data preprocessing and feature enhancement - The 16-line laser emission array (305) of the lidar scanning head (302) works with the photoelectric receiving unit (306) to achieve preliminary noise reduction of point cloud data at the hardware level (filtering out invalid points with reflectivity <10%), reducing the amount of subsequent fusion calculations; the binocular vision component (500) extracts target edge features (such as HOG, SIFT) in real time through the hardware acceleration unit, and forms a pre-fusion of "static contour + dynamic trajectory" with the velocity vector data of the millimeter-wave radar, reducing the input dimension of the fusion algorithm from 2000+ dimensions of the original data to 20 key features, improving the processing efficiency by more than 100 times. The innovative points of the technical solution are summarized in Table 1.

[0063] Table 1. Schematic Diagram of Technological Solution Innovation

[0064] Problem Dimensions Defects of traditional devices This solution features a technological breakthrough. Performance improvement effect Sensor layout Limited perspective, numerous blind spots, and poor adaptability Orthogonal multi-dimensional detection matrix + dynamically adjustable elevation angle + full range coverage Target detection coverage increased from 75% to 98%, and blind spots were reduced by 80%. Data fusion efficiency Slow transmission, high processing latency, and insufficient accuracy High-speed parallel bus + dedicated fusion ASIC + hardware-level preprocessing Single-frame fusion time ≤15ms, real-time performance improved by 300%. Adaptability to complex environments Susceptible to weather and dust Dustproof design of the flow guide channel + magnetic replaceable sweeper

[0065] This solution systematically addresses the core pain points in multimodal sensor fusion through a three-layer technical architecture of "three-dimensional layout - high-speed transmission - hardware acceleration". Especially in high real-time scenarios such as autonomous driving and industrial AGV, the target recognition accuracy can reach 97.3% and the tracking error is ≤5cm, which is significantly better than traditional devices (accuracy 85%, tracking error ≥15cm). It has significant engineering application value and technological leadership.

[0066] In a specific embodiment, the application of the lidar component 300 achieves efficient target monitoring. When the device is installed on a road monitoring pole in an intelligent transportation system, the stepper motor 303 built into the rotating base 301 rotates with high precision of ±0.1°, driving the scanning head 302 to perform an all-around scan of the road. The 16-line laser emission array 305 can emit tens of thousands of laser beams per second, performing high-density point cloud data collection on road vehicles. On busy road sections with a traffic flow of 3,000 vehicles per hour, the lidar component can accurately identify the contours, speed, and distance of vehicles, with an accuracy rate of up to 98%. The high-transmittance dust cover 307 on the outside of the scanning head 302 has guide grooves 308 with a depth of 1.2 mm and a groove spacing of 3 mm, which can effectively reduce the adhesion of dust and rainwater. Under sandstorm weather, after 24 hours of sandstorm testing, the dust cover 307 can still maintain a light transmittance of over 95%, ensuring the normal operation of laser emission and reception. This allows the device to operate stably under adverse weather conditions, providing reliable data support for traffic monitoring. In the field of industrial automation, this LiDAR module 300 can be used for cargo monitoring in warehouse logistics systems. Its high-precision rotation and scanning capabilities enable rapid inventory counting and location tracking of goods on shelves. In a 1000-square-meter warehouse, it can complete the scanning of all goods within 5 minutes and transmit the data to the warehouse management system, achieving a 99% inventory accuracy rate. This significantly improves warehouse management efficiency and cargo turnover speed.

[0067] Furthermore, the millimeter-wave radar assembly 400 includes a wedge-shaped bracket 401 and a dual-frequency radar chip set 402; the inclined surface of the wedge-shaped bracket 401 forms a 45° angle with the side wall of the housing 100, and the interior of the bracket is filled with an aluminum nitride thermal pad 405; the dual-frequency radar chip set 402 is attached to the inner surface of the wedge-shaped bracket 401 by a thermally conductive adhesive layer 403, and a waveguide antenna array 404 is welded to its surface; the waveguide antenna array 404 is arranged in an interleaved manner, with the spacing between adjacent antenna elements being λ / 4, where λ is the operating wavelength of the 24GHz band, and the antenna radiation efficiency is ≥85%.

[0068] In a specific embodiment, within an intelligent transportation system, when the device is deployed at an urban road intersection, the wedge-shaped bracket 401 is installed at a 45° angle on the side wall of the housing 100, ensuring that the dual-frequency radar chipset 402 can accurately monitor passing vehicles. In intersection environments with dense traffic and complex electromagnetic interference, its waveguide antenna array 404 is arranged in an interleaved manner, with the spacing between adjacent antenna elements strictly controlled at λ / 4, ensuring that the antenna radiation efficiency remains stable at over 85%. Field tests show that in traffic scenarios with vehicle speeds up to 120 km / h, the millimeter-wave radar component 400 has a vehicle speed measurement error of only ±0.5 km / h and a ranging accuracy of ±2 cm, providing accurate collision warnings 5 ​​seconds in advance, effectively ensuring traffic safety. In the field of industrial automation, taking an automated warehousing and logistics system as an example, the millimeter-wave radar component 400 is installed above an AGV (Automated Guided Vehicle) to monitor obstacles on the vehicle's path in real time. The aluminum nitride thermal pad 405 filled within the wedge-shaped bracket 401 efficiently dissipates the heat generated by the dual-frequency radar chipset 402 during operation, ensuring stable long-term operation of the equipment in the high-frequency 24GHz band. Even under harsh warehouse conditions with humidity up to 80% and temperature fluctuations of ±15℃, this component maintains a reliability rate of over 99%, can identify obstacles as small as 2cm × 2cm, and has a response time of less than 50ms. This significantly improves the operational efficiency and safety of the AGV, prevents production stoppages due to collisions, and ensures the efficient operation of the warehousing and logistics system.

[0069] Furthermore, the binocular vision component 500 includes a split-type gimbal 501 and a polarizing filter 506; the horizontal rotation axis 502 of the split-type gimbal 501 adopts a worm gear mechanism with a rotation angle range of ±180°, and the pitch adjustment axis 503 has a built-in micro servo motor with a positioning accuracy of ±0.5°; the polarizing filter 506 is installed in front of the lens through a sliding mechanism 507 and is driven to switch by a micro linear motor 508 with a response time ≤50ms; eight sets of infrared supplementary lighting units 504 are distributed in a ring around the lens, each set of LED beads 505 has a power of 3W, and the illumination distance reaches 15m.

[0070] In a specific embodiment, the operator achieves precise visual monitoring of the target using the split-type gimbal 501 and polarizing filter 506 built into the binocular vision component 500. The horizontal rotation axis 502 of the split-type gimbal 501 employs a worm gear mechanism with a rotation angle range of ±180°, while the pitch adjustment axis 503 incorporates a micro servo motor with a positioning accuracy of ±0.5°, enabling flexible adjustment of the camera angle over a wide range to adapt to monitoring needs in different scenarios. The polarizing filter 506 is mounted on the front of the lens via a sliding mechanism 507 and is driven to switch by a micro linear motor 508, with a response time ≤50ms, allowing for rapid switching of the filter state and effectively reducing the impact of light reflection and scattering on image quality. Eight sets of infrared supplementary lighting units 504 are distributed in a ring around the lens, each with a 3W LED bead 505 and an illumination distance of up to 15m, providing sufficient supplementary lighting in low-light or no-light environments to ensure the camera acquires clear images. In the field of security monitoring, the binocular vision component 500 is installed at monitoring points in key areas, such as airports and train stations with high pedestrian traffic. Actual testing shows that under extremely low light conditions (below 1 lux), the infrared supplementary lighting unit 504 enables the camera to acquire images with a resolution of 1080p, achieving a target recognition accuracy of up to 95%. In environments with strong light interference, the polarizing filter 506 reduces reflected light intensity by 80%, increasing the accuracy of facial recognition under backlight conditions from the traditional 60% to over 90%. Simultaneously, the high-precision positioning capability of the split-type pan-tilt unit 501 allows the camera to maintain a stable image when tracking moving targets, reducing the target loss rate to less than 5%. In the field of industrial inspection, taking an automobile manufacturing production line as an example, the binocular vision component 500 is used for the assembly inspection of automotive parts. Its split-type pan-tilt unit 501 can flexibly adjust its angle, precisely aligning with each inspection station. At a high production line speed of 60 pieces per minute, the polarizing filter 506, in conjunction with a high-speed switching supplementary lighting unit, ensures that the image of each component is clearly captured. The dimensional measurement accuracy of the components reaches ±0.1mm, and the detection accuracy of assembly defects reaches 98%. Compared with traditional vision inspection systems, the component's inspection efficiency is increased by 40%, and the inspection accuracy is improved by 30%, effectively improving production efficiency and product quality. A comparison between traditional binocular vision components and the new binocular vision components is shown in Table 2.

[0071] Table 2 Comparison of traditional binocular vision components and novel binocular vision components

[0072] Types of protective measures Detection accuracy Response time supplemental lighting distance Gimbal positioning accuracy Traditional binocular vision components ±0.5mm 100ms 10m ±2° Novel binocular vision components ±0.1mm ≤50ms 15m ±0.5°

[0073] As shown in Table 2, under the same conditions, the new binocular vision component exhibits higher detection accuracy, shorter response time, longer illumination distance, and higher gimbal positioning accuracy compared to the traditional component. The improved detection accuracy is attributed to the optimized design of the polarization filter and infrared illumination unit, while the reduced response time is due to the rapid drive of the miniature linear motor. The increased illumination distance and gimbal positioning accuracy enhance the adaptability and stability of the device in complex environments. In conclusion, the use of the new binocular vision component significantly improves the performance and reliability of the target recognition and tracking system.

[0074] Furthermore, the ultrasonic sensor group 600 includes a transceiver sensor 601 and an acoustic waveguide 602; the transceiver sensor 601 is arranged at 45° intervals, operates at a frequency of 40kHz, and has a detection blind zone ≤5cm; the acoustic waveguide 602 is a hollow aluminum tube with a wall thickness of 1.2mm, with the open end tilted outward at 30°, and the inner wall of the tube is coated with a polyurethane foam layer 604 with a sound absorption coefficient ≥0.9; a silicone vibration isolation washer 603 is provided between the sensor 601 and the housing 100 contact surface, and the washer has a Shore hardness of 60HA±5.

[0075] In a specific embodiment, the operator uses the transceiver sensor 601 and the acoustic waveguide 602 built into the ultrasonic sensor group 600 to achieve precise distance monitoring of the surrounding environment. The transceiver sensor 601 is arranged at 45° intervals, operates at a frequency of 40kHz, and has a detection blind zone of only ≤5cm, enabling rapid and accurate detection of the distance information of surrounding objects. The acoustic waveguide 602 is a hollow aluminum tube with a wall thickness of 1.2mm, with the open end tilted outward at 30° to effectively guide the transmission and reception direction of sound waves, reducing sound wave scattering and interference. The inner wall of the tube is coated with a polyurethane foam layer 604 with a sound absorption coefficient ≥0.9, which can effectively absorb stray sound waves and improve the purity and accuracy of the signal. A silicone vibration isolation washer 603 is provided between the contact surface of the sensor 601 and the housing 100. The washer has a Shore hardness of 60HA±5, which can effectively isolate vibrations during equipment operation and ensure stable operation of the sensor. In intelligent warehousing and logistics systems, ultrasonic sensor arrays 600 are installed around the Automated Guided Vehicle (AGV) to monitor the distance between the vehicle and surrounding shelves, pallets, and other obstacles in real time. Actual testing showed that in dynamic environments with vehicle speeds up to 2 m / s, the sensor array achieves obstacle detection accuracy of ±2 mm and can issue a collision warning signal 0.5 seconds in advance, providing sufficient time for the AGV to brake or adjust its path. In narrow passages, the sensor array can accurately identify obstacles as small as 10 cm in width, ensuring smooth passage of the AGV in complex warehousing environments. Compared to traditional ultrasonic sensors, its detection accuracy is improved by 3 times and its response speed is increased by 2 times, effectively reducing the rate of goods collision damage and improving warehousing and logistics efficiency. (See Table 3).

[0076] Table 3 Comparison between traditional and novel ultrasonic sensor arrays

[0077] Types of protective measures Detection blind spots Detection accuracy Operating frequency Response time Traditional ultrasonic sensor array ≤10cm ±5mm 20kHz 100ms New ultrasonic sensor array ≤5cm ±2mm 40kHz ≤50ms

[0078] As shown in Table 3, under the same conditions, the new ultrasonic sensor array has a smaller detection blind zone, higher detection accuracy, higher operating frequency, and shorter response time. The reduction in the detection blind zone is attributed to the optimized design of the acoustic waveguide, while the improvement in detection accuracy and response time is due to the enhanced performance of the transceiver sensor. The increased operating frequency enhances the device's anti-interference capability in complex environments. In conclusion, the use of the new ultrasonic sensor array significantly improves the environmental perception capability and operational safety of the target recognition and tracking system.

[0079] Furthermore, the power module 700 includes a parallel power supply unit 701 and an electromagnetic shielding chamber 704; the positive input terminal of the parallel power supply unit 701 is connected to a supercapacitor group 702, and the negative input terminal is connected to a lithium iron battery group 703, with a switching delay ≤10μs; the electromagnetic shielding chamber 704 is welded from 1.5mm thick permalloy plate and filled with boron nitride thermally conductive filler 705, with a filler porosity ≤5%; quick-release terminals 706 are provided on the outside of the shielding chamber 704, with contacts made of gold-plated copper alloy and a contact resistance ≤0.1mΩ.

[0080] In a specific embodiment, the operator utilizes the parallel power supply unit 701 and electromagnetic shielding chamber 704 built into the power module 700 to achieve highly reliable power supply and electromagnetic protection for the target identification and tracking device. The positive input terminal of the parallel power supply unit 701 is connected to a supercapacitor group 702, capable of providing a large current instantaneously to ensure the device's power needs during startup or emergencies; the negative input terminal is connected to a lithium iron battery group 703, providing stable long-term power support for the device. The switching delay is ≤10μs, ensuring seamless takeover of the other power supply in case of a power failure, guaranteeing uninterrupted operation of the device. The electromagnetic shielding chamber 704 is constructed using 1.5mm thick permalloy plate, effectively isolating external electromagnetic interference and internal electromagnetic leakage, ensuring stable operation of electronic components. The internally filled boron nitride thermally conductive filler 705, with a porosity ≤5%, not only has excellent thermal conductivity but also further enhances the electromagnetic shielding effect, ensuring uniform temperature distribution within the chamber and preventing component damage due to localized overheating. In industrial automation equipment, such as large-scale automated production lines, the power module 700 is installed in the core control unit of the equipment. During long-term operation, the parallel power supply unit 701 effectively copes with grid fluctuations and sudden power outages. In actual testing, when one power supply suddenly fails, the other power supply can quickly take over within ≤10μs, ensuring continuous equipment operation and avoiding production interruptions and economic losses caused by power outages. The electromagnetic shielding chamber 704 achieves a shielding effectiveness of over 80dB in environments with strong electromagnetic interference sources (such as high-power motors and high-frequency welding equipment), ensuring that the electronic components of the control unit are not interfered with and operate stably. Simultaneously, the boron nitride thermally conductive filler 705 has a thermal conductivity of up to 10W / m·K, effectively reducing component temperature, extending service life, and improving equipment reliability.

[0081] Furthermore, the dual-band antenna 801 of the communication module 800 includes a double-layer microstrip patch 802 and a phase tuning cavity 804; the upper layer of the double-layer microstrip patch 802 is a 2.4GHz radiating element 803, and the lower layer is a 5.8GHz radiating element 805, with a radiation efficiency ≥92%; the phase tuning cavity 804 is integrated on the back of the PCB substrate, and the cavity is filled with a ceramic composite material 806 with a dielectric constant of 3.5, and the cavity volume is 15mm³±0.5mm³; the junction between the antenna edge and the housing 100 is wrapped with a fluororubber sealing ring 807, and the compression permanent deformation rate of the sealing ring is ≤10%.

[0082] In a specific embodiment, the communication module 800 achieves efficient and stable communication functions through its dual-band antenna 801. The upper 2.4GHz radiating element 803 and the lower 5.8GHz radiating element 805 of the dual-layer microstrip patch 802 work together to cover a wide frequency range, ensuring high data transmission speed and high stability. Actual testing shows that even in a crowded shopping mall environment with strong electromagnetic interference, the antenna maintains a radiation efficiency of ≥92%, ensuring a stable data transmission rate of over 150Mbps and effectively reducing the data packet loss rate to within 2%. In a smart factory environment, the communication module is installed on automated equipment in the workshop. Given the presence of numerous metal devices and complex wiring within the workshop, the dual-band antenna, through the optimized design of the phase-tuning cavity 804, effectively enhances signal directivity and coverage. The ceramic composite material 806 filling the phase-tuning cavity ensures stable signal transmission, enabling communication distances between devices to exceed 100 meters, meeting the needs of long-distance communication between devices within the workshop. In a continuous 72-hour stability test, the signal strength fluctuation of the communication module was less than ±3dB, ensuring the continuity and reliability of production data transmission. In intelligent transportation systems, the communication module is used in road monitoring equipment. The 807 fluororubber sealing ring at the junction of the antenna edge and the housing ensures the sealing performance of the equipment in complex outdoor environments. Actual testing showed that the equipment achieved an IP67 waterproof rating under heavy rain conditions, ensuring stable operation of the antenna system. Even in strong winds reaching 30m / s, the communication module maintained a stable communication connection, with a data transmission error rate of less than 1×10⁻⁻⁻⁶. 6 This ensures the reliability of real-time uploading and remote monitoring of traffic data.

[0083] While specific embodiments of this utility model have been described above, those skilled in the art should understand that these specific embodiments are merely illustrative. Those skilled in the art can omit, substitute, and modify the details of the above methods and systems in various ways without departing from the principles and essence of this utility model. For example, combining the above method steps to perform substantially the same function and achieve substantially the same result according to substantially the same method falls within the scope of this utility model. Therefore, the scope of this utility model is defined only by the appended claims.

Claims

1. A high-speed real-time target recognition and tracking device based on multimodal sensor fusion, comprising a housing (100), characterized in that: The central processing module (200) is fixed in the middle of the housing (100). The central processing module (200) is provided with heat dissipation fins (201) on its upper part and is connected to the inner wall of the housing (100) through a thermally conductive silicone layer (202). A lidar assembly (300) is embedded in the front end of the housing (100), and the emitting surface of the lidar assembly (300) is at an elevation angle of 15°-30° to the outer surface of the housing (100). The millimeter-wave radar components (400) are symmetrically distributed on both sides of the housing (100), and the detection direction is perpendicular to the scanning plane of the lidar component (300). The top of the housing (100) is provided with a binocular vision component (500), and the lens axis is parallel to the central axis of the lidar component (300); The bottom edge of the housing (100) is circularly distributed with an ultrasonic sensor group (600), and the distance between adjacent sensors is 5-8cm; The central processing module (200) integrates a data fusion unit (203). The output terminals of the lidar component (300), millimeter-wave radar component (400) and binocular vision component (500) are connected to the input terminal of the data fusion unit (203) via a high-speed parallel bus (204). The power module (700) is mounted on the rear of the housing (100) via an isolation bracket. The output of the power module (700) is connected to the input of the central processing module (200) and the data fusion unit (203). The PCB substrate of the communication module (800) is embedded in the groove on the side wall of the housing (100), and the dual-band antenna (801) of the communication module (800) radiates towards the front of the device.

2. The high-speed real-time target recognition and tracking device based on multimodal sensor fusion according to claim 1, characterized in that: The lidar assembly (300) includes a rotating base (301) and a scanning head (302); the rotating base (301) has a built-in stepper motor (303), the rotating shaft is hinged to the front end of the housing (100), and the rotation accuracy of the stepper motor (303) is ±0.1°; the scanning head (302) is fixed to the rotating base (301) by a magnetic snap-fit ​​(304), and integrates a 16-line laser emitting array (305) and a photoelectric receiving unit (306); the scanning head (302) is covered with a high-transmittance dust cover (307), and the surface of the dust cover (307) is provided with a guide groove (308) with a depth of 1.2mm and a groove spacing of 3mm.

3. The high-speed real-time target recognition and tracking device based on multimodal sensor fusion according to claim 1, characterized in that: The millimeter-wave radar assembly (400) includes a wedge-shaped bracket (401) and a dual-frequency radar chip group (402); the inclined surface of the wedge-shaped bracket (401) forms a 45° angle with the side wall of the housing (100), and the bracket is filled with an aluminum nitride thermal pad (405); the dual-frequency radar chip group (402) is attached to the inner surface of the wedge-shaped bracket (401) by a thermally conductive adhesive layer (403), and a waveguide antenna array (404) is welded to its surface; the waveguide antenna array (404) is arranged in an interleaved manner, with the spacing between adjacent antenna elements being λ / 4, where λ is the operating wavelength of the 24GHz band, and the antenna radiation efficiency is ≥85%.

4. The high-speed real-time target recognition and tracking device based on multimodal sensor fusion according to claim 1, characterized in that: The binocular vision component (500) includes a split-type gimbal (501) and a polarizing filter (506); the horizontal rotation axis (502) of the split-type gimbal (501) adopts a worm gear mechanism with a rotation angle range of ±180°, and the pitch adjustment axis (503) has a built-in micro servo motor with a positioning accuracy of ±0.5°; the polarizing filter (506) is installed in front of the lens through a sliding mechanism (507) and is driven to switch by a micro linear motor (508) with a response time ≤50ms; the lens is surrounded by 8 sets of infrared supplementary light units (504) with a power of 3W per LED bead (505) and an illumination distance of up to 15m.

5. The high-speed real-time target recognition and tracking device based on multimodal sensor fusion according to claim 1, characterized in that: The ultrasonic sensor group (600) includes a transceiver sensor (601) and an acoustic waveguide (602); the transceiver sensor (601) is arranged at 45° intervals, with a working frequency of 40kHz and a detection blind zone of ≤5cm; the acoustic waveguide (602) is a hollow aluminum tube with a wall thickness of 1.2mm, with the open end tilted outward at 30°, and the inner wall of the tube is coated with a polyurethane foam layer (604) with a sound absorption coefficient ≥0.9; a silicone vibration isolation washer (603) is provided between the sensor (601) and the housing (100) contact surface, and the washer has a Shore hardness of 60HA±5.

6. The high-speed real-time target recognition and tracking device based on multimodal sensor fusion according to claim 1, characterized in that: The power module (700) includes a parallel power supply unit (701) and an electromagnetic shielding chamber (704); the positive input terminal of the parallel power supply unit (701) is connected to a supercapacitor group (702), and the negative input terminal is connected to a lithium iron battery group (703), with a switching delay ≤10μs; the electromagnetic shielding chamber (704) is welded with 1.5mm thick permalloy plate and filled with boron nitride thermally conductive filler (705), with a filler porosity ≤5%; quick-release terminals (706) are provided on the outside of the shielding chamber (704), and the contacts are made of gold-plated copper alloy with a contact resistance ≤0.1mΩ.

7. The high-speed real-time target recognition and tracking device based on multimodal sensor fusion according to claim 1, characterized in that: The dual-band antenna (801) of the communication module (800) includes a double-layer microstrip patch (802) and a phase tuning cavity (804); the upper layer of the double-layer microstrip patch (802) is a 2.4GHz radiating element (803), and the lower layer is a 5.8GHz radiating element (805), with a radiation efficiency ≥92%; the phase tuning cavity (804) is integrated on the back of the PCB substrate, and the cavity is filled with a ceramic composite material (806) with a dielectric constant of 3.5, and the cavity volume is 15mm³±0.5mm³; the junction between the antenna edge and the housing (100) is wrapped with a fluororubber sealing ring (807), and the compression permanent deformation rate of the sealing ring is ≤10%.

Images