Intelligent blind area early warning system for large engineering vehicle

Abstract

The application relates to a large engineering vehicle blind area intelligent early warning system, which comprises a master control unit, a millimeter wave radar module, an ultrasonic wave radar module, a camera module and a display alarm module; the master control unit is connected with the millimeter wave radar module, the ultrasonic wave radar module, the camera module and the display alarm module; the millimeter wave radar module is used for acquiring distance, speed and angle information of an obstacle; the ultrasonic wave radar module is used for acquiring distance information of a near-distance obstacle; the camera module is used for acquiring an image and a video signal of a vehicle periphery; the master control unit is configured to receive data of the modules, identify static and dynamic obstacles in a blind area through a multi-source information fusion algorithm, and control the display alarm module to output early warning information; the application provides a stable and reliable blind area detection scheme, can cover a visual blind area of a large vehicle, improves the safety of the engineering vehicle during operation, and promotes the development of intelligent management.

Description

Technical Field

[0001] This invention belongs to the field of intelligent transportation and vehicle safety technology, and in particular relates to an intelligent early warning system for blind spots of large engineering vehicles. Background Technology

[0002] With the continuous acceleration of urbanization and the ongoing advancement of infrastructure construction, the frequency of use of engineering vehicles at various construction sites has increased significantly, such as excavators, bulldozers, and concrete mixer trucks.

[0003] However, construction vehicles often operate in narrow and complex environments, which easily creates blind spots. Blind spots refer to areas that the driver cannot directly observe during operation. This phenomenon is particularly pronounced in large construction vehicles, seriously affecting safety at construction sites. For example, while a construction vehicle is in motion, the driver has difficulty visually observing the environment to the right and rear of the vehicle. When the vehicle turns right or reverses, if there are cyclists or pedestrians in the blind spot, it can easily lead to a collision, causing irreversible harm to vulnerable groups on the road.

[0004] Therefore, developing an efficient and reliable blind spot detection system is of great significance for improving the safety and operational efficiency of engineering vehicles and ensuring the safety of construction workers and pedestrians in the surrounding area. Summary of the Invention

[0005] To solve the above-mentioned technical problems, the present invention adopts the following technical solution:

[0006] This invention provides an intelligent early warning system for blind spots of large engineering vehicles, comprising: a main control unit, a millimeter-wave radar module, an ultrasonic radar module, a camera module, and a display and alarm module;

[0007] The main control unit is connected to the millimeter-wave radar module, the ultrasonic radar module, the camera module, and the display and alarm module, respectively.

[0008] The millimeter-wave radar module is used to acquire distance, speed, and angle information of obstacles;

[0009] The ultrasonic radar module is used to acquire distance information of nearby obstacles;

[0010] The camera module is used to acquire images and video signals around the vehicle;

[0011] The main control unit is configured to receive data from the above modules, identify static and dynamic obstacles in the blind zone through a multi-source information fusion algorithm, and control the display alarm module to output early warning information.

[0012] Preferably, the main control unit adopts an MCU chip with multiple communication serial interfaces and is connected to a serial interface expansion module;

[0013] The serial interface expansion module is used to uniformly receive the output data of multiple ultrasonic radar modules, and send the integrated data to the main control unit or millimeter-wave radar chip for processing through the serial interface, so as to solve the problem of the limited number of serial interfaces of millimeter-wave radar chips.

[0014] Preferably, the multi-source information fusion algorithm includes coordinate transformation and spatial fusion steps:

[0015] Establish radar coordinate system, world coordinate system, camera coordinate system, image coordinate system, and pixel coordinate system;

[0016] After camera calibration is completed, the target points detected by millimeter-wave radar and ultrasonic radar are mapped to the image coordinate system through a coordinate transformation matrix, thereby achieving spatial alignment and fusion of radar data and video images.

[0017] Preferably, the main control unit runs a PV-DBSCAN point cloud clustering algorithm based on polar coordinate voxelization, which is used to process point cloud data from millimeter-wave radar.

[0018] The PV-DBSCAN point cloud clustering algorithm uses a polar coordinate spatial distance function to replace the Euclidean distance in the Cartesian coordinate system, and divides the radar point cloud into polar coordinate voxel grids. When searching for neighboring points, it only calculates the points in the grid where the target point is located and the points in the surrounding grids.

[0019] Preferably, the polar coordinate spatial distance function introduces a polar coordinate angular distance weight parameter, and the distance between two points is defined as a weighted sum of the radial distance difference and the angular distance difference.

[0020] Preferably, the main control unit runs a multi-target association algorithm based on bipartite graph matching, which uses the Hungarian algorithm to solve the minimum weight matching problem of multiple targets in the millimeter-wave radar point cloud of two consecutive frames;

[0021] When the intersection-union ratio (IUU) between a target and the detection boxes of all existing target prediction results is less than a specified threshold, a new target ID is created for it. When a target ID is not matched for several consecutive frames, it is destroyed.

[0022] Preferably, the main control unit runs a target tracking module based on a filtering algorithm;

[0023] The associated target tracking module establishes the motion state equation and observation equation of the millimeter-wave radar point cloud target, and uses an extended Kalman filter to approximate the nonlinear relationship into a linear relationship through Taylor series expansion, so as to generate a smooth motion trajectory of the associated target.

[0024] Preferably, it also includes a trajectory management module, which divides the target trajectory into idle state, detection state and active state;

[0025] When a new target is detected and cannot be associated with an old target, the trajectory instance enters the detection state.

[0026] Once a target in the detection state is successfully associated with multiple consecutive frames, it transitions to the active state.

[0027] When the number of failed association frames of an active trajectory instance exceeds a predetermined threshold, the tracking trajectory is deleted and enters an idle state.

[0028] Preferably, it also includes a temperature compensation module, which is used to monitor the ambient temperature in real time and correct the ultrasonic ranging velocity.

[0029] Preferably, the display alarm module includes a character fusion module and an alarm unit;

[0030] The character fusion module fuses radar distance data, speed data and camera video signals to generate a video signal containing obstacle information.

[0031] The alarm unit provides graded warnings at different alarm frequencies based on the comparison between the distance to the obstacle and a preset threshold.

[0032] The present invention has at least the following beneficial effects:

[0033] This invention provides a stable and reliable blind spot detection solution that can cover the blind spots of large vehicles, thereby improving the safety of engineering vehicles during operation and promoting the development of intelligent management.

[0034] This invention utilizes ultrasound to fill the near-field and ground clutter blind spots of millimeter-wave radar, and uses millimeter-wave radar to compensate for the long-range and velocity measurement deficiencies of ultrasound, significantly improving the reliability and detection range of the system.

[0035] This invention effectively solves the clustering problem of long-distance sparse point clouds in engineering scenarios through the PV-DBSCAN algorithm, and improves the continuity and stability of target tracking through a dynamic trajectory management mechanism.

[0036] This invention incorporates a temperature compensation mechanism to ensure ranging accuracy under different climatic conditions and has strong environmental adaptability. Attached Figure Description

[0037] Figure 1 This is a schematic diagram of a blind spot intelligent early warning system for large engineering vehicles according to an embodiment of the present invention.

[0038] Figure 2 This is the main flowchart of the intelligent early warning system for blind spots of large engineering vehicles according to an embodiment of the present invention.

[0039] Figure 3 This is a coordinate transformation diagram of the intelligent early warning system for blind spots of large engineering vehicles according to an embodiment of the present invention.

[0040] Figure 4 This is a spatial fusion diagram of the intelligent early warning system for blind spots of large engineering vehicles according to an embodiment of the present invention.

[0041] Figure 5 This is a flowchart illustrating the target association and matching process of the intelligent early warning system for blind spots of large engineering vehicles according to an embodiment of the present invention.

[0042] Figure 6 This is a flowchart illustrating the temperature compensation and correction process of the intelligent early warning system for blind spots of large engineering vehicles, as described in an embodiment of the present invention.

[0043] Figure 7 This is a flowchart illustrating the sound and light alarm and distance display of the intelligent blind spot warning system for large engineering vehicles, as described in an embodiment of the present invention.

[0044] Figure 8 This is a distance calculation diagram of the DBSCAN clustering algorithm for the intelligent early warning system for blind spots of large engineering vehicles according to an embodiment of the present invention.

[0045] Figure 9 This is a polar coordinate voxel grid division diagram of the point cloud of the intelligent early warning system for blind spots of large engineering vehicles according to an embodiment of the present invention.

[0046] Figure 10 This is a schematic diagram of the millimeter-wave radar measuring moving targets process of the intelligent early warning system for blind spots of large engineering vehicles according to an embodiment of the present invention.

[0047] Figure 11 This is a schematic diagram of the state switching of the trajectory management module of the intelligent early warning system for blind spots of large engineering vehicles according to an embodiment of the present invention. Detailed Implementation

[0048] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0049] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, apparatus, product, or device that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or devices.

[0050] The embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0051] like Figure 1 As shown, this invention provides an intelligent blind spot warning system for large engineering vehicles, comprising: a main control unit, a millimeter-wave radar module, an ultrasonic radar module, a camera module, and a display alarm module; the main control unit is connected to the millimeter-wave radar module, the ultrasonic radar module, the camera module, and the display alarm module respectively; the millimeter-wave radar module is used to acquire distance, speed, and angle information of obstacles; the ultrasonic radar module is used to acquire distance information of nearby obstacles; the camera module is used to acquire images and video signals around the vehicle; the main control unit is configured to receive data from the above modules, identify static and dynamic obstacles in the blind spot through a multi-source information fusion algorithm, and control the display alarm module to output warning information.

[0052] This invention provides a stable and reliable blind spot detection solution that can cover the blind spots of large vehicles, thereby improving the safety of engineering vehicles during operation and promoting the development of intelligent management.

[0053] In a preferred embodiment, the main control unit uses an MCU chip with multiple serial communication interfaces and is connected to a serial interface expansion module. The serial interface expansion module is used to uniformly receive the output data from multiple ultrasonic radar modules and send the integrated data to the main control unit or millimeter-wave radar chip for processing through the serial interface, thereby solving the problem of the limited number of serial interfaces of the millimeter-wave radar chip.

[0054] In a preferred embodiment, the multi-source information fusion algorithm includes coordinate transformation and spatial fusion steps: establishing a radar coordinate system, a world coordinate system, a camera coordinate system, an image coordinate system, and a pixel coordinate system; after completing camera calibration, the target points detected by the millimeter-wave radar and the ultrasonic radar are mapped to the image coordinate system through a coordinate transformation matrix, thereby achieving spatial alignment and fusion of radar data and video images.

[0055] In a preferred embodiment, the main control unit runs a PV-DBSCAN point cloud clustering algorithm based on polar coordinate voxels to process the point cloud data of millimeter-wave radar. The PV-DBSCAN point cloud clustering algorithm uses a polar coordinate spatial distance function to replace the Euclidean distance in the Cartesian coordinate system to divide the radar point cloud into polar coordinate voxel grids. When searching for neighboring points, it only calculates the points in the grid where the target point is located and the points in the surrounding grids.

[0056] In a preferred embodiment, the polar coordinate spatial distance function introduces a polar coordinate angular distance weight parameter, and the distance between two points is defined as the weighted sum of the radial distance difference and the angular distance difference.

[0057] In a preferred embodiment, the main control unit runs a multi-target association algorithm based on bipartite graph matching. The multi-target association algorithm uses the Hungarian algorithm to solve the minimum weight matching problem of multiple targets in the millimeter-wave radar point cloud of two consecutive frames. When the cross-union ratio between a target and the detection boxes of all existing target prediction results is less than a specified threshold, a new target ID is created for it. When the matching of the tracked target is not achieved for several consecutive frames, the target ID is destroyed.

[0058] In a preferred embodiment, the main control unit runs a target tracking module based on a filtering algorithm. The target tracking module establishes the motion state equation and observation equation of the millimeter-wave radar point cloud target, and uses an extended Kalman filter to approximate the nonlinear relationship into a linear relationship through Taylor series expansion, so as to generate a smooth motion trajectory of the target.

[0059] In a preferred embodiment, the system further includes a trajectory management module, which divides the target trajectory into an idle state, a detection state, and an active state. When a new target is detected and cannot be associated with an old target, the trajectory instance enters the detection state. When a target in the detection state is successfully associated with multiple consecutive frames, it transitions to the active state. When the number of failed association frames for a trajectory instance in the active state exceeds a predetermined threshold, the tracking trajectory is deleted and the system transitions to the idle state.

[0060] In a preferred embodiment, a temperature compensation module is also included, which is used to monitor the ambient temperature in real time and correct the ultrasonic ranging velocity.

[0061] In a preferred embodiment, the alarm display module includes a character fusion module and an alarm unit. The character fusion module fuses radar distance data, speed data, and camera video signals to generate a video signal containing obstacle information. The alarm unit provides graded warnings at different alarm frequencies based on the comparison results between the obstacle distance and a preset threshold.

[0062] Millimeter-wave radar has a large detection range and can acquire information such as the position, speed, and trajectory of multiple targets. However, it is difficult to cover all blind spots behind the vehicle and is easily interfered with by ground reflection clutter, affecting obstacle detection performance and reliability. Ultrasonic radar can only acquire obstacle distance information, not object position information, and cannot distinguish multiple obstacles. It is inexpensive, has stable performance, and is not affected by ground reflection clutter, but its detection range is relatively short. Since the two types of radar have complementary advantages in detection performance, a fusion scheme of ultrasonic and millimeter-wave radar can be adopted to achieve more stable, reliable, and intelligent blind spot detection in large vehicle driving scenarios. Considering the limited number of serial interfaces of millimeter-wave radar chips, an additional serial interface expansion module is used to uniformly receive ultrasonic radar information. All received ultrasonic radar detection information is then aggregated and sent to the control unit through the serial interface.

[0063] Ultrasonic sensors located at the rear of the vehicle acquire distance data, while millimeter-wave sensors acquire information such as obstacle speed and angle. This data, along with images and video data acquired by cameras, is transmitted to a character fusion module. Through multi-source information fusion technology, a PAL video signal is generated. After PAL signal decoding, the signal is transmitted to a TW2824 control chip for signal transcoding and spatial conversion. Finally, the signal is connected to an LCD display module via the main controller to display the PAL signal containing obstacle data.

[0064] To meet the serial interface expansion requirements between millimeter-wave radar and ultrasonic radar, this project plans to use an STM32F407 series MCU chip with five serial communication interfaces, a main frequency of 170MHz, and a 32-bit ARM4 processor, which can achieve adaptive acceleration via Flash. This chip includes three Universal Synchronous / Asynchronous Receivers (USARTs) and two UARTs. During operation, the serial interface expansion module receives the output data from the ultrasonic radar, integrates it, and sends it to the millimeter-wave radar chip for processing.

[0065] The STM32F407 chip operates on a power supply voltage range of 1.7V to 3.6V, includes a clock crystal oscillator ranging from 4MHz to 26MHz, and features up to 2MB of Flash storage with fully synchronous read and write capabilities. It contains up to 256MB of SRAM, including 64KB of core-coupled memory, and a 32-bit data bus for simultaneous data transmission and exchange. Externally, it supports various types of external memory, such as SRAM, PSRAM, and SDRAM, to meet different read / write speed and access permission requirements.

[0066] The IWR series millimeter-wave radar chips operate at frequencies from 60GHz to 64GHz. The chip integrates an on-chip antenna, eliminating the need for additional antenna design. Its built-in CPU is an ARM R4F, with a built-in FFT accelerator and a dedicated DSP to accelerate the processing of millimeter-wave radar signals. This effectively reduces the size of the blind zone detection system while improving hardware integration, facilitating testing, installation, and subsequent product manufacturing.

[0067] The ultrasonic radar module primarily transmits and receives ultrasonic signals, and the controller calculates the ultrasonic propagation time based on the time difference between transmission and reception to perform distance measurement. Because ultrasonic signals attenuate during propagation, they are relatively weak. Therefore, the ultrasonic receiving circuit typically amplifies, filters, and processes the signal through peak detection and shaping circuits before sending it to the controller. The HC-SR04M series ultrasonic radar uses a waterproof wired transceiver probe, integrating amplification and filtering circuits, resulting in advantages such as small size and low power consumption, making it suitable for short-range obstacle detection.

[0068] The process of processing multi-source data in this invention is as follows: Figure 2 As shown, the radar target data and video image data are first transformed into coordinates, and then time and space are fused to determine the position of the monitored target in the video display. The alarm program is then called to determine whether the threshold is exceeded.

[0069] Coordinate transformation relationship as follows Figure 3 As shown, coordinate transformation can spatially correlate the same target detected by both cameras and radar sensors; this process is called spatial fusion. Spatial fusion involves the radar coordinate system. World coordinate system Camera coordinate system Image coordinate system and pixel coordinate system The mutual conversion between them.

[0070] The process of spatial integration is as follows Figure 4 As shown, after camera calibration is completed, spatial fusion of the camera and radar can be achieved through coordinate transformation, laying the foundation for the final realization of multi-sensor fusion.

[0071] The target association and matching process is as follows: Figure 5As shown. When tracking targets, it is necessary to create IDs for new targets and delete the IDs of those that have disappeared. When the IoU between a target and the detection boxes of all existing target prediction results is less than a specified threshold, an ID is created for it, and it is considered a newly appeared target. If the predicted position and detection box of a tracked target do not match for several consecutive frames, the tracking ID is destroyed, and the target is considered to have disappeared.

[0072] Temperature compensation correction process as follows Figure 6 As shown, the ultrasonic ranging system incorporates a temperature correction module based on the DS18B20. This module measures the ambient temperature simultaneously with ultrasonic ranging to correct the sound velocity and reduce ranging errors. After the main control chip sends a temperature measurement command, the DS18B20 is activated. Once the temperature measurement is complete, the DS18B20 sends the temperature data back to the main control chip for sound velocity correction.

[0073] The system alarm and display process is as follows: Figure 7 As shown, audible and visual alarms and distance displays are the most direct manifestations of the blind spot warning system's warning function. Their stable and reliable operation is crucial to ensure the driver receives timely and accurate driving information. When the distance to an obstacle reaches the set alarm threshold, the audible and visual alarm system activates, and the alarm frequency varies depending on the threshold.

[0074] The blind zone target detection scheme based on radar information fusion has superior blind zone monitoring capabilities, effectively identifying and providing early warnings of obstacles at greater distances. Within the information fusion framework, the PV-DBSCAN clustering algorithm, based on polar coordinate voxels, is introduced. This algorithm uses bipartite graph matching to associate multiple targets and generate their motion trajectories, identifying static and dynamic obstacles within the blind zone and updating target status in real time, thus achieving efficient real-time monitoring and early warning.

[0075] The PV-DBSCAN (Polar-Voxel-based DBSCAN) clustering algorithm based on polar coordinate voxels improves the original DBSCAN algorithm in terms of both clustering accuracy and computational efficiency for long-range radar point clouds.

[0076] Firstly, addressing the issue of sparse point clouds in millimeter-wave radar for distant targets, PV-DBSCAN implements the DBSCAN algorithm in polar coordinates by using polar coordinate spatial distance. For example... Figure 8 As shown, the classic DBSCAN uses the Euclidean distance function in Cartesian coordinates, where the point... and The Cartesian coordinates are respectively and The polar coordinates are respectively and Then the Euclidean distance between the two points in the Cartesian coordinate system can be defined as:

[0077] .

[0078] To compensate for the sparsity of point clouds of distant targets, this study proposes a polar coordinate spatial distance:

[0079]

[0080] in This is the polar coordinate angular distance weighting parameter. By calculating this distance in polar coordinate space, the sparsity of radar point clouds of distant targets can be avoided from affecting the clustering effect.

[0081] Subsequently, to address the computational complexity of the DBSCAN algorithm, a voxel grid approach was adopted to improve the computational efficiency of the clustering algorithm. In the original DBSCAN algorithm, when searching for the neighborhood points of a point, it is necessary to calculate the distance of that point to all other points and determine whether the distance is less than the neighborhood radius, thus resulting in high time complexity. To reduce computational complexity, the radar point cloud is divided into polar coordinate voxel grids, such as... Figure 9 As shown, for each point whose neighborhood is to be searched, its voxel grid is represented as orange, and the grids surrounding it are represented as green. When searching for the neighborhood points of this point, it is only necessary to perform neighborhood point search on the points in the search area composed of the orange area and the green area.

[0082] The PV-DBSCAN point cloud clustering algorithm is used to cluster millimeter-wave radar point cloud data, establishing a correspondence between the point cloud and real targets. Based on the cluster centers, the spatial position of the real targets and their Doppler velocity relative to the millimeter-wave radar are estimated, thus achieving target detection. However, in practical vehicle-mounted millimeter-wave radar applications, it is necessary not only to detect targets in single frames of radar point cloud data but also to correlate and match multiple targets detected in multiple frames of radar point cloud data. This allows the system to monitor the surrounding environment in real time and provide early warnings before unexpected events occur.

[0083] The Hungarian algorithm is a combinatorial optimization algorithm that solves the task assignment problem in multinomial time. It is a concrete implementation of the global nearest neighbor data association idea and is often used to solve the minimum weight matching problem in weighted bipartite graphs. Assume the cost matrix is... The matrix represents the number of multiple targets in the point cloud of the millimeter-wave radar between two consecutive frames.

[0084] The specific steps of the Hungarian algorithm are as follows:

[0085] Subtract the minimum element of each row from the elements of the cost matrix, and each row will contain at least one 0.

[0086] Subtract the minimum element of each column from the elements of the cost matrix;

[0087] Cover all the zeros in the matrix with as few column or row markers as possible, and then check if the number of row and column markers is greater than 1. ;

[0088] If the number of markers is not less than If the condition is met, the algorithm terminates. Otherwise, retrieve the smallest element that is not covered by any row or column markings. Subtract all elements not covered by row and column markers Elements whose rows and columns are covered by the marker plus Then repeat the steps until the number of row and column markers is not less than .

[0089] In blind zone detection algorithms based on millimeter-wave radar, after target detection and association of millimeter-wave radar point clouds using PV-DBSCAN and the Hungarian algorithm, a filtering algorithm is needed to generate the motion trajectory of the associated targets to achieve target tracking. Furthermore, since the vehicle-mounted millimeter-wave radar moves continuously with the vehicle, new targets will constantly enter the detection area, and previously detected targets will constantly leave the detection area. Therefore, trajectory management of multi-target tracking results is required.

[0090] First, it is necessary to establish the state equations and observation equations for the motion of targets in millimeter-wave radar point clouds. The process of millimeter-wave radar measuring moving targets is as follows: Figure 10 As shown.

[0091] In practical applications of vehicle-mounted millimeter-wave radar, the motion of most targets can be simplified to uniform linear motion within a short time. We can assume the target at time [time value missing]. The state vector is:

[0092]

[0093] in , and The target is respectively in The three-dimensional spatial coordinates at time [time]. , and Let be the velocities of the target in the three axial directions. The state update interval, i.e., the sampling interval of the millimeter-wave radar, is the time interval at which the target is located. and The relationship between the states of motion at any given time can be expressed as:

[0094]

[0095] Therefore, the state equation for the target is:

[0096]

[0097] in for The process noise vector at time t, and its corresponding noise covariance is: , used to describe the actual state of the system.

[0098] Deviation from the motion model, Let be the state transition matrix.

[0099] Because millimeter-wave radar uses a polar coordinate system for measurement, therefore its... The measurement vector of the target at time t can be written as:

[0100]

[0101] Using millimeter-wave radar as the center of polar coordinates, This represents the radial distance between the target and the radar. This represents the azimuth of the target. This represents the elevation angle of the target. This represents the radial velocity of the target, i.e., the Doppler velocity relative to the radar.

[0102] The relationship between the measurement vector and the state vector can be described by the measurement equation:

[0103]

[0104] in Represents the measurement matrix. To measure the noise matrix.

[0105] Kalman filtering is a linear filtering method, where the target's motion state vector... Measurement vector of the target by millimeter-wave radar The relationship between them is nonlinear, so an extended Kalman filter (EKF) is used to track the detected target. The nonlinear filtering is approximated as linear filtering by Taylor series expansion.

[0106] First-order EKF by preserving The first term of the Taylor expansion is used to linearize the state vector. With measurement vector The relationship between them. Regarding the prior state estimation vector The first-order Taylor expansion is:

[0107]

[0108] Where the prior state estimation vector Representative based on The posterior estimate at time t The prior estimate obtained by predicting the state at time step. This represents the Jacobian matrix.

[0109] Therefore, the state vector The approximate linear relationship between the measurement vector and the measurement vector is:

[0110]

[0111] The target's trajectory is obtained using the extended Kalman filter described above.

[0112] Considering the need to design a reliable trajectory management module in actual testing, it is necessary to save the tracking trajectory to avoid tracking loss, create a new tracking trajectory when a new tracking target is encountered, and delete the trajectory after the target leaves the detection range.

[0113] In this trajectory management module, the target trajectory can be divided into three states: idle state, detection state, and active state. The state switching process is as follows: Figure 11 As shown. The idle state indicates that the instance has no corresponding tracking target and trajectory. When a new detected target appears, and this target cannot be associated with the previous tracking target, the instance enters the detection state, indicating that a new tracking target may have appeared, but it has not yet been recognized as a real tracking target. When a target in the detection state is successfully associated for several consecutive frames, it is recognized as a real tracking target and enters the active state. If the number of failed association frames for a trajectory instance in the active state exceeds a predetermined threshold, the tracking trajectory is deleted and the instance enters the idle state.

[0114] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0115] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0116] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0117] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0118] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the specific implementation of the present invention. Any modifications or equivalent substitutions that do not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A blind spot intelligent early warning system for large engineering vehicles, characterized in that, include: Main control unit, millimeter-wave radar module, ultrasonic radar module, camera module, display and alarm module; The main control unit is connected to the millimeter-wave radar module, the ultrasonic radar module, the camera module, and the display and alarm module, respectively. The millimeter-wave radar module is used to acquire distance, speed, and angle information of obstacles; The ultrasonic radar module is used to acquire distance information of nearby obstacles; The camera module is used to acquire images and video signals around the vehicle; The main control unit is configured to receive data from the above modules, identify static and dynamic obstacles in the blind zone through a multi-source information fusion algorithm, and control the display alarm module to output early warning information.

2. The intelligent early warning system for blind spots of large engineering vehicles according to claim 1, characterized in that, The main control unit uses an MCU chip with multiple serial communication interfaces and is connected to a serial interface expansion module. The serial interface expansion module is used to uniformly receive the output data of multiple ultrasonic radar modules, and send the integrated data to the main control unit or millimeter-wave radar chip for processing through the serial interface, so as to solve the problem of the limited number of serial interfaces of millimeter-wave radar chips.

3. The intelligent early warning system for blind spots of large engineering vehicles according to claim 1, characterized in that, The multi-source information fusion algorithm includes coordinate transformation and spatial fusion steps: Establish radar coordinate system, world coordinate system, camera coordinate system, image coordinate system, and pixel coordinate system; After camera calibration is completed, the target points detected by millimeter-wave radar and ultrasonic radar are mapped to the image coordinate system through a coordinate transformation matrix, thereby achieving spatial alignment and fusion of radar data and video images.

4. The intelligent early warning system for blind spots of large engineering vehicles according to claim 1, characterized in that, The main control unit runs a PV-DBSCAN point cloud clustering algorithm based on polar coordinate voxelization, which is used to process point cloud data from millimeter-wave radar. The PV-DBSCAN point cloud clustering algorithm uses a polar coordinate spatial distance function to replace the Euclidean distance in the Cartesian coordinate system, and divides the radar point cloud into polar coordinate voxel grids. When searching for neighboring points, it only calculates the points in the grid where the target point is located and the points in the surrounding grids.

5. The intelligent early warning system for blind spots of large engineering vehicles according to claim 4, characterized in that, The polar coordinate spatial distance function introduces a polar coordinate angular distance weight parameter, and the distance between two points is defined as the weighted sum of the radial distance difference and the angular distance difference.

6. The intelligent early warning system for blind spots of large engineering vehicles according to claim 1, characterized in that, The main control unit runs a multi-target association algorithm based on bipartite graph matching. The multi-target association algorithm uses the Hungarian algorithm to solve the minimum weight matching problem of multiple targets in the millimeter-wave radar point cloud of two frames. When the intersection-union ratio (IUU) between a target and the detection boxes of all existing target prediction results is less than a specified threshold, a new target ID is created for it. When a target ID is not matched for several consecutive frames, it is destroyed.

7. The intelligent early warning system for blind spots of large engineering vehicles according to claim 1, characterized in that, The main control unit runs a target tracking module based on a filtering algorithm; The associated target tracking module establishes the motion state equation and observation equation of the millimeter-wave radar point cloud target, and uses an extended Kalman filter to approximate the nonlinear relationship into a linear relationship through Taylor series expansion, so as to generate a smooth motion trajectory of the associated target.

8. The intelligent early warning system for blind spots of large engineering vehicles according to claim 7, characterized in that, It also includes a trajectory management module, which divides the target trajectory into idle state, detection state and active state; When a new target is detected and cannot be associated with an old target, the trajectory instance enters the detection state. Once a target in the detection state is successfully associated with multiple consecutive frames, it transitions to the active state. When the number of failed association frames of an active trajectory instance exceeds a predetermined threshold, the tracking trajectory is deleted and enters an idle state.

9. The intelligent early warning system for blind spots of large engineering vehicles according to claim 1, characterized in that, It also includes a temperature compensation module, which is used to monitor the ambient temperature in real time and correct the ultrasonic ranging speed.

10. The intelligent early warning system for blind spots of large engineering vehicles according to claim 1, characterized in that, The display alarm module includes a character fusion module and an alarm unit; The character fusion module fuses radar distance data, speed data and camera video signals to generate a video signal containing obstacle information. The alarm unit provides graded warnings at different alarm frequencies based on the comparison between the distance to the obstacle and a preset threshold.

Images