Radar dome camera target tracking algorithm based on complementary filtering

Abstract

The invention discloses a radar dome camera target tracking algorithm based on complementary filtering, and particularly relates to the field of target detection and tracking, and the algorithm comprises the following steps: S1, a radar system collects the distance, speed and angle data of a target at a certain time interval, and transmits the data to a data processing unit; the dome camera monitoring equipment acquires video image data of a monitoring area in real time and transmits the video image data to the data processing unit; s2, preprocessing the distance, speed and angle data of the target, and preprocessing the video image data; according to the method, target information of the radar and the dome camera is effectively fused, the problem that the false alarm rate of the radar is high is solved, the accuracy and real-time performance of target detection and tracking and the adaptability to the complex environment are improved, human intervention is reduced, and therefore a more reliable target tracking solution is provided for the fields of security and protection monitoring and the like.

Description

Technical Field

[0001] The present invention relates to the field of target detection and tracking, and more specifically, to a radar ball camera target tracking algorithm based on complementary filtering. Background Art

[0002] In the fields of security monitoring, intelligent transportation, and smart parks, target detection and tracking are crucial tasks. As an active detection device, radar can accurately measure the distance, speed, angle and other information of the target under various weather conditions. It has the advantages of wide detection range and is not affected by light. However, the target information obtained by the radar is usually abstract, and it is difficult to intuitively determine the specific identity and behavioral characteristics of the target. Therefore, the traditional radar ball camera linkage tracking technology is usually generated. A single sensor decision is adopted, that is, the target is detected by radar, and then the ball camera is controlled to rotate to the position of the target within the ball camera screen, and it is manually determined whether the currently captured target is a real target, but this technology has the following defects:

[0003] 1. Limited resolution

[0004] Traditional radars have relatively low resolution in terms of distance, angle, speed, etc. In a multi-target scenario, it is difficult to accurately distinguish adjacent and close targets, and it is easy to misjudge multiple targets as one target, or it is impossible to accurately obtain the specific location and feature information of each target.

[0005] 2. There are detection blind spots and ambiguity

[0006] Some radar systems have short-range blind spots, that is, areas close to the radar cannot be effectively detected. At the same time, for long-distance targets, there may be distance ambiguity and speed ambiguity problems, and it is impossible to accurately judge the true distance and speed of the target.

[0007] 3. High false alarm rate

[0008] Due to the complexity of radar echo signals and noise interference, traditional radar target detection algorithms are prone to misjudge noise or clutter as targets, resulting in false alarms. This may lead to unnecessary waste of resources and misoperation in some application scenarios that require high detection accuracy.

[0009] In order to solve the above problems, a technical solution is now provided. Summary of the invention

[0010] In order to overcome the above-mentioned defects of the prior art, an embodiment of the present invention provides a radar ball camera target tracking algorithm based on complementary filtering. By effectively fusing the target information of the radar and the ball camera, the problem of high false alarm rate of the radar is solved, the accuracy, real-time performance and adaptability to complex environments of target detection and tracking are improved, and human intervention is reduced, thereby providing a more reliable target tracking solution for fields such as security monitoring, and effectively solving the problems raised in the above-mentioned background technology.

[0011] To achieve the above object, the present invention provides the following technical solutions:

[0012] A radar ball camera target tracking algorithm based on complementary filtering includes the following steps:

[0013] Step S1, the radar system collects the distance, speed and angle data of the target at a certain time interval and transmits it to the data processing unit, the ball camera monitoring device obtains the video image data of the monitoring area in real time, and transmits the video image data to the data processing unit;

[0014] Step S2, preprocessing the distance, speed and angle data of the target, and preprocessing the video image data;

[0015] Step S3, converting the coordinate information of the radar target and the pixel coordinates of the ball camera target in the image into a unified spatial coordinate system;

[0016] Step S4, establishing target state models of radar and ball camera;

[0017] Step S5, combining the radar distance information and the fused angle information to construct the position information of the target in three-dimensional space.

[0018] In a preferred embodiment, step S2 specifically includes the following contents:

[0019] After receiving the target's distance, speed and angle data, the data processing unit uses a mean filter method to perform denoising on the target's distance, speed and angle data to obtain accurate target distance, speed and angle data.

[0020] In a preferred embodiment, step S2 specifically further includes the following contents:

[0021] The video image data is subjected to a histogram equalization image enhancement operation, and the foreground target image is extracted using a background subtraction algorithm. The target contour extraction and feature extraction are used to obtain the characteristic information of the target's position, shape, and color.

[0022] In a preferred embodiment, step S3 specifically includes the following contents:

[0023] Calculate the spatial distance deviation and speed deviation between the radar target and the ball camera target, and set the threshold;

[0024] The continuity of the target's motion trajectory is combined for judgment. If the motion trajectory trends of the radar target and the ball camera target are similar and continuous, they are determined to be the same target and an association relationship is established.

[0025] In a preferred embodiment, step S4 specifically includes the following contents:

[0026] The target state model of radar and ball camera is:

[0027]

[0028] In the formula, is the distance change rate, is the radial acceleration, α r is the radial acceleration measured by the radar;

[0029] The ball camera target state model is:

[0030]

[0031] In the formula, ωθ c and are the angular velocities of the target in the horizontal and vertical directions measured by the ball camera, αθ c and are the angular accelerations in the corresponding directions respectively.

[0032] In a preferred embodiment, step S4 specifically further includes the following contents:

[0033] The complementary filtering algorithm is used to fuse the angle information of the radar and the ball camera. The fused angle information is θ f and The calculation formula is as follows:

[0034]

[0035] In the formula, k 1 and k 2 is the complementary filter coefficient, and its value range is between 0 and 1.

[0036] In a preferred embodiment, step S5 specifically includes the following contents:

[0037] Combine the radar distance information r and the fused angle information θ f , Construct the position information of the target in three-dimensional space:

[0038]

[0039] In a preferred embodiment, step S5 specifically further includes the following contents:

[0040] Based on the three-dimensional spatial position information of the target, the Kalman filter tracking algorithm is used to track the target's motion trajectory;

[0041] The Kalman filter tracking algorithm predicts the target's position at the next moment based on the target's current position and speed information, and corrects the prediction result in combination with new measurement data to achieve stable tracking of the target;

[0042] During the tracking process, the target's position and speed information are continuously updated, and the target is continuously monitored according to the preset tracking strategy.

[0043] The technical effects and advantages of the radar ball camera target tracking algorithm based on complementary filtering of the present invention are as follows:

[0044] 1. The target information of radar and ball camera is integrated through complementary filtering, giving full play to the advantages of both and making up for their respective shortcomings, effectively improving the accuracy of target detection and tracking, and reducing the false detection rate and missed detection rate.

[0045] 2. In the process of data processing and fusion, this algorithm adopts efficient filtering and matching algorithms, which reduces the computational complexity and can meet application scenarios with high real-time requirements, such as vehicle monitoring in intelligent transportation.

[0046] 3. Due to the integration of radar and ball camera information, the algorithm has stronger adaptability to environmental factors such as lighting changes, occlusion, and complex background, and can stably achieve target detection and tracking tasks under various harsh conditions. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 The present invention is a flowchart of a radar ball camera target tracking algorithm based on complementary filtering. DETAILED DESCRIPTION

[0048] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0049] Example 1

[0050] Figure 1 The present invention provides a radar ball camera target tracking algorithm based on complementary filtering, which specifically includes the following steps:

[0051] Step S1, the radar system collects the distance, speed and angle data of the target at a certain time interval and transmits it to the data processing unit, the ball camera monitoring device obtains the video image data of the monitoring area in real time, and transmits the video image data to the data processing unit;

[0052] Step S2, preprocessing the distance, speed and angle data of the target, and preprocessing the video image data;

[0053] Step S3, converting the coordinate information of the radar target and the pixel coordinates of the ball camera target in the image into a unified spatial coordinate system;

[0054] Step S4, establishing target state models of radar and ball camera;

[0055] Step S5, combining the radar distance information and the fused angle information to construct the position information of the target in three-dimensional space.

[0056] Step S2 specifically includes the following contents:

[0057] After receiving the target's distance, speed and angle data, the data processing unit uses a mean filter method to perform denoising on the target's distance, speed and angle data to obtain accurate target distance, speed and angle data.

[0058] It should be noted that when the radar collects data, it will be affected by various electromagnetic interferences in the environment and its own system noise, resulting in certain random errors in the collected target distance, speed and angle data. The mean filter method can effectively smooth these random noises by averaging the data within a certain range. For example, suppose that the target distance data collected by the radar is interfered by a short electromagnetic pulse at a certain moment, resulting in an abnormal value in the distance data at that moment. The mean filter can weaken the influence of this abnormal value on the result by calculating the average value of multiple surrounding data points, thereby obtaining target distance data that is closer to the true value.

[0059] In target tracking algorithms, such as the Kalman filter tracking algorithm, the quality of the input data is high. Data processed by the mean filter can better meet the assumptions of the Kalman filter, that is, the data conforms to certain statistical laws. Accurate distance, speed and angle data enable the Kalman filter to more accurately predict the next moment of the target and more effectively combine new measurement data for correction, thereby improving the accuracy and stability of target tracking.

[0060] Step S2 specifically also includes the following contents:

[0061] The video image data is subjected to a histogram equalization image enhancement operation, and the foreground target image is extracted using a background subtraction algorithm. The target contour extraction and feature extraction are used to obtain the characteristic information of the target's position, shape, and color.

[0062] It should be noted that histogram equalization can redistribute the pixel intensity of the image, so that the contrast of the image is significantly improved. In the monitoring scene, the original video image may have a low contrast between the target and the background due to factors such as uneven lighting and dim environment. For example, in a security monitoring scene at night, the target object (such as pedestrians or vehicles) may be difficult to distinguish due to dim light. Through histogram equalization, the grayscale range of the image can be stretched, making the target part brighter and more prominent, and the background part darker, thereby enhancing the visual difference between the target and the background, facilitating subsequent target detection and recognition.

[0063] Background subtraction algorithms can effectively separate foreground targets from complex backgrounds. In practical application scenarios such as security monitoring, the background is usually complex and changeable, including buildings, trees, roads, etc. Through background subtraction, background interference can be eliminated and only those moving or newly appearing targets can be focused on, thereby greatly reducing the amount of data to be processed later and improving the efficiency of target detection. For example, in intelligent transportation systems, for road monitoring videos, background subtraction can quickly extract targets such as vehicles and pedestrians from backgrounds such as roads and buildings, providing clearer target objects for subsequent target tracking and behavior analysis.

[0064] Target contour extraction can accurately determine the position boundary of the target in the image. This is critical for accurately tracking the position changes of the target. For example, in intelligent traffic monitoring, by extracting the contour of the vehicle, the position information of the vehicle on the road can be accurately obtained, thereby realizing vehicle trajectory tracking and traffic flow monitoring.

[0065] Step S3 specifically includes the following contents:

[0066] Calculate the spatial distance deviation and speed deviation between the radar target and the ball camera target, and set the threshold;

[0067] The continuity of the target's motion trajectory is combined for judgment. If the motion trajectory trends of the radar target and the ball camera target are similar and continuous, they are determined to be the same target and an association relationship is established.

[0068] It should be noted that by calculating the spatial distance deviation and speed deviation and setting the threshold, the radar target and ball camera target combinations that may be the same target can be screened out in the initial stage. This value-based judgment method provides a basis for subsequent precise matching and avoids false matching caused by blind association. For example, in a complex traffic monitoring scene, there are multiple vehicles driving at the same time. By calculating the distance and speed deviation, the association range can be narrowed and the target combinations with lower probability can be excluded.

[0069] After establishing an accurate association, the radar and the ball camera can work together to track the same target in the subsequent tracking process. This stable association enables the system to continuously obtain multiple information about the target (such as the distance and speed information of the radar and the image details of the ball camera), avoiding tracking confusion caused by incorrect target association. For example, in security monitoring, for tracking a suspicious person, a stable association can ensure that the system will not lose the target or mistakenly track other unrelated persons when switching between different sensors.

[0070] Through accurate target association, the system can avoid unnecessary calculations and processing of irrelevant targets. In multi-target scenarios, computing resources are limited. Accurate association enables the system to focus resources on the targets that really need to be tracked, improving the overall operating efficiency of the system. For example, in an intelligent park monitoring system, there may be multiple people and vehicles at the same time. If accurate association is not performed, the system may perform complex calculations on a large number of irrelevant target combinations. Effective target association can reduce these invalid calculations and allow the system to process truly important target information more quickly.

[0071] Step S4 specifically includes the following contents:

[0072] The target state model of radar and ball camera is:

[0073]

[0074] In the formula, is the distance change rate, is the radial acceleration, α r is the radial acceleration measured by the radar;

[0075] The ball camera target state model is:

[0076]

[0077] In the formula, ωθ c and are the angular velocities of the target in the horizontal and vertical directions measured by the ball camera, αθ c and are the angular accelerations in the corresponding directions respectively.

[0078] Step S4 specifically also includes the following contents:

[0079] The complementary filtering algorithm is used to fuse the angle information of the radar and the ball camera. The fused angle information is θ f and The calculation formula is as follows:

[0080]

[0081] In the formula, k 1 and k 2 is the complementary filter coefficient, and its value range is between 0 and 1.

[0082] It should be noted that the adjustment can be made according to the measurement accuracy of the radar and ball camera and the actual application scenario. By reasonably setting the complementary filter coefficient, the advantages of the radar and ball camera in angle measurement can be fully utilized to improve the accuracy of the angle information.

[0083] Parameters such as the distance change rate and radial acceleration in the radar target state model can accurately reflect the target's motion state in the distance and speed directions. In the entire tracking system, the radar has a high accuracy in measuring the target distance. Through this model, the radar's high-precision ranging characteristics can be effectively utilized to provide accurate distance information for subsequent target position determination and tracking.

[0084] The complementary filtering algorithm is used to fuse the angle information of the radar and the ball camera. By reasonably setting the complementary filtering coefficient (the value range is between 0 and 1), the weights of the two in the angle information fusion can be dynamically allocated according to the measurement accuracy of the radar and the ball camera and the actual application scenario. This method can make full use of the advantages of radar and ball cameras in angle measurement and overcome their respective shortcomings. For example, if the radar has low accuracy in long-distance angle measurement, while the ball camera is more accurate in close-range angle positioning, the complementary filtering coefficient can be adjusted to make the angle information of the ball camera account for a larger proportion in the fusion, thereby improving the accuracy of the fused angle information.

[0085] Accurate angle information is essential for constructing the position information of the target in three-dimensional space. By combining the fused angle information with the distance information of the radar, the position of the target in three-dimensional space can be determined more accurately. This helps to improve the accuracy of target tracking. In application scenarios such as security monitoring and intelligent transportation, it can more accurately determine whether the target is in a dangerous area or violates traffic regulations.

[0086] Step S5 specifically includes the following contents:

[0087] Combine the radar distance information r and the fused angle information θ f , Construct the position information of the target in three-dimensional space:

[0088]

[0089] Step S5 specifically also includes the following contents:

[0090] Based on the three-dimensional spatial position information of the target, the Kalman filter tracking algorithm is used to track the target's motion trajectory;

[0091] The Kalman filter tracking algorithm predicts the target's position at the next moment based on the target's current position and speed information, and corrects the prediction result in combination with new measurement data to achieve stable tracking of the target;

[0092] During the tracking process, the target's position and speed information are continuously updated, and the target is continuously monitored according to the preset tracking strategies (such as re-search mechanism after target loss, target approach warning, etc.).

[0093] It should be noted that during the entire algorithm operation process, the above steps are repeated continuously to achieve continuous detection, fusion and tracking of the target, providing accurate and reliable target information for applications such as security monitoring and intelligent transportation.

[0094] Combining the radar's distance information and the fused angle information to construct the target's position information in three-dimensional space can achieve all-round and three-dimensional positioning of the target. In actual application scenarios, for example, the area covered by security monitoring is often three-dimensional space (with different dimensions of length, width, and height). Whether it is an indoor place or a complex outdoor environment, accurate three-dimensional position can accurately know the location of the target, unlike relying solely on two-dimensional information, which may cause positioning ambiguity. For example, in intelligent traffic monitoring, vehicles on the upper and lower floors of viaducts or vehicles in three-dimensional parking lots can be clearly distinguished and located through three-dimensional spatial position information, avoiding misjudgment due to the limitations of plane positioning.

[0095] The Kalman filter tracking algorithm predicts the next position of the target based on its current position and speed information. This prediction is a reasonable calculation based on mathematical models and probability statistics principles, which can predict the target's movement trend in advance. The prediction results are then corrected in combination with new measurement data. This "prediction-correction" mechanism makes full use of existing prior information and new data acquired in real time, effectively reducing the impact of factors such as measurement errors and environmental interference, making the tracking of the target's motion trajectory more accurate, reducing jitter and deviation during the tracking process, and ensuring that the system can always grasp the target's accurate motion state.

[0096] The above formulas are all dimensionless and numerical calculations. The formula is a formula for the most recent real situation obtained by collecting a large amount of data and performing software simulation. The preset parameters and thresholds in the formula are set by technicians in this field according to actual conditions.

[0097] The above embodiments can be implemented in whole or in part by software, hardware, firmware or any other combination. When implemented using software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that contains one or more available media sets. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium. The semiconductor medium can be a solid-state hard disk.

[0098] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0099] In the several embodiments provided in the present application, it should be understood that the disclosed methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of the units is only a logical function division, and there may be other division methods in actual implementation. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces, indirect coupling or communication connection of devices or units, which may be electrical, mechanical or other forms.

[0100] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, and may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0101] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0102] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0103] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0104] Finally: The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A radar ball camera target tracking algorithm based on complementary filtering, characterized in that: The steps include: Step S1, the radar system collects the distance, speed and angle data of the target at a certain time interval and transmits it to the data processing unit, the ball camera monitoring device obtains the video image data of the monitoring area in real time, and transmits the video image data to the data processing unit; Step S2, preprocessing the distance, speed and angle data of the target, and preprocessing the video image data; Step S3, converting the coordinate information of the radar target and the pixel coordinates of the ball camera target in the image into a unified spatial coordinate system; Step S4, establishing target state models of radar and ball camera; Step S5, combining the radar distance information and the fused angle information to construct the position information of the target in three-dimensional space.

2. The radar ball camera target tracking algorithm based on complementary filtering according to claim 1 is characterized in that: Step S2 specifically includes the following contents: After receiving the target's distance, speed and angle data, the data processing unit uses a mean filter method to perform denoising on the target's distance, speed and angle data to obtain accurate target distance, speed and angle data.

3. The radar ball camera target tracking algorithm based on complementary filtering according to claim 2 is characterized in that: Step S2 specifically also includes the following contents: The video image data is subjected to a histogram equalization image enhancement operation, and the foreground target image is extracted using a background subtraction algorithm. The target contour extraction and feature extraction are used to obtain the characteristic information of the target's position, shape, and color.

4. The radar ball camera target tracking algorithm based on complementary filtering according to claim 3 is characterized in that: Step S3 specifically includes the following contents: Calculate the spatial distance deviation and speed deviation between the radar target and the ball camera target, and set the threshold; The continuity of the target's motion trajectory is combined for judgment. If the motion trajectory trends of the radar target and the ball camera target are similar and continuous, they are determined to be the same target and an association relationship is established.

5. The radar ball camera target tracking algorithm based on complementary filtering according to claim 4 is characterized in that: Step S4 specifically includes the following contents: The target state model of radar and ball camera is: In the formula, is the distance change rate, is the radial acceleration, α r is the radial acceleration measured by the radar; The ball camera target state model is: In the formula, ωθ c and are the angular velocities of the target in the horizontal and vertical directions measured by the ball camera, αθ c and are the angular accelerations in the corresponding directions respectively.

6. The radar ball camera target tracking algorithm based on complementary filtering according to claim 5 is characterized in that: Step S4 specifically also includes the following contents: The complementary filtering algorithm is used to fuse the angle information of the radar and the ball camera. The fused angle information is θ f and The calculation formula is as follows: Where k1 and k2 are complementary filter coefficients, and their values ​​range from 0 to 1.

7. The radar ball camera target tracking algorithm based on complementary filtering according to claim 6 is characterized in that: Step S5 specifically includes the following contents: Combine the radar distance information r and the fused angle information θ f , Construct the position information of the target in three-dimensional space:

8. The radar ball camera target tracking algorithm based on complementary filtering according to claim 7 is characterized in that: Step S5 specifically also includes the following contents: Based on the three-dimensional spatial position information of the target, the Kalman filter tracking algorithm is used to track the target's motion trajectory; The Kalman filter tracking algorithm predicts the target's position at the next moment based on the target's current position and speed information, and corrects the prediction result in combination with new measurement data to achieve stable tracking of the target; During the tracking process, the target's position and speed information are continuously updated, and the target is continuously monitored according to the preset tracking strategy.

Images