A target detection system based on radar and video fusion

Abstract

The application discloses a target detection system based on radar and video fusion, which comprises a monitoring target identification module, a monitoring target tracking module, an image occlusion analysis module, an image occlusion processing module, an image occlusion analysis module, a target speed monitoring module and a target speed analysis module; the monitoring target in a monitoring area and the action track of the monitoring target are identified, a suitable camera is selected to track the monitoring target, whether the tracking of the monitoring target by the camera is affected by occlusion is judged, if yes, whether other cameras can be used for assistance is judged, if not, when the monitoring target leaves the occlusion range, the monitoring target is detected and identified by using image detection, and the monitoring target is tracked by using radar detection, so that the continuity of the monitoring process of the monitoring target is ensured; the application analyzes the fusion of video detection and radar detection when the video monitoring is occluded, and realizes accurate monitoring of the target.

Description

Technical Field

[0001] This invention belongs to the field of target detection technology, specifically a target detection system based on radar and video fusion. Background Technology

[0002] Accurate target detection is crucial in many fields, including intelligent surveillance. Traditional single-sensor target detection methods, such as relying solely on radar or video image detection, have certain limitations. While radar detection can accurately acquire information such as the distance and speed of a target, it is weak in recognizing detailed features such as the target's category and shape. On the other hand, relying solely on video image detection is greatly affected by environmental factors. For example, in severe weather conditions, image quality deteriorates significantly, leading to a decrease in the accuracy and reliability of target detection. Furthermore, image detection can sometimes be obstructed by occlusions, preventing it from functioning as intended.

[0003] This invention provides a target detection system based on radar and video fusion. It integrates radar detection and video detection, predicts the movement trajectory of the monitored target through the radar detection system, optimizes the camera layout to reduce the impact of occlusion, and thus improves the detection accuracy and efficiency of the monitored target. When the occlusion cannot be avoided, it improves the accuracy of tracking and detecting the target by analyzing the image after the monitored target leaves the occlusion and the speed of the monitored target in the occluded trajectory. Summary of the Invention

[0004] The purpose of this invention is to provide a target detection system based on radar and video fusion to solve at least one of the aforementioned problems in the prior art.

[0005] In a first aspect, the present invention provides a target detection system based on radar and video fusion, comprising the following modules:

[0006] Target identification module: Acquires information about the monitoring area through millimeter-wave radar and surveillance cameras, analyzes and processes the information, and identifies the monitoring targets and their movement trajectories within the monitoring area;

[0007] Target tracking module: Analyzes the images of the monitored targets acquired by the camera to obtain image representation values, and selects an appropriate camera to track the monitored targets based on the magnitude of the image representation values;

[0008] Image occlusion analysis module: By analyzing the predicted time of the monitored target passing through the occluded trajectory, it determines whether the tracking of the monitored target is affected by occlusion. If so, it generates an occlusion signal.

[0009] Image occlusion processing module: Based on the occlusion signal, it determines whether other unoccluded cameras can obtain the occluded trajectory information. If not, it analyzes the image data after the monitored target leaves the occlusion termination point to obtain the image matching value.

[0010] Image occlusion analysis module: compares the image matching value with the image matching threshold, and determines whether the target in the image after leaving the occlusion termination point is the original monitoring target based on the comparison result. If so, a matching signal is generated.

[0011] Target velocity monitoring module: Based on the matching signal, it acquires the velocity of the monitored target through millimeter-wave radar, processes the data, and obtains the velocity characterization value;

[0012] Target velocity analysis module: compares the velocity characterization value with the velocity characterization threshold, and determines whether the acquired target velocity is normal based on the comparison result. If so, it generates a normal monitoring signal; based on the normal monitoring signal, it continuously monitors the target.

[0013] Secondly, the present invention provides a target detection method based on radar and video fusion, comprising the following steps:

[0014] Step 1: Acquire monitoring area information through millimeter-wave radar and surveillance cameras, analyze and process the information, and identify the monitoring targets and their movement trajectories within the monitoring area;

[0015] Step 2: Analyze the images of the monitored target acquired by the camera to obtain image representation values, and select an appropriate camera to track the monitored target based on the magnitude of the image representation values;

[0016] Step 3: By analyzing the predicted time of the monitored target passing through the obstructed trajectory, determine whether the tracking of the monitored target is affected by the obstruction. If so, generate an obstruction signal.

[0017] Step 4: Based on the occlusion signal, determine whether other unobstructed cameras can obtain the occluded trajectory information. If not, analyze the image data after the monitored target leaves the occlusion termination point to obtain the image matching value.

[0018] Step 5: Compare the image matching value with the image matching threshold. Based on the comparison result, determine whether the target in the image after leaving the occlusion termination point is the original monitoring target. If so, generate a matching signal.

[0019] Step 6: Based on the matched signal, acquire the velocity of the monitored target using millimeter-wave radar, process the data, and obtain the velocity characterization value;

[0020] Step 7: Compare the velocity characterization value with the velocity characterization threshold. Based on the comparison result, determine whether the acquired target velocity is normal. If so, generate a normal monitoring signal. Based on the normal monitoring signal, continuously monitor the target.

[0021] The beneficial effects of this invention are:

[0022] 1. The technical solution of this invention is as follows: Detection of the monitoring target is achieved by using millimeter-wave radar and a camera. This fully utilizes the high precision of millimeter-wave radar in distance and speed measurement and the advantages of video in target classification and recognition, enabling more accurate monitoring of the target's location, category, and other information. Compared with single-sensor detection methods, this improves the overall performance of the detection system. By predicting the movement trajectory of the monitoring target, analyzing the shooting angle and occlusion of the camera along the target's movement trajectory, and optimizing the camera layout, the impact of occlusion is further reduced, thereby improving the detection accuracy and efficiency of the monitoring target.

[0023] 2. The technical solution of this invention is as follows: By analyzing the predicted time of the monitored target passing through the obstructed trajectory, it is determined whether the system's tracking of the monitored target is affected by the trajectory obstruction. If so, an obstruction signal is generated. Based on the obstruction signal, it is determined whether other unobstructed cameras can obtain the obstructed trajectory information. If not, the image data of the monitored target after leaving the obstruction termination point is analyzed to obtain an image matching value. The image matching value is compared with the image matching threshold. Based on the comparison result, it is determined whether the target in the image after leaving the obstruction termination point is the original monitored target. If so, a matching signal is generated. This invention can accurately determine whether the system's tracking of the monitored target is affected by the trajectory obstruction. It helps to promptly detect abnormal situations in the tracking process. It can determine whether other unobstructed cameras can obtain the obstructed trajectory information. If they can, data from other cameras can be used to supplement the information, enhancing the system's ability to cope with obstruction. Analyzing the image data of the monitored target after leaving the obstruction termination point ensures that even after the target is obstructed, the system can still accurately identify its identity, ensuring the continuity and accuracy of tracking.

[0024] 3. The technical solution of this invention is as follows: Based on the matching signal, the speed of the monitored target is acquired through millimeter-wave radar, data processing is performed to obtain a speed characterization value, and the accuracy of the monitored target is further judged; the speed characterization value is compared with a speed characterization threshold, and the acquired speed of the monitored target is judged according to the comparison result. If it is normal, a normal monitoring signal is generated; based on the normal monitoring signal, the monitored target is continuously monitored; This invention analyzes the speed of the monitored target. In complex scenes and when there are multiple similar targets in the images, radar and video are fused, and the monitoring data of the monitored target by millimeter-wave radar is used to assist in the judgment, thereby improving the accuracy of the system in identifying the monitored target, helping to avoid misjudgment, and enhancing the reliability of the system. Attached Figure Description

[0025] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0026] Figure 1 This is a schematic diagram of a target detection system based on radar and video fusion provided by the present invention;

[0027] Figure 2 This is a flowchart of a target detection system based on radar and video fusion for obtaining image representation values, provided in Embodiment 1 of the present invention.

[0028] Figure 3 This is a flowchart of an image matching value acquisition system based on radar and video fusion provided in Embodiment 2 of the present invention;

[0029] Figure 4 This is a flowchart of a target detection system based on radar and video fusion for obtaining velocity characterization values, provided in Embodiment 3 of the present invention.

[0030] Figure 5 This is a flowchart illustrating a target detection method based on radar and video fusion provided in Embodiment 4 of the present invention. Detailed Implementation

[0031] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0032] Example 1

[0033] like Figure 1 , Figure 2 As shown in the figure, the target detection system based on radar and video fusion provided by this embodiment of the invention specifically includes the following modules:

[0034] Target identification module: Acquires information about the monitoring area through millimeter-wave radar and surveillance cameras, analyzes and processes the information, and identifies the monitoring targets and their movement trajectories within the monitoring area;

[0035] Millimeter-wave radar is used. A specific frequency millimeter-wave signal is transmitted through a signal transmitting unit. The reflected echo signal is received through a signal receiving unit and converted into an electrical signal. The electrical signal is transmitted to a signal processing unit, where it is amplified, filtered, and mixed. The processed electrical signal is then compared with a preset threshold. Based on the comparison result, it is determined whether a monitoring target exists in the monitoring area. If it exists, parameters such as the target's distance, position, speed, and direction are extracted.

[0036] The system acquires the location information of the monitored target and the video information captured by the camera at the corresponding location; it then extracts several frames of images from the video captured by the camera, compares and analyzes the images, identifies the monitored target in the images, and acquires information such as the color and shape of the monitored target.

[0037] The millimeter-wave radar data features and video data features of the monitored targets are extracted and fused. A deep learning-based fusion network is used, which includes multiple convolutional layers, pooling layers and fully connected layers. The fusion method of the two data features is learned through training to generate a fused feature vector of the monitored targets. Based on the fused feature vector, a target detection algorithm is used to detect the targets and further determine the target's position, speed, shape, color and other information.

[0038] Target tracking module: Analyzes the images of the monitored targets acquired by the camera to obtain image representation values, and selects an appropriate camera to track the monitored targets based on the magnitude of the image representation values;

[0039] The target's movement direction and speed are obtained through millimeter-wave radar; the target's movement trajectory is predicted, and camera information along the trajectory is obtained, including the direction in which the camera captures the movement trajectory and whether there are any obstructions when the camera captures the movement trajectory.

[0040] For example, when the movement trajectory of a monitored target passes through a certain monitoring area, both camera A and camera B can perform video detection on the corresponding monitoring area; however, camera A and camera B have different shooting angles for the corresponding monitoring area.

[0041] Obtain the movement trajectory of the monitored target;

[0042] Camera A is used to capture images of the movement trajectory of the monitored target and these images are then analyzed. The length of the monitored target's movement trajectory in the captured images is recorded as the capture trajectory length c1. The captured images are then used to determine whether there are any obstructions blocking the monitored target's movement trajectory. If there are obstructions, the length of the monitored target's movement trajectory obstructed by the obstruction is recorded as the obstruction length c2. If there are no obstructions, the obstruction length c2 is set to zero.

[0043] The ratio of the occlusion length c2 to the shooting trajectory length c1 is processed to obtain the occlusion shooting length ratio, which is denoted as YX;

[0044] The Sobe l operator is used to calculate the horizontal gradient Gx and vertical gradient Gy of the image. The gradient magnitude G is then calculated using the formula: Calculate the average value of the gradient magnitude of the image, denoted as the gradient mean GM; the larger the gradient mean GM, the higher the image sharpness.

[0045] Data processing was performed on the gradient mean GM and the occlusion shooting length ratio YX, using the formula... The image representation value PG is obtained; where a1 and a2 are preset scaling coefficients.

[0046] It should be noted that the image characterization value PG reflects a comprehensive assessment of the image quality and occlusion situation captured by the camera. The larger the image characterization value PG, the higher the image quality and the smaller the impact of occlusion.

[0047] In the same way as obtaining the image representation value PG of camera A, obtain the image representation value PG of camera B; compare the image representation values ​​PG of camera A and camera B, and select the camera with the larger image representation value for target detection to ensure the accuracy and reliability of the detection results;

[0048] The technical solution of this invention is as follows: The target is detected using millimeter-wave radar and a camera. This fully utilizes the high precision of millimeter-wave radar in distance and velocity measurement, as well as the advantages of video in target classification and recognition. This allows for more accurate monitoring of the target's location, category, and other information, improving the overall performance of the detection system compared to single-sensor detection methods. Furthermore, by predicting the target's movement trajectory and analyzing the camera's shooting angle and occlusion along the target's trajectory, the camera layout is optimized to further reduce occlusion effects and improve the accuracy and efficiency of target detection.

[0049] Example 2

[0050] like Figure 1 , Figure 3 As shown in the figure, the target detection system based on radar and video fusion provided by this embodiment of the invention specifically includes the following modules:

[0051] Image occlusion analysis module: By analyzing the predicted time of the monitored target passing through the occluded trajectory, it determines whether the tracking of the monitored target is affected by occlusion. If so, it generates an occlusion signal.

[0052] Acquire image data of the movement trajectory of the monitored target captured by the camera, analyze the occlusion information of the monitored target's movement trajectory, such as the length of the occluded trajectory, and calculate the degree of influence of the occluded trajectory on the tracking of the monitored target's movement trajectory;

[0053] Analyze the occlusion length c2; denote the starting point of the occlusion length c2 as the occlusion start point and the ending point as the occlusion end point;

[0054] The moving speed v of the monitored target is obtained through millimeter-wave radar; the blocking time t is calculated based on the moving speed v and the blocking length c2.

[0055] The occlusion time t is compared with the occlusion time threshold. If the occlusion time t is greater than or equal to the occlusion threshold, it is determined that the system's tracking of the monitored target is affected by the trajectory occlusion, and an occlusion signal is generated. Otherwise, it is determined that the system's tracking of the monitored target is not affected by the trajectory occlusion, and a normal signal is generated.

[0056] Based on normal signals, no processing is performed;

[0057] Image occlusion processing module: Based on the occlusion signal, it determines whether other unoccluded cameras can obtain the occluded trajectory information. If not, it analyzes the image data after the monitored target leaves the occlusion termination point to obtain the image matching value.

[0058] Based on the obstruction signal, determine whether other unobstructed cameras can obtain trajectory information of the obstruction length;

[0059] If other unobstructed cameras can obtain trajectory information of the obstruction length;

[0060] Based on the movement speed and trajectory of the monitored target, the estimated start time for the monitored target to move to the occlusion starting point is calculated. Before the estimated start time, an unobstructed camera is used to monitor the occluded trajectory and obtain image data of the occluded trajectory.

[0061] Based on the movement speed and trajectory of the monitored target, the estimated termination time of the monitored target moving to the occlusion termination point is calculated, and the original camera is used to continuously monitor the monitored target before the estimated termination time;

[0062] By fusing the obstructed trajectory image data obtained using an unobstructed camera with the original camera data, complete and continuous trajectory information of the monitored target is formed.

[0063] If other unobstructed cameras cannot obtain trajectory information of the obstruction length;

[0064] The millimeter-wave radar continuously tracks the target from the point where it enters the obstruction to the point where it leaves the obstruction.

[0065] When the monitored target leaves the occlusion termination point, the original camera is immediately used for monitoring; the image data after the monitored target leaves the occlusion termination point is compared and analyzed with the image data before the monitored target enters the occlusion start point.

[0066] Acquire an image of the monitored target after it leaves the occlusion termination point, extract the outline of the monitored target in the image, and compare it with the outline of the monitored target before it enters the occlusion start point to obtain the overlapping area of ​​the outline of the monitored target after it leaves the occlusion termination point and the outline of the monitored target before it enters the occlusion start point.

[0067] Calculate the ratio of the overlapping area to the total area of ​​the monitored target to obtain the overlapping area ratio, denoted as MJ;

[0068] Acquire an image of the monitored target after it leaves the occlusion termination point, and extract feature points of the monitored target in the image, such as the color type, color distribution, texture, and shape of the monitored target;

[0069] Acquire the termination feature points contained in the image after the monitored target leaves the occlusion termination point, and count the number of termination feature points s1; and acquire the starting feature points contained in the image of the monitored target before it enters the occlusion start point, and count the number of starting feature points s2.

[0070] The difference between the number of terminating feature points s1 and the number of initial feature points s2 is calculated and the absolute value is taken to obtain the feature point number difference; the ratio between the number of initial feature points s2 and the feature point number difference is calculated to obtain the feature point number ratio.

[0071] The termination feature point is matched with the starting feature point; the number of overlaps between the termination feature point and the starting feature point is obtained to obtain the overlapping feature points; the number of overlapping feature points is counted, and the ratio of the number of overlapping feature points to the number of starting feature points is calculated to obtain the overlapping feature point ratio.

[0072] The weighted sum of the ratio of the number of feature points and the ratio of overlapping feature points is used to obtain the comprehensive feature matching degree, which is denoted as PD.

[0073] Data analysis was performed on the overlap area ratio (MJ) and the comprehensive feature matching degree (PD), using the formula... Obtain the image matching value TX; where b1 and b2 are preset scaling coefficients;

[0074] It should be noted that the image matching value TX reflects the image similarity of the monitored target before and after entering the occlusion trajectory; the larger the image matching value TX, the higher the image similarity of the monitored target before and after entering the occlusion trajectory.

[0075] Image occlusion analysis module: compares the image matching value with the image matching threshold, and determines whether the target in the image after leaving the occlusion termination point is the original monitoring target based on the comparison result. If so, a matching signal is generated.

[0076] The image matching value is compared with the image matching threshold. The specific process is as follows:

[0077] If the image matching value is greater than or equal to the image matching threshold, a matching signal is generated;

[0078] If the image matching value is less than the image matching threshold, a non-matching signal is generated;

[0079] Based on the non-matching signal, it is determined that the target in the image after leaving the occlusion termination point is not the original monitoring target; the remaining targets in the image after leaving the occlusion termination point are re-identified until the original monitoring target is found.

[0080] The technical solution of this invention is as follows: By analyzing the predicted time of the monitored target passing through the obstructed trajectory, it is determined whether the system's tracking of the monitored target is affected by the trajectory obstruction. If so, an obstruction signal is generated. Based on the obstruction signal, it is determined whether other unobstructed cameras can obtain the obstructed trajectory information. If not, the image data of the monitored target after leaving the obstruction termination point is analyzed to obtain an image matching value. The image matching value is compared with an image matching threshold. Based on the comparison result, it is determined whether the target in the image after leaving the obstruction termination point is the original monitored target. If so, a matching signal is generated. This invention can accurately determine whether the system's tracking of the monitored target is affected by trajectory obstruction. It helps to promptly detect abnormal situations during the tracking process. It can determine whether other unobstructed cameras can obtain the obstructed trajectory information. If they can, data from other cameras can be used to supplement the information, enhancing the system's ability to cope with obstruction. Analyzing the image data of the monitored target after leaving the obstruction termination point ensures that even after the target is obstructed, the system can still accurately identify its identity, ensuring the continuity and accuracy of tracking.

[0081] Example 3

[0082] like Figure 1 , Figure 4 As shown in the figure, the target detection system based on radar and video fusion provided by this embodiment of the invention specifically includes the following modules:

[0083] Target velocity monitoring module: Based on the matching signal, it acquires the velocity of the monitored target through millimeter-wave radar, processes the data, and obtains the velocity characterization value;

[0084] Millimeter-wave radar is used to acquire monitoring data of the target before it enters the obstruction starting point, when the target is within the obstruction trajectory, and after the target leaves the obstruction ending point. The millimeter-wave radar monitoring data includes the target's moving speed, etc.

[0085] The monitoring target's speed is obtained before it enters the occlusion start point, while it is within the occlusion trajectory, and after it leaves the occlusion end point.

[0086] The velocity v of the monitored target is acquired by millimeter-wave radar at preset time intervals, denoted as v1, v2, v3…vn; the acceleration x1, x2, x3…xn of the monitored target is acquired at preset time intervals.

[0087] Obtain the first-order difference ΔX of the acceleration x1, x2, x3…xn of the monitored target at preset time intervals. i , where ΔX i =X i+1 -X i The first-order difference of acceleration ΔX i Compared with the first-order difference threshold of acceleration, if the first-order difference of acceleration ΔX i If the value is greater than or equal to the first-order difference threshold of acceleration, then the corresponding first-order difference ΔX of acceleration will be... i This is denoted as the deviation difference; conversely, the corresponding first-order acceleration difference ΔX is denoted as... i This is recorded as a normal difference;

[0088] The number of deviations is counted, and the number of deviations is compared with the total number of deviations to obtain the deviation ratio, denoted as PL;

[0089] The first-order difference of acceleration ΔX corresponding to the deviation difference is... i The difference is calculated by subtracting from the first-order difference threshold of acceleration to obtain the difference deviation value; the difference deviation values ​​of all deviation differences are counted, summed and averaged to obtain the difference deviation mean; the difference deviation mean is compared with the first-order difference threshold of acceleration to obtain the difference deviation degree value, which is marked as CD.

[0090] Data processing is performed on the deviation ratio PL and the deviation degree value CD, using the formula... The velocity characterization value BZ is obtained; where k1 and k2 are preset proportional coefficients.

[0091] It should be noted that the velocity characterization value BZ is used to evaluate the true motion state of the target, thereby improving the accuracy of the target detection system. The larger the velocity characterization value BZ, the higher the velocity smoothness of the monitored target. The smaller the velocity characterization value BZ, the more unstable or abnormal fluctuations the monitored target has during its motion, and the greater the possibility that the corresponding target is not the original monitored target.

[0092] Target velocity analysis module: compares the velocity characterization value with the velocity characterization threshold, determines whether the acquired target velocity is normal based on the comparison result, and generates a normal monitoring signal based on the normal monitoring signal; and continuously monitors the target based on the normal monitoring signal.

[0093] The velocity characterization value is compared with the velocity characterization threshold, and the specific process is as follows:

[0094] If the velocity characterization value is greater than or equal to the velocity characterization threshold, a normal monitoring signal is generated;

[0095] If the velocity characterization value is less than the velocity characterization threshold, a monitoring anomaly signal is generated;

[0096] Based on the monitoring of normal signals, the monitoring target is continuously monitored;

[0097] Based on the abnormal monitoring signal, it is determined that the corresponding target is not the original monitoring target; the remaining targets in the monitoring area are re-identified until the original monitoring target is found.

[0098] The technical solution of this invention is as follows: Based on the matching signal, the speed of the monitored target is acquired through millimeter-wave radar, and the data is processed to obtain a speed characterization value, which is then used to further determine the accuracy of the monitored target; the speed characterization value is compared with a speed characterization threshold, and the comparison result is used to determine whether the acquired speed of the monitored target is normal. If so, a normal monitoring signal is generated; based on the normal monitoring signal, the monitored target is continuously monitored; This invention analyzes the speed of the monitored target. In complex scenes with multiple similar targets, radar and video fusion is used, and the monitoring data of the monitored target by millimeter-wave radar is used to assist in the judgment, improving the accuracy of the system in identifying the monitored target, helping to avoid misjudgment, and enhancing the reliability of the system.

[0099] Example 4

[0100] like Figure 5 As shown in the figure, the target detection method based on radar and video fusion provided by the present invention specifically includes the following steps:

[0101] Step 1: Acquire monitoring area information through millimeter-wave radar and surveillance cameras, analyze and process the information, and identify the monitoring targets and their movement trajectories within the monitoring area;

[0102] Step 2: Analyze the images of the monitored target acquired by the camera to obtain image representation values, and select an appropriate camera to track the monitored target based on the magnitude of the image representation values;

[0103] Step 3: By analyzing the predicted time of the monitored target passing through the obstructed trajectory, determine whether the tracking of the monitored target is affected by the obstruction. If so, generate an obstruction signal.

[0104] Step 4: Based on the occlusion signal, determine whether other unobstructed cameras can obtain the occluded trajectory information. If not, analyze the image data after the monitored target leaves the occlusion termination point to obtain the image matching value.

[0105] Step 5: Compare the image matching value with the image matching threshold. Based on the comparison result, determine whether the target in the image after leaving the occlusion termination point is the original monitoring target. If so, generate a matching signal.

[0106] Step 6: Based on the matched signal, acquire the velocity of the monitored target using millimeter-wave radar, process the data, and obtain the velocity characterization value;

[0107] Step 7: Compare the velocity characterization value with the velocity characterization threshold. Based on the comparison result, determine whether the acquired target velocity is normal. If so, generate a normal monitoring signal. Based on the normal monitoring signal, continuously monitor the target.

[0108] The foregoing has provided a detailed description of one embodiment of the present invention, but this description is merely a preferred embodiment and should not be construed as limiting the scope of the invention. All equivalent variations and modifications made within the scope of the claims of this invention should still fall within the patent coverage of this invention.

Claims

1. A target detection system based on radar and video fusion, characterized in that, The method comprises the following modules: A monitoring target identification module: obtaining monitoring area information through a millimeter wave radar and a monitoring camera, analyzing and processing, identifying monitoring targets in the monitoring area and the action trajectory of the monitoring targets; A monitoring target tracking module: analyzing the monitoring target image obtained by the camera to obtain an image representation value, and selecting the camera with a larger image representation value to track the monitoring target; An image occlusion analysis module: judging whether the tracking of the monitoring target is affected by the occlusion by analyzing the predicted time of the monitoring target passing through the occluded trajectory, and generating an occlusion signal if so; An image occlusion processing module: judging whether other cameras not affected by the occlusion can obtain the occluded trajectory information based on the occlusion signal, and analyzing the image data of the monitoring target after leaving the occlusion termination point to obtain an image matching value if not; An image occlusion analysis module: comparing the image matching value with an image matching threshold value, and judging whether the target in the image after leaving the occlusion termination point is the original monitoring target according to the comparison result, and generating a matching signal if so; A target speed monitoring module: obtaining the speed of the monitoring target through the millimeter wave radar based on the matching signal, and processing the data to obtain a speed representation value; A target speed analysis module: comparing the speed representation value with a speed representation threshold value, and judging whether the obtained monitoring target speed is normal according to the comparison result, and generating a monitoring normal signal if so; Based on the monitoring normal signal, the monitoring target is continuously monitored; The image representation value is obtained in the following manner: The length of the action trajectory of the monitoring target occluded by the occluded object is analyzed to obtain an occlusion shooting length ratio, marked as YX; The clarity of the obtained image is analyzed, recorded as gradient mean GM; Data processing is performed on the gradient mean GM and the occlusion shooting length ratio YX, and an image representation value PG is obtained by using a formula ; wherein a1 and a2 are preset proportion coefficients. The image matching value is obtained in the following manner: The image after the monitoring target leaves the occlusion termination point is analyzed to obtain a coincidence area ratio MJ; The feature points in the image after the monitoring target leaves the occlusion termination point are analyzed to obtain a comprehensive feature matching degree PD; Data analysis is performed on the coincidence area ratio MJ and the comprehensive feature matching degree PD, and an image matching value TX is obtained by using a formula ; wherein b1 and b2 are preset proportion coefficients. The coincidence area ratio is obtained in the following manner: The image after the monitoring target leaves the occlusion termination point is obtained, the contour of the monitoring target in the image is extracted, and the contour of the monitoring target after leaving the occlusion termination point is compared with the contour of the monitoring target before entering the occlusion starting point to obtain the coincidence area of the contour of the monitoring target after leaving the occlusion termination point and the contour of the monitoring target before entering the occlusion starting point; The contour area of the monitoring target image before the monitoring target enters the occlusion starting point is recorded as the total area of the monitoring target; The ratio of the coincidence area to the total area of the monitoring target is calculated to obtain the coincidence area ratio, marked as MJ; The comprehensive feature matching degree is obtained in the following manner: The termination feature points are matched with the starting feature points; the number of coincident feature points is obtained by matching the termination feature points with the starting feature points; the number of coincident feature points is counted, and the number of coincident feature points is processed by ratio with the number of starting feature points to obtain a coincident feature point ratio; The feature point ratio and the coincident feature point ratio are weighted and summed to obtain a comprehensive feature matching degree, marked as PD. 2.The target detection system based on radar and video fusion according to claim 1, characterized in that, The occlusion shooting length ratio is obtained in the following manner: Acquire and analyze the captured images; in the captured images, obtain the length of the target's movement trajectory, denoted as the capture trajectory length c1; acquire the captured images and determine whether there are any obstructions blocking the target's movement trajectory. If there are obstructions, obtain the length of the target's movement trajectory obstructed by the obstruction, denoted as the obstruction length c2. If there is no occlusion, the occlusion length c2 is zero. The ratio of the occlusion length c2 to the shooting trajectory length c1 is processed to obtain the occlusion shooting length ratio, which is denoted as YX. 3.The target detection system based on radar and video fusion of claim 1, wherein, The method for obtaining the feature point ratio is as follows: Acquire the termination feature points contained in the image after the monitored target leaves the occlusion termination point, and count the number of termination feature points s1; and acquire the starting feature points contained in the image of the monitored target before it enters the occlusion start point, and count the number of starting feature points s2. The difference between the number of terminating feature points s1 and the number of starting feature points s2 is calculated and the absolute value is taken to obtain the feature point number difference; the ratio between the number of starting feature points s2 and the feature point number difference is calculated to obtain the feature point number ratio.

4. The target detection system based on fusion of radar and video according to claim 1, characterized in that, The method for obtaining the velocity characterization value is as follows: The deviation difference of the acceleration of the monitored target within a preset time interval is analyzed to obtain the deviation difference ratio, which is denoted as PL; The deviation of the acceleration of the monitored target within a preset time interval is analyzed to obtain the deviation value, which is marked as CD; The data of the deviation differential ratio PL and the differential deviation degree value CD are processed by using the formula to obtain the speed characteristic value BZ. Where k1 and k2 are preset proportional coefficients.

5. The radar and video fusion based target detection system of claim 4, wherein, The deviation difference ratio is obtained as follows: Obtaining the first-order difference of the acceleration x1, x2, x3…xn of the monitoring target in a preset time interval Wherein The acceleration first-order difference Is compared with the acceleration first-order difference threshold value, if the acceleration first-order difference Is greater than or equal to the acceleration first-order difference threshold value, the corresponding acceleration first-order difference Is recorded as the deviation difference The number of deviations is counted, and the number of deviations is compared with the total number of deviations to obtain the deviation ratio, which is denoted as PL.

6. The radar and video fusion based target detection system of claim 4, wherein, The method for obtaining the difference deviation value is as follows: The acceleration first-order difference corresponding to the deviation difference The difference between the acceleration first-order difference threshold value and the difference value is processed to obtain the difference deviation value; the difference deviation values of all deviation differences are summed and averaged to obtain the difference deviation mean value; The difference deviation from the mean is processed by the ratio of the first-order difference threshold of acceleration to obtain the difference deviation value, which is marked as CD.

Images