Target detection system based on radar and video fusion

Abstract

The invention discloses a target detection system based on radar and video fusion. The target detection system comprises a monitoring target identification module, a monitoring target tracking module, an image shielding analysis module, an image shielding processing module, an image shielding analysis module, a target speed monitoring module and a target speed analysis module. The method comprises the following steps: identifying a monitoring target in a monitoring area and an action track of the monitoring target, selecting a proper camera to track the monitoring target, judging whether tracking of the camera on the monitoring target is affected by shielding, if so, judging whether other cameras can be used for assisting, and if not, when the monitoring target leaves the shielding range, performing the tracking of the camera on the monitoring target. A monitoring target is recognized through image detection, the monitoring target is tracked through radar detection, and the continuity of the monitoring process of the monitoring target is ensured; according to the invention, when the video monitoring is shielded, the fusion of video detection and radar detection is utilized, so that the accurate monitoring of the target is realized.

Description

Technical Field

[0001] The invention belongs to the technical field of target detection, in particular to a target detection system based on radar and video fusion. Background Art

[0002] Accurate target detection is crucial in many fields such as intelligent monitoring. Traditional single-sensor target detection methods, such as relying solely on radar or video image detection, have certain limitations. Although radar detection alone can accurately obtain information such as the distance and speed of the target, it has a weak ability to identify detailed features such as the target's category and appearance. Detection that relies solely on video images is greatly affected by environmental factors. For example, in severe weather conditions, the image quality is severely degraded, resulting in reduced accuracy and reliability of target detection. In addition, image detection is sometimes blocked by obstructions, which makes image detection unable to play its due role.

[0003] The present invention provides a target detection system based on radar and video fusion, which integrates radar detection with video detection, predicts the movement trajectory of the monitored target through the radar detection system, optimizes the camera layout to reduce the impact of occlusion, and thus improves the detection accuracy and efficiency of the monitored target; when the occlusion cannot be avoided, the accuracy of tracking and detecting the target is improved by analyzing the image of the monitored target after leaving the occlusion and the speed of the monitored target in the occluded trajectory. Summary of the invention

[0004] The object of the present invention is to provide a target detection system based on radar and video fusion to solve at least one of the above-mentioned problems in the prior art.

[0005] In a first aspect, the present invention provides a target detection system based on radar and video fusion, comprising the following modules:

[0006] Monitoring target identification module: obtains monitoring area information through millimeter wave radar and surveillance cameras, performs analysis and processing, and identifies monitoring targets and their movement trajectories within the monitoring area;

[0007] Monitoring target tracking module: analyzes the monitoring target image obtained by the camera, obtains the image representation value, and selects the appropriate camera to track the monitoring target according to the size of the image representation value;

[0008] Image occlusion analysis module: By analyzing the predicted time of the monitored target passing through the occluded track, it is determined whether the tracking of the monitored target is affected by the occlusion. If so, an occlusion signal is generated;

[0009] Image occlusion processing module: Based on the occlusion signal, determine whether other unobstructed cameras can obtain the obstructed trajectory information. If not, analyze the image data after the monitored target leaves the occlusion end point to obtain the image matching value;

[0010] Image occlusion analysis module: compares the image matching value with the image matching threshold, and determines whether the target in the image after leaving the occlusion end point is the original monitoring target based on the comparison result. If so, a matching signal is generated;

[0011] Target speed monitoring module: Based on the matching signal, the speed of the monitored target is obtained through the millimeter wave radar, and data processing is performed to obtain the speed representation value;

[0012] Target speed analysis module: compares the speed characterization value with the speed characterization threshold, and determines whether the acquired monitoring target speed is normal based on the comparison result. If so, a monitoring normal signal is generated; based on the monitoring normal signal, the monitoring target is continuously monitored.

[0013] In a second aspect, the present invention provides a target detection method based on radar and video fusion, comprising the following steps:

[0014] Step 1: Obtain monitoring area information through millimeter wave radar and surveillance cameras, analyze and process it, and identify the monitoring targets and their movement trajectories in the monitoring area;

[0015] Step 2: Analyze the monitoring target image acquired by the camera to obtain the image representation value, and select a suitable camera to track the monitoring target according to the size of the image representation value;

[0016] Step 3: By analyzing the predicted time of the monitored target passing through the blocked track, it is determined whether the tracking of the monitored target is affected by the blockage. If so, an occlusion signal is generated;

[0017] Step 4: Based on the occlusion signal, determine whether other unobstructed cameras can obtain the obstructed trajectory information. If not, analyze the image data after the monitored target leaves the occlusion end point to obtain the image matching value;

[0018] Step 5: Compare the image matching value with the image matching threshold, and determine whether the target in the image after leaving the occlusion end point is the original monitoring target based on the comparison result. If so, generate a matching signal;

[0019] Step 6: Based on the matching signal, the speed of the monitored target is obtained through the millimeter wave radar, and data processing is performed to obtain a speed representation value;

[0020] Step 7: Compare the speed characterization value with the speed characterization threshold, and determine whether the acquired monitoring target speed is normal based on the comparison result. If so, generate a monitoring normal signal; based on the monitoring normal signal, continuously monitor the monitoring target.

[0021] Beneficial effects of the present invention:

[0022] 1. The technical solution of the embodiment of the present invention is: to detect the monitoring target through millimeter wave radar and camera, make full use of the high precision of millimeter wave radar in distance and speed measurement and the advantages of video in target classification and recognition, so as to more accurately monitor the position, category and other information of the target, and improve the comprehensive performance of the detection system compared with the single sensor detection method; by predicting the movement trajectory of the monitoring target, analyzing the shooting angle and occlusion of the camera on the movement trajectory of the monitoring target, optimizing the camera layout, further reducing the influence of occlusion, and improving the detection accuracy and efficiency of the monitoring target.

[0023] 2. The technical solution of the embodiment of the present invention is: by analyzing the predicted time of the monitored target passing through the obstructed trajectory, it is judged whether the system's tracking of the monitored target is affected by the trajectory obstruction, and if so, an occlusion signal is generated; based on the occlusion signal, it is judged whether other unobstructed cameras can obtain the obstructed trajectory information, if not, the image data after the monitored target leaves the occlusion end point is analyzed to obtain an image matching value; the image matching value is compared with the image matching threshold, and according to the comparison result, it is judged whether the target of the image after leaving the occlusion end point is the original monitored target, and if so, a matching signal is generated; the present invention can accurately judge whether the system's tracking of the monitored target is affected by the trajectory obstruction; it is helpful to timely discover abnormal situations in the tracking process; it is judged whether other unobstructed cameras can obtain the obstructed trajectory information, and if so, the information can be supplemented by using other camera data to enhance the system's ability to cope with occlusion; the image data after the monitored target leaves the occlusion end point is analyzed to ensure that even if the target is obstructed, the system can still accurately identify its identity and ensure the continuity and accuracy of tracking.

[0024] 3. The technical solution of the embodiment of the present invention is: based on the matching signal, the speed of the monitored target is obtained through the millimeter-wave radar, data processing is performed to obtain a speed characterization value, and the accuracy of the monitored target is further judged; the speed characterization value is compared with the speed characterization threshold, and whether the acquired speed of the monitored target is normal is judged according to the comparison result, and if so, a normal monitoring signal is generated; based on the normal monitoring signal, the monitored target is continuously monitored; the present invention analyzes the speed of the monitored target, and in complex scenes and when there are multiple targets with similar images, the radar and video are fused, and the monitoring data of the monitored target by the millimeter-wave radar is used to assist in the judgment, thereby improving the accuracy of the system in identifying the monitored target, helping to avoid misjudgment, and enhancing the reliability of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0026] Figure 1 It is a module schematic diagram of a target detection system based on radar and video fusion provided by the present invention;

[0027] Figure 2 This is a flow chart of obtaining image representation values ​​by a target detection system based on radar and video fusion provided in Embodiment 1 of the present invention;

[0028] Figure 3 This is a flow chart of obtaining image matching values ​​by a target detection system based on radar and video fusion provided by Embodiment 2 of the present invention;

[0029] Figure 4 This is a flow chart of obtaining a speed representation value by a target detection system based on radar and video fusion provided by Embodiment 3 of the present invention;

[0030] Figure 5 It is a flow chart of a target detection method based on radar and video fusion provided in Embodiment 4 of the present invention. DETAILED DESCRIPTION

[0031] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0032] Embodiment 1

[0033] like Figure 1 , Figure 2 As shown, an object detection system based on radar and video fusion provided by an embodiment of the present invention specifically includes the following modules:

[0034] Monitoring target identification module: obtains monitoring area information through millimeter wave radar and surveillance cameras, performs analysis and processing, and identifies monitoring targets and their movement trajectories within the monitoring area;

[0035] The millimeter wave radar is used to transmit a millimeter wave signal of a specific frequency through a signal transmitting unit; the reflected echo signal is received by a signal receiving unit and converted into an electrical signal; the electrical signal is transmitted to a signal processing unit, and the received electrical signal is amplified, filtered, mixed, and processed, and then the processed electrical signal is compared with a preset threshold value. According to the comparison result, it is determined whether there is a monitoring target in the monitoring area. If there is, the target's distance, position, speed, direction and other parameters are extracted;

[0036] Obtain the location information of the monitoring target and the video information taken by the camera at the corresponding location; extract several frames of images from the video taken by the camera, compare and analyze the images, identify the monitoring target in the image, and obtain the color, shape and other information of the monitoring target;

[0037] Extract the millimeter-wave radar data features and video data features of the monitored target, perform data fusion, and use a fusion network based on deep learning. The fusion network includes multiple convolutional layers, pooling layers, and fully connected layers. Through training, learn the fusion method of the two data features to generate a fusion feature vector of the monitored target. Based on the fusion feature vector, use the target detection algorithm to perform target detection and further determine the target's position, speed, shape, color and other information.

[0038] Monitoring target tracking module: analyzes the monitoring target image obtained by the camera, obtains the image representation value, and selects the appropriate camera to track the monitoring target according to the size of the image representation value;

[0039] The movement direction and speed of the monitored target are obtained through the millimeter wave radar; the movement trajectory of the monitored target is predicted, and the camera information on the movement trajectory is obtained. The camera information includes the direction in which the camera shoots the movement trajectory, and whether there is any obstruction when the camera shoots the movement trajectory;

[0040] For example, when the movement trajectory of the monitoring target passes through a certain monitoring area, both camera A and camera B can perform video detection on the corresponding monitoring area; wherein, camera A and camera B have different shooting angles on the corresponding monitoring area;

[0041] Obtain the action trajectory of the monitored target;

[0042] Use camera A to shoot the movement trajectory of the monitoring target, obtain the captured image, and analyze it; obtain the length of the movement trajectory of the monitoring target in the captured image, which is recorded as the shooting trajectory length c1; obtain the captured image, and determine whether there is an obstruction blocking the movement trajectory of the monitoring target. If there is an obstruction, obtain the length of the movement trajectory of the monitoring target blocked by the obstruction, which is recorded as the obstruction length c2; if there is no obstruction, the obstruction length c2 takes a zero value;

[0043] The occlusion length c2 is ratioed to the shooting trajectory length c1 to obtain the occlusion shooting length ratio, which is marked as YX;

[0044] The Sobel operator is used to calculate the horizontal gradient Gx and the vertical gradient Gy of the image, and the gradient amplitude G of the image is calculated using the formula: Calculate the average value of the image gradient amplitude, recorded as the gradient mean GM; the larger the gradient mean GM is, the higher the image clarity is;

[0045] The gradient mean GM and the occlusion shooting length ratio YX are processed and the formula is used Obtain an image representation value PG, wherein a1 and a2 are preset proportional coefficients;

[0046] It should be noted that the image representation value PG reflects a comprehensive evaluation of the image quality and occlusion conditions captured by the camera. The larger the image representation value PG is, the higher the image quality is and the smaller the occlusion effect is.

[0047] Obtain the image representation value PG of camera B in the same manner as the image representation value PG of camera A; compare the image representation values ​​PG of camera A and camera B, and select the camera with the larger image representation value for target detection to ensure the accuracy and reliability of the detection result;

[0048] The technical solution of the embodiment of the present invention is: to detect the monitoring target through millimeter wave radar and camera, make full use of the high precision of millimeter wave radar in distance and speed measurement and the advantages of video in target classification and recognition, so as to more accurately monitor the position, category and other information of the target, and improve the comprehensive performance of the detection system compared with the single sensor detection method; by predicting the movement trajectory of the monitoring target, analyzing the shooting angle and occlusion of the camera on the movement trajectory of the monitoring target, optimizing the camera layout, further reducing the influence of occlusion, and improving the detection accuracy and efficiency of the monitoring target.

[0049] Embodiment 2

[0050] like Figure 1 , Figure 3 As shown, an object detection system based on radar and video fusion provided by an embodiment of the present invention specifically includes the following modules:

[0051] Image occlusion analysis module: By analyzing the predicted time of the monitored target passing through the occluded track, it is determined whether the tracking of the monitored target is affected by the occlusion. If so, an occlusion signal is generated;

[0052] Obtain image data of the target's action trajectory captured by the camera, analyze the blocked trajectory information of the target's action trajectory, such as the blocked trajectory length, and calculate the influence of the blocked trajectory on the target's action trajectory tracking;

[0053] Analyze the occlusion length c2; record the starting point of the occlusion length c2 as the occlusion starting point, and record the ending point as the occlusion ending point;

[0054] The moving speed v of the monitored target is obtained through the millimeter wave radar; the occlusion time t is calculated according to the moving speed v and the occlusion length c2;

[0055] The occlusion time t is compared with the occlusion time threshold. If the occlusion time t is greater than or equal to the occlusion threshold, it is determined that the system's tracking of the monitored target is affected by the trajectory occlusion, and an occlusion signal is generated; otherwise, it is determined that the system's tracking of the monitored target is not affected by the trajectory occlusion, and a normal signal is generated;

[0056] Based on normal signals, no processing is performed;

[0057] Image occlusion processing module: Based on the occlusion signal, determine whether other unobstructed cameras can obtain the obstructed trajectory information. If not, analyze the image data after the monitored target leaves the occlusion end point to obtain the image matching value;

[0058] Based on the occlusion signal, determine whether other unobstructed cameras can obtain trajectory information of the occlusion length;

[0059] If other unobstructed cameras can obtain trajectory information of the obstruction length;

[0060] According to the movement speed and movement trajectory of the monitored target, the estimated start time of the monitored target moving to the occlusion start point is calculated, and before the estimated start time, the unobstructed camera is used to monitor the obstructed trajectory to obtain image data of the obstructed trajectory;

[0061] According to the movement speed and movement trajectory of the monitored target, the estimated end time of the monitored target moving to the end point of the occlusion is calculated, and the monitored target is continuously monitored using the original camera before the estimated end time;

[0062] The blocked trajectory image data obtained by the unblocked camera is fused with the original camera data to form complete and continuous trajectory information of the monitoring target;

[0063] If other unobstructed cameras cannot obtain trajectory information of the obstruction length;

[0064] The millimeter-wave radar is used to continuously track the monitoring target from the start point of entering the occlusion to the end point of leaving the occlusion;

[0065] When the monitoring target leaves the occlusion end point, the original camera is used for monitoring immediately; the image data after the monitoring target leaves the occlusion end point is compared and analyzed with the image data before the monitoring target enters the occlusion start point, specifically;

[0066] Obtain an image of the monitored target after it leaves the occlusion end point, extract the outline of the monitored target in the image, and compare it with the outline of the monitored target before entering the occlusion start point to obtain the overlapping area between the outline of the monitored target after it leaves the occlusion end point and the outline of the monitored target before entering the occlusion start point;

[0067] Calculate the ratio of the overlap area to the total area of ​​the monitored target to obtain the overlap area ratio, marked as MJ;

[0068] Obtain an image of the monitored target after it leaves the occlusion termination point, and extract feature points of the monitored target in the image, such as the color type, color distribution, texture, and shape of the monitored target;

[0069] Obtain the terminal feature points contained in the image after the monitoring target leaves the occlusion terminal point, and count the number of terminal feature points s1, and the starting feature points contained in the image of the monitoring target before the monitoring target enters the occlusion starting point, and count the number of starting feature points s2;

[0070] Perform difference processing on the ending feature point number s1 and the starting feature point number s2 and take the absolute value to obtain the feature point number difference; perform ratio processing on the starting feature point number s2 and the feature point number difference to obtain the feature point number ratio;

[0071] The end feature point is overlapped and matched with the start feature point; the number of overlaps between the end feature point and the start feature point is obtained to obtain the overlap feature point; the number of overlap feature points is counted, and the number of overlap feature points is ratioed with the number of start feature points to obtain the overlap feature point ratio;

[0072] The weighted sum of the feature point quantity ratio and the overlapping feature point ratio is performed to obtain the comprehensive feature matching degree, which is marked as PD.

[0073] The overlap area ratio MJ and the comprehensive feature matching degree PD were analyzed and the formula was used Obtain the image matching value TX; wherein b1 and b2 are preset proportional coefficients;

[0074] It should be noted that the image matching value TX reflects the image similarity of the monitoring target before and after entering the occlusion trajectory; the larger the image matching value TX is, the higher the image similarity of the monitoring target before and after entering the occlusion trajectory;

[0075] Image occlusion analysis module: compares the image matching value with the image matching threshold, and determines whether the target in the image after leaving the occlusion end point is the original monitoring target based on the comparison result. If so, a matching signal is generated;

[0076] Compare the image matching value with the image matching threshold. The specific process is as follows:

[0077] If the image matching value is greater than or equal to the image matching threshold, a matching signal is generated;

[0078] If the image matching value is less than the image matching threshold, a non-matching signal is generated;

[0079] Based on the non-matching signal, it is determined that the target in the image after leaving the occlusion end point is not the original monitoring target; the remaining targets in the image after leaving the occlusion end point are re-identified until the original monitoring target is found;

[0080] The technical solution of the embodiment of the present invention is: by analyzing the predicted time of the monitored target passing through the obstructed trajectory, it is judged whether the system's tracking of the monitored target is affected by the trajectory occlusion, and if so, an occlusion signal is generated; based on the occlusion signal, it is judged whether other unobstructed cameras can obtain the obstructed trajectory information, if not, the image data after the monitored target leaves the occlusion end point is analyzed to obtain an image matching value; the image matching value is compared with the image matching threshold, and according to the comparison result, it is judged whether the target of the image after leaving the occlusion end point is the original monitored target, and if so, a matching signal is generated; the present invention can accurately judge whether the system's tracking of the monitored target is affected by the trajectory occlusion; it is helpful to timely discover abnormal situations in the tracking process; it is judged whether other unobstructed cameras can obtain the obstructed trajectory information, and if so, the information can be supplemented by using other camera data to enhance the system's ability to cope with occlusion; the image data after the monitored target leaves the occlusion end point is analyzed to ensure that even after the target is obstructed, the system can still accurately identify its identity and ensure the continuity and accuracy of tracking.

[0081] Embodiment 3

[0082] like Figure 1 , Figure 4 As shown, an object detection system based on radar and video fusion provided by an embodiment of the present invention specifically includes the following modules:

[0083] Target speed monitoring module: Based on the matching signal, the speed of the monitored target is obtained through the millimeter wave radar, and data processing is performed to obtain the speed representation value;

[0084] The millimeter-wave radar is used to obtain the millimeter-wave radar monitoring data before the monitoring target enters the shielding start point, when the monitoring target is within the shielding trajectory, and after the monitoring target leaves the shielding end point, wherein the millimeter-wave radar monitoring data includes the moving speed of the monitoring target, etc.;

[0085] Obtain the moving speed of the monitoring target before it enters the occlusion starting point, the monitoring target within the occlusion trajectory, and the monitoring target after it leaves the occlusion ending point;

[0086] The speed v of the monitored target is obtained by the millimeter wave radar at a preset time interval, which is recorded as v1, v2, v3...vn; the acceleration x1, x2, x3...xn of the monitored target at the preset time interval is obtained;

[0087] Get the first-order difference ΔX of the acceleration x1, x2, x3…xn of the monitoring target at the preset time interval i , where ΔX i =X i+1 -X i , the first-order difference of acceleration ΔX i Compared with the first-order difference threshold of acceleration, if the first-order difference of acceleration ΔX i If the acceleration is greater than or equal to the first-order difference threshold, the corresponding first-order difference of acceleration ΔX i Recorded as deviation difference; conversely, the corresponding acceleration first-order difference ΔX i Recorded as normal difference;

[0088] Count the number of deviation differences, compare the number of deviation differences with the total number of differences, and obtain the deviation difference ratio, which is marked as PL;

[0089] The first-order difference of acceleration corresponding to the deviation difference ΔX i Perform difference processing with the first-order difference threshold of acceleration to obtain the difference deviation value; count the difference deviation values ​​of all deviation differences, sum and average them to obtain the difference deviation mean; perform ratio processing on the difference deviation mean and the first-order difference threshold of acceleration to obtain the difference deviation degree value, marked as CD;

[0090] The deviation difference ratio PL and the difference deviation degree value CD are processed and the formula is used Get the speed characterization value BZ; where k1 and k2 are preset proportional coefficients;

[0091] It should be noted that the speed characterization value BZ is used to evaluate the real motion state of the target, thereby improving the accuracy of the target detection system; the larger the speed characterization value BZ is, the higher the speed smoothness of the monitored target is; the smaller the speed characterization value BZ is, the more unstable or abnormal fluctuations there are in the motion of the monitored target, and the greater the possibility that the corresponding target is not the original monitored target;

[0092] Target speed analysis module: compares the speed representation value with the speed representation threshold, and determines whether the acquired monitoring target speed is normal according to the comparison result. If so, a monitoring normal signal is generated; based on the monitoring normal signal, the monitoring target is continuously monitored;

[0093] The speed characterization value is compared with the speed characterization threshold. The specific process is as follows:

[0094] If the speed characterization value is greater than or equal to the speed characterization threshold, a normal monitoring signal is generated;

[0095] If the speed characterization value is less than the speed characterization threshold, a monitoring abnormality signal is generated;

[0096] Based on the monitoring of normal signals, the monitoring target is continuously monitored;

[0097] Based on the abnormal monitoring signal, it is determined that the corresponding target is not the original monitoring target; the remaining targets in the monitoring area are re-identified until the original monitoring target is found;

[0098] The technical solution of the embodiment of the present invention is: based on the matching signal, the speed of the monitored target is obtained through the millimeter-wave radar, data processing is performed to obtain a speed characterization value, and the accuracy of the monitored target is further judged; the speed characterization value is compared with the speed characterization threshold, and whether the acquired speed of the monitored target is normal is judged according to the comparison result, and if so, a normal monitoring signal is generated; based on the normal monitoring signal, the monitored target is continuously monitored; the present invention analyzes the speed of the monitored target, and in complex scenes and when there are multiple targets with similar images, the radar and video are fused, and the monitoring data of the monitored target by the millimeter-wave radar is used to assist in the judgment, thereby improving the accuracy of the system in identifying the monitored target, helping to avoid misjudgment, and enhancing the reliability of the system.

[0099] Embodiment 4

[0100] like Figure 5 As shown, an embodiment of the present invention provides a target detection method based on radar and video fusion, which specifically includes the following steps:

[0101] Step 1: Obtain monitoring area information through millimeter wave radar and surveillance cameras, analyze and process it, and identify the monitoring targets and their movement trajectories in the monitoring area;

[0102] Step 2: Analyze the monitoring target image acquired by the camera to obtain the image representation value, and select a suitable camera to track the monitoring target according to the size of the image representation value;

[0103] Step 3: By analyzing the predicted time of the monitored target passing through the blocked track, it is determined whether the tracking of the monitored target is affected by the blockage. If so, an occlusion signal is generated;

[0104] Step 4: Based on the occlusion signal, determine whether other unobstructed cameras can obtain the obstructed trajectory information. If not, analyze the image data after the monitored target leaves the occlusion end point to obtain the image matching value;

[0105] Step 5: Compare the image matching value with the image matching threshold, and determine whether the target in the image after leaving the occlusion end point is the original monitoring target based on the comparison result. If so, generate a matching signal;

[0106] Step 6: Based on the matching signal, the speed of the monitored target is obtained through the millimeter wave radar, and data processing is performed to obtain a speed representation value;

[0107] Step 7: Compare the speed characterization value with the speed characterization threshold, and determine whether the acquired monitoring target speed is normal based on the comparison result. If so, generate a monitoring normal signal; based on the monitoring normal signal, continuously monitor the monitoring target.

[0108] The above is a detailed description of an embodiment of the present invention, but the content is only a preferred embodiment of the present invention and cannot be considered to limit the scope of implementation of the present invention. All equivalent changes and improvements made within the scope of the present invention should still fall within the scope of the patent coverage of the present invention.

Claims

1. A target detection system based on radar and video fusion, characterized in that: Includes the following modules: Monitoring target identification module: obtains monitoring area information through millimeter wave radar and surveillance cameras, performs analysis and processing, and identifies monitoring targets and their movement trajectories within the monitoring area; Monitoring target tracking module: analyzes the monitoring target image obtained by the camera, obtains the image representation value, and selects the appropriate camera to track the monitoring target according to the size of the image representation value; Image occlusion analysis module: By analyzing the predicted time of the monitored target passing through the occluded track, it is determined whether the tracking of the monitored target is affected by the occlusion. If so, an occlusion signal is generated; Image occlusion processing module: Based on the occlusion signal, determine whether other unobstructed cameras can obtain the obstructed trajectory information. If not, analyze the image data after the monitored target leaves the occlusion end point to obtain the image matching value; Image occlusion analysis module: compares the image matching value with the image matching threshold, and determines whether the target in the image after leaving the occlusion end point is the original monitoring target based on the comparison result. If so, a matching signal is generated; Target speed monitoring module: Based on the matching signal, the speed of the monitored target is obtained through the millimeter wave radar, and data processing is performed to obtain the speed representation value; Target speed analysis module: compares the speed representation value with the speed representation threshold, and determines whether the acquired monitoring target speed is normal according to the comparison result. If so, a monitoring normal signal is generated; Based on the monitoring of normal signals, the monitoring target is continuously monitored.

2. The target detection system based on radar and video fusion according to claim 1, characterized in that: The image characterization value is obtained in the following manner: The length of the movement trajectory of the monitoring target blocked by the occluder is analyzed to obtain the occlusion shooting length ratio, which is marked as YX; Perform clarity analysis on the acquired image, recorded as the gradient mean GM; The gradient mean GM and the occlusion shooting length ratio YX are processed and the formula is used The image representation value PG is obtained, wherein a1 and a2 are preset proportional coefficients.

3. The target detection system based on radar and video fusion according to claim 2, characterized in that: The method for obtaining the occlusion shooting length ratio is as follows: Obtain the captured image and analyze it; obtain the length of the monitoring target's movement trajectory in the captured image, which is recorded as the captured trajectory length c1; obtain the captured image and determine whether there is an obstruction blocking the monitoring target's movement trajectory. If there is an obstruction, obtain the length of the monitoring target's movement trajectory blocked by the obstruction, which is recorded as the obstruction length c2; If there is no occlusion, the occlusion length c2 takes zero value; The occlusion length c2 is ratioed to the shooting trajectory length c1 to obtain the occlusion shooting length ratio, which is marked as YX.

4. The target detection system based on radar and video fusion according to claim 1, characterized in that: The image matching value is obtained in the following way: Analyze the image of the monitored target after it leaves the occlusion end point to obtain the overlap area ratio MJ; Analyze the feature points in the image after the monitored target leaves the occlusion termination point to obtain the comprehensive feature matching degree PD; The overlap area ratio MJ and the comprehensive feature matching degree PD were analyzed and the formula was used The image matching value TX is obtained; wherein b1 and b2 are preset proportional coefficients.

5. The target detection system based on radar and video fusion according to claim 4, characterized in that: The overlapping area ratio is obtained as follows: Obtain an image of the monitored target after it leaves the occlusion end point, extract the outline of the monitored target in the image, and compare it with the outline of the monitored target before entering the occlusion start point to obtain the overlapping area between the outline of the monitored target after it leaves the occlusion end point and the outline of the monitored target before entering the occlusion start point; The image contour area of ​​the monitoring target before the monitoring target enters the occlusion starting point is recorded as the total area of ​​the monitoring target; Calculate the ratio of the overlap area to the total area of ​​the monitored target to obtain the overlap area ratio, which is marked as MJ.

6. The target detection system based on radar and video fusion according to claim 4, characterized in that: The comprehensive feature matching degree is obtained as follows: The end feature point is overlapped and matched with the start feature point; the number of overlaps between the end feature point and the start feature point is obtained to obtain the overlap feature point; the number of overlap feature points is counted, and the number of overlap feature points is ratioed with the number of start feature points to obtain the overlap feature point ratio; The weighted sum of the feature point quantity ratio and the overlapping feature point ratio is performed to obtain the comprehensive feature matching degree, which is marked as PD.

7. The target detection system based on radar and video fusion according to claim 6, characterized in that: The method for obtaining the feature point quantity ratio is as follows: Obtain the terminal feature points contained in the image after the monitoring target leaves the occlusion terminal point, and count the number of terminal feature points s1, and the starting feature points contained in the image of the monitoring target before the monitoring target enters the occlusion starting point, and count the number of starting feature points s2; The end feature point number s1 and the start feature point number s2 are subjected to difference processing and the absolute value is taken to obtain the feature point number difference; the start feature point number s2 and the feature point number difference are subjected to ratio processing to obtain the feature point number ratio.

8. The target detection system based on radar and video fusion according to claim 1, characterized in that: The speed characterization value is obtained in the following manner: Analyze the deviation difference of the acceleration of the monitoring target in the preset time interval to obtain the deviation difference ratio, which is marked as PL; Analyze the deviation difference of the acceleration of the monitoring target at the preset time interval to obtain the difference deviation value, which is marked as CD; The deviation difference ratio PL and the difference deviation degree value CD are processed and the formula is used Obtain the speed characterization value BZ; Where k1 and k2 are preset proportional coefficients.

9. The target detection system based on radar and video fusion according to claim 8, characterized in that: The deviation difference ratio is obtained as follows: Get the first-order difference ΔX of the acceleration x1, x2, x3…xn of the monitoring target at the preset time interval i , where ΔX i =X i+1 -X i , the first-order difference of acceleration ΔX i Compared with the first-order difference threshold of acceleration, if the first-order difference of acceleration ΔX i If the acceleration is greater than or equal to the first-order difference threshold, the corresponding first-order difference of acceleration ΔX i Recorded as deviation difference; The number of deviation differences is counted, and the number of deviation differences is compared with the total number of differences to obtain the deviation difference ratio, which is marked as PL.

10. The target detection system based on radar and video fusion according to claim 8, characterized in that: The method for obtaining the difference deviation value is as follows: The first-order difference of acceleration corresponding to the deviation difference ΔX i Perform difference processing with the first-order difference threshold of acceleration to obtain a difference deviation value; count all the difference deviation values ​​that deviate from the difference, sum and average them, and obtain the difference deviation mean; The difference deviation mean is ratioed with the acceleration first-order difference threshold to obtain the difference deviation degree value, which is marked as CD.

Images