A method for detecting and tracing high-altitude objects by integrating kinematics and machine vision

By integrating kinematics and machine vision, high-definition surveillance video is used to detect and trace the source of high-altitude throwing objects, combined with quadratic function fitting and inspection statistics, the source location of the parabola is determined, solving the problems of poor real-time and difficult traceability in the existing technology. , realizes efficient identification and traceability of parabolic motion trajectory.

CN115049606BActive Publication Date: 2024-12-03WUHAN UNIV OF TECH
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202210652512.0
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-06-07
Publication Date
2024-12-03
Estimated Expiration
2042-06-07

AI Technical Summary

Technical Problem

The existing high-altitude trajectory detection and traceability methods lack the use of the kinematic characteristics of parabolic motion for trajectory analysis, resulting in poor real-time performance and difficulty in traceability, and it is difficult to effectively screen the trajectory unless parabolic motion is parabolic, which increases the workload for subsequent processing.

Method used

Using a method of fusion kinematics and machine vision, preprocessing and background modeling is performed through high-definition surveillance camera video, quasi-parabolic motion trajectory is screened, parabolic motion is determined using quadratic function fitting and testing statistics, and parabolic source is determined based on curve characteristics. Position and time reduce the amount of subsequent fitting operations and improve processing speed.

Benefits of technology

Real-time detection and traceability of objects thrown at high altitudes, reduce the identification error of non-parabolic motion trajectories, improve processing speed and accuracy, and can trace the source of parabolics with starting points outside the picture, with high detection rate and resistance. Noise capability.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN115049606B_ABST
    Figure CN115049606B_ABST
Patent Text Reader

Abstract

The present invention proposes a high-altitude projectile detection and tracing method that integrates kinematics and machine vision, including: S1 preprocessing images captured by monitoring, background modeling of the preprocessed images, post-processing of foreground images obtained by background modeling, obtaining circumscribed rectangular frames of all foreground areas, expanding all foreground frames, and fusing foreground frames with intersecting areas to obtain rectangular frames of various moving targets; S2 multi-target tracking of all fused rectangular frames to obtain a trajectory point sequence of each target movement; S3 preliminarily screening out a quasi-parabolic motion trajectory from the tracked trajectory, fitting a quadratic function of the ordinate of the quasi-parabolic motion trajectory to time, further determining whether it is a parabolic motion by a test statistic indicator, and if so, the fitting result is a parabolic motion trajectory equation, saving relevant forensic videos, and calculating the vertex of the parabolic trajectory by the quadratic function vertex coordinate formula, which is the source of the parabola.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, such as background modeling, target tracking, etc., and also combines the kinematic characteristics of parabolic motion, specifically referring to a high-altitude parabolic detection and tracing method that integrates kinematics and machine vision. Background Art

[0002] The randomness, suddenness and high speed of objects thrown from high places have been called the "pain hanging over the city". Throwing objects from buildings or other high places, if the circumstances are serious, is a violation of the criminal law.

[0003] The current community prevention and control methods are still relatively old. After an incident, it is often necessary to search for only a few seconds of parachuting video in more than ten hours of surveillance footage. This not only wastes a lot of manpower and resources, but also has poor real-time performance and is difficult to trace the source.

[0004] The methods for detecting and tracing high-altitude objects provided by the prior art lack research on using the inherent kinematic characteristics of parabolic motion to analyze trajectories and trace the source based on the vertices of parabolas. For example, the Chinese patent "A method for detecting high-altitude objects with real-time multi-environment adaptive trajectory automatic tracing" (ZL202110340688.8) only performs background modeling, directly performs connected domain analysis on the results of morphological filtering, generates a trajectory probability map, and obtains the trajectory with the highest probability. However, there is a problem that the differential results are highly dispersed and thus aggregated into multiple targets, and the method of expanding the foreground frame and then fusing it used in the present invention can solve this problem. Another example is the Chinese patent "A system and method for detecting high-altitude objects based on computer vision" (ZL201811015244.1), after target tracking, it only turns the tracking starting point back on the trajectory to a point where the y-axis displacement is less than the threshold as the source of the parabola, and still does not break through the limitation of the tracking point. Others, such as the Chinese patent "Method, Equipment and Storage Medium for Detecting High-Altitude Thrown Objects" (ZL202010701041.9), although the trajectory was preliminarily screened and relevant physical formulas were used, the starting point coordinates were directly obtained by subtracting the motion displacement from the end point coordinates. The acceleration and velocity in the formula used only used the last two points in the trajectory, without considering the situation that noise points will appear at the end of the trajectory obtained by actual tracking, and the data of previous points are not used. Summary of the invention

[0005] The purpose of the present invention is to address the deficiencies of the prior art and propose a method for detecting and tracing high-altitude objects that integrates kinematics and machine vision. The video captured by a high-definition surveillance camera is used to achieve real-time detection and tracing of high-altitude objects. The inherent kinematic characteristics of high-altitude objects are fully utilized in trajectory analysis. Most of the motion trajectories that do not belong to objects are filtered out in the preliminary screening, reducing the workload of subsequent trajectory fitting and improving the processing speed. In the trajectory fitting, the curve characteristics and actual scenes are combined to propose a method for determining the location and time of the source of objects under various conditions.

[0006] To achieve the above purpose, the present invention is a method for detecting and tracing high-altitude objects by integrating kinematics and machine vision, which is special in that the method comprises the steps of:

[0007] S1 pre-processes the building video captured by the high-definition camera used for monitoring as the input image, performs background modeling on the pre-processed image, and post-processes the foreground image obtained by background modeling to obtain the circumscribed rectangular frame of all foreground areas, namely the foreground frame, expands all foreground frames, and merges the foreground frames with intersecting areas to obtain the rectangular frames of each moving target;

[0008] S2 performs multi-target tracking on all the fused moving target rectangles to obtain the trajectory point sequence of each target;

[0009] S3 preliminarily screens out the quasi-parabolic motion trajectory from the tracked trajectory, selects the quasi-parabolic motion point sequence, fits the ordinate of the quasi-parabolic motion point sequence to time as a quadratic function, and further determines whether it is a parabolic motion by the test statistic indicator. If so, the fitting result is the parabolic motion trajectory equation, saves the relevant forensic video, and calculates the vertex of the parabolic trajectory by the quadratic function vertex coordinate formula, which is the parabola source.

[0010] Preferably, the specific steps of step S1 include:

[0011] S11 preprocesses each frame of input image to remove noise in the image and sharpen and enhance the image edge;

[0012] S12 uses the kNN background subtraction method with shadow detection function to model the background of the preprocessed image sequence and obtain a grayscale image with pixel values ​​distributed in [0,255];

[0013] S13 performs binarization processing on the obtained grayscale image, taking 244 as a threshold value, and judging the pixel value greater than the threshold value as a foreground point, and setting the pixel value to 255, otherwise it is judged as a background point, and the pixel value is set to 0, thereby obtaining a binary image;

[0014] S14 performs morphological filtering on the binary image;

[0015] S15 calculates the area of ​​each foreground region obtained after morphological filtering, and the region whose area exceeds the set threshold is framed by its circumscribed rectangle;

[0016] S16 expands all circumscribed rectangles to a set distance;

[0017] S17 searches for common areas between the expanded rectangular frames, and if so, merges them to obtain a final foreground frame list.

[0018] Preferably, the specific steps of step S2 include:

[0019] S21 extracts the center of each moving object from the foreground frame list;

[0020] S22 updates a multi-target tracker with the list of moving target centers. The multi-target tracker adopts a SORT algorithm, which is a fusion of a Hungarian algorithm and a Kalman filter. The Hungarian algorithm assigns target centers to existing trajectories. The Kalman filter establishes a motion model for each trajectory, takes the points newly assigned to the trajectory as observation values, and adds the predicted true position of the target center to the trajectory point sequence of the trajectory.

[0021] The S23 multi-target tracker outputs a sequence of trajectory point coordinates for each trajectory.

[0022] Preferably, the specific steps of step S3 include:

[0023] S31 preliminarily screening out quasi-parabolic motion trajectories that conform to the basic law of parabolic motion from the trajectories tracked by the multi-target tracker;

[0024] S32 For the quasi-parabolic motion trajectory, the ordinates of each trajectory point are fitted with a quadratic function to obtain the analytical expression of the fitted parabola and the coefficient of determination R 2 ;

[0025] S33 for the coefficient of determination R 2 Set the threshold R2_thresh. The trajectory greater than the threshold is considered as a parabolic motion trajectory, and save the evidence video;

[0026] S34 calculates the horizontal and vertical coordinates of the vertex (f 0 ,y min );

[0027] S35 performs a linear fit on the horizontal coordinates of each trajectory point to obtain a linear function and calculates the trajectory point at the fth 0 The horizontal coordinate value x of the vertex in the frame image 0 ;

[0028] S36 is composed of (f 0 ,x 0 ,y min)Obtain the time and specific position when the parabolic motion occurs.

[0029] Preferably, the preliminary screening scheme in step S31 is specifically as follows:

[0030] For the ordinate sequence [y 1 , y 2 , y 3 ,..., y len2 of the trajectory to be analyzed from the back to the front in time, calculate [h 1 = y 1 - y 2 , h 2 = y 2 - y 3 ,..., h len2-1 = y len2-1 - y len2 . Then the length len of the h sequence is len2 - 1, which represents the displacement of the target in the y direction between each frame. Also, since the time interval between each frame of the video captured by the camera is equal, the h sequence is equivalent to the velocity sequence of the target's motion in the y direction between each frame;

[0031] Design two indicators: fall height judgment and acceleration judgment:

[0032] 1) Fall height judgment: Define a fall height threshold. When the algebraic sum of the h sequence exceeds this threshold, the fall height judgment result is true;

[0033] 2) Acceleration judgment: Define an acceleration point ratio threshold. When the ratio of the acceleration points in the h sequence exceeds this threshold, the acceleration judgment result is true;

[0034] Take the acceleration judgment and / or fall height judgment result as the preliminary parabolic screening result, and the trajectory passing through this round of parabolic determination is the quasi-parabolic motion trajectory.

[0035] Preferably, the definition method of the acceleration point is

[0036] a) For each point h[i] in the h sequence, define the point h[i] that satisfies h[i] > h[i + 1] as the acceleration point, where i <= len - 1;

[0037] And / or b) Supplement the acceleration judgment threshold parameter, compare h[i] (i <= len - 1) with all h[j] that satisfy i < j <= len. When the ratio of the points h[j] that satisfy h[i] > h[j] exceeds the threshold parameter, it is judged as an acceleration point.

[0038] Preferably, the calculation method of the vertex ordinate in step S34 is specifically as follows:

[0039] Perform a quadratic function fitting on the ordinate with respect to the number of frames. The obtained parabolic trajectory equation is:

[0040] y=b 1 f 2 +b 2 f+b 3

[0041] The vertex coordinates are:

[0042]

[0043]

[0044] Where b 1 is the coefficient of its quadratic term, b 2 is the coefficient of its first-order term, b 3 is its constant term.

[0045] Preferably, in step S35, the horizontal coordinate value x at the vertex is calculated. 0 The specific method is:

[0046] The horizontal axis is fitted with a function of the frame number, and the linear equation of the horizontal axis obtained by fitting is:

[0047] x=c 1 f+c 2

[0048] Its determination coefficient is In f 0 Department:

[0049] x 0 =c 1 f 0 +c 2

[0050] In the formula, c 1 Its first-order coefficient, c 2 is its constant term;

[0051] Get all the information about the parabolic motion vertex [f 0 ,x 0 ,y min ];

[0052] In step S36, the time and specific location of the parabolic motion are determined to pass the preliminary screening and R 2 >R2_thresh is used as the criterion for parabolic motion and is calculated from the fitting results [x 0 ,y min ], then the coordinates of the parabola source in the video frame [x s ,y s The calculation formula of ] is:

[0053]

[0054]

[0055] Let t be the moment when the trajectory is judged to be parabolic motion 1 , the number of frames saved by the tracker is len, the frame rate of the camera is fps, and the time when the parabola occurs is:

[0056]

[0057] Preferably, the method for determining whether there is a common area between the two rectangular frames in step S17 is as follows:

[0058] Suppose the boundary positions of the two rectangular boxes are [left1, top1, right1, bottom1] and [left2, top2, right2, bottom2] respectively, inters indicates whether they intersect, and the expression is:

[0059] inters=! (left2>right1||top2>bottom1||right2 <left1||bottom2<top1)

[0060] When adding a rectangular box, compare it with all the rectangular boxes in the result and determine whether they intersect. If they intersect, update the rectangular box to be added to the minimum external rectangular box that can accommodate both rectangular boxes at the same time, delete the fused rectangular box in the result, compare and merge the fused rectangular box with all the rectangular boxes in the result again, until they cannot be merged, and add the fused rectangular box to the result.

[0061] The present invention also proposes a computer device, which includes a memory, a processor, and program instructions stored in the memory for the processor to run, wherein the processor executes the program instructions to implement the above-mentioned high-altitude thrown object detection and tracing method that integrates kinematics and machine vision.

[0062] The present invention proposes a high-altitude projectile detection and tracing method that integrates kinematics and machine vision. The kinematic characteristics of high-altitude projectiles are used for preliminary screening before fitting, thereby reducing the amount of computational complexity of parabola fitting. After fitting, instead of simply taking the starting point of trajectory tracing as the starting point of the parabola, the vertex of the parabola with a longitudinal velocity of 0 is taken as the source of the parabola, so that the algorithm can even trace the source of a parabola whose starting point falls outside the range of the picture.

[0063] Compared with the prior art methods for detecting and tracing objects thrown from high altitudes, the present invention has the following advantages:

[0064] 1) Make full use of the kinematic characteristics of high-altitude parabolic objects. Instead of simply taking the starting point of trajectory tracking as the starting point of the parabola, the vertex of the parabola with a longitudinal velocity of 0 is taken as the source of the parabola. This gives the algorithm an inference function and enables it to trace the source of parabolic objects whose starting points fall outside the image range.

[0065] 2) Using the kinematic characteristics of high-altitude parabolic objects, the trajectories of all tracking targets are preliminarily screened. The preliminary screening scheme can screen out most non-parabolic movements, including the movement of insects, birds, people, and cars, and the shaking of leaves caused by wind, which greatly reduces the amount of computation for subsequent regression analysis. At the same time, the scheme has a certain degree of noise resistance, and can identify a certain number of noise points in the real parabolic motion trajectory with a high recall rate;

[0066] 3) By expanding and fusing the foreground frame, a large number of adjacent or even intersecting small target frames are merged into a large target frame, avoiding the problem of target frame splitting under textured backgrounds and improving the accuracy of subsequent target tracking;

[0067] 4) In the trajectory update method of the Hungarian algorithm, the retention of the unupdated trajectory allows the target frame to be correctly tracked even if it is temporarily lost, thus improving the stability of target tracking;

[0068] 5) The final parabolic trajectory must pass both preliminary screening and the determination coefficient test of regression analysis to ensure the accuracy of the detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] Figure 1 The present invention is a flow chart of a method for detecting and tracing high-altitude projectiles by integrating kinematics and machine vision.

[0070] Figure 2 Schematic diagram of the fitting curve in the embodiment of the present invention. DETAILED DESCRIPTION

[0071] In order to make the technical problems, technical solutions and advantages to be solved by the present invention more clear, they will be described in detail below with reference to the accompanying drawings and specific embodiments. However, the following embodiments are only illustrative, and the protection scope of the present invention is not limited to these embodiments.

[0072] like Figure 1 As shown, an embodiment of the present invention provides a method for detecting and tracing high-altitude objects by integrating kinematics and machine vision, comprising the following steps:

[0073] S1 pre-processes the building video captured by the high-definition camera used for monitoring as the input image, performs background modeling on the pre-processed image, and post-processes the foreground image obtained by background modeling to obtain the circumscribed rectangular frame of all foreground areas, namely the foreground frame, expands all foreground frames, and merges the foreground frames with intersecting areas to obtain the rectangular frames of each moving target. The specific steps include:

[0074] S11 preprocesses each frame of input image, including removing noise in the image by using methods such as median filtering, and the size of the median filter template is 3*3; sharpening and enhancing the image edge, and the Laplace sharpening operator template used is [0,-1,0; -1,5,-1; 0,-1,0].

[0075] S12 uses the kNN background subtraction method with shadow detection function to model the background of the preprocessed image sequence and obtain a grayscale image with pixel values ​​distributed in [0, 255]. The kNN background subtraction method is implemented using OpenCV and has two main parameters: (1) history, which is the number of historical frames considered when determining the foreground point, and it is recommended to be set to 30; (2) dist2Threshold, which is the distance threshold from the background model, and it is recommended to be set to 2000.

[0076] S13 performs binarization processing on the obtained grayscale image, taking 244 as the threshold value, and judging the pixel value greater than the threshold value as the foreground point, and setting the pixel value to 255, otherwise it is judged as the background point, and the pixel value is set to 0, thereby obtaining a binary image.

[0077] S14 performs morphological filtering on the binary image. The specific method of morphological filtering is to perform two consecutive opening operations using an elliptical kernel with a size of (3, 3).

[0078] S15 calculates the area of ​​each foreground region obtained after morphological filtering, and the region whose area exceeds the set threshold is framed by its circumscribed rectangle; the threshold is recommended to be set to 200.

[0079] S16 expands all bounding rectangles to the set distance; the recommended expansion distance is 5, unless it hits the image boundary.

[0080] S17 searches for common areas between the expanded rectangular frames, and if so, merges them to obtain a final foreground frame list. The method for determining whether two rectangular frames have common areas is as follows:

[0081] Suppose the boundary positions of the two rectangular boxes are [left1, top1, right1, bottom1] and [left2, top2, right2, bottom2] respectively, inters indicates whether they intersect, and the expression is:

[0082] inters=! (left2>right1||top2>bottom1||right2 <left1||bottom2<top1)

[0083] When adding a rectangular box, compare it with all the rectangular boxes in the result and determine whether they intersect. If they intersect, update the rectangular box to be added to the minimum external rectangular box that can accommodate both rectangular boxes at the same time, delete the fused rectangular box in the result, compare and merge the fused rectangular box with all the rectangular boxes in the result again, until they cannot be merged, and add the fused rectangular box to the result.

[0084] S2 performs multi-target tracking on all the fused moving target rectangles to obtain the trajectory point sequence of each target. The specific steps include:

[0085] S21 extracts the center of each moving object from the foreground frame list.

[0086] S22 updates the multi-target tracker with a list of moving target centers. The multi-target tracker uses the SORT algorithm, which is a fusion of the Hungarian algorithm and the Kalman filter. The Hungarian algorithm assigns the target center to the existing trajectory. The Kalman filter establishes a motion model for each trajectory, takes the newly assigned points to the trajectory as observation values, and adds the predicted true position of the target center to the trajectory point sequence of the trajectory.

[0087] There are three main parameters when using the Hungarian algorithm for allocation: (1) dist_thresh, which is the maximum point distance for allocation, and is recommended to be set to 100; (2) max_frame_to_skip, which is the maximum number of frames retained for trajectories that are not allocated to points, and is recommended to be set to 8; (3) max_trace_length, which is the maximum number of path points retained for each target, and is recommended to be set to 15.

[0088] It is recommended that dt in the Kalman filter model be set to 0.2.

[0089] The S23 multi-target tracker outputs a sequence of trajectory point coordinates for each trajectory.

[0090] The specific method of trajectory update in the Hungarian algorithm is: when a trajectory is not assigned to a new point in a certain round of update, the trajectory is still retained and continues to participate in the allocation in the next round. It is not determined that the target corresponding to the trajectory is lost until it is not updated for 8 consecutive times. This ensures that the target can be found even if it is temporarily lost during tracking.

[0091] S3 preliminarily screens out the quasi-parabolic motion trajectory from the tracked trajectory, selects the quasi-parabolic motion point sequence, fits the ordinate of the quasi-parabolic motion point sequence to time as a quadratic function, and further determines whether it is a parabolic motion by the test statistic index. If it is, the fitting result is the parabolic motion trajectory equation, saves the relevant forensic video, and calculates the parabolic trajectory vertex as the parabola source by the quadratic function vertex coordinate formula. The specific steps include:

[0092] S31 uses a preliminary screening scheme to select quasi-parabolic motion trajectories that conform to the basic laws of parabolic motion from the trajectories obtained by SORT tracking.

[0093] S32 aligns the parabolic motion trajectory, fits the ordinates of each trajectory point with a quadratic function, and obtains the analytical expression of the fitted parabola and the coefficient of determination R 2 It is recommended to set R2_thresh = 0.8.

[0094] S33 vs R 2 Set the threshold R2_thresh. Trajectories greater than the threshold are considered parabolic motion trajectories and the forensic video is saved.

[0095] S34 calculates the horizontal and vertical coordinates of the vertex (f 0 ,y min ).

[0096] S35 performs a linear fit on the horizontal coordinates of each trajectory point to obtain a linear function and calculates its 0 The value x at 0 .

[0097] S36 is composed of (f 0 ,x 0 ,y min ) to obtain the time and specific location of the parabolic motion.

[0098] In this embodiment, the preliminary screening scheme in step S31 is specifically as follows:

[0099] The ordinate sequence of the trajectory to be analyzed [y 1 ,y 2 ,y 3 ,...,y len2 ](from back to front in time) to find [h 1 =y 1 -y 2 ,h 2 =y 2 -y 3 ,...,h len2-1 =y len2-1 -y len2] represents the displacement of the target in the y direction between each frame. Since the time interval between each frame of the video shot by the camera is equal, the h sequence can be equivalent to the speed sequence of the target moving in the y direction between each frame. Considering the two basic characteristics of parabolic motion: accelerated motion and long and downward displacement in the y direction, the following two indicators are designed:

[0100] 1) Drop height judgment: define a drop height threshold. When the algebraic sum of the h sequence exceeds the threshold, the drop height judgment result is true;

[0101] 2) Acceleration judgment: define the acceleration point ratio threshold. When the ratio of acceleration points in the h sequence exceeds the threshold, the acceleration judgment result is true.

[0102] Among them, the definition of the acceleration point in the second indicator acceleration judgment is specifically the following two, both of which are within the scope of protection of the present invention:

[0103] a) Define the point h[i] that satisfies h[i]>h[i+1] as the acceleration point;

[0104] b) Supplement the acceleration judgment threshold parameter, compare h[i] with all h[j] (j>i), and judge it as an acceleration point only when the proportion of points h[j] satisfying h[i]>h[j] exceeds the threshold.

[0105] The acceleration judgment and drop height judgment results obtained can be combined with actual effects, and one of them can be selected as the parabolic judgment result.

[0106] In this embodiment, the drop height judgment threshold is set to 1 / 4 of the screen height, the acceleration ratio threshold in the acceleration judgment is recommended to be set to 0.7, and the acceleration point (definition 2) judgment threshold is recommended to be set to 0.7. It is recommended to perform an OR operation on the acceleration judgment and the drop height judgment results as the final parabolic judgment result.

[0107] In this embodiment, the vertex calculation method in step S34 is specifically as follows:

[0108] Fit the quadratic function of the ordinate to the frame number, and the parabolic trajectory equation obtained by fitting is:

[0109] y=b 1 f 2 +b 2 f+b 3

[0110] b 1 is the coefficient of its quadratic term, b 2 is the coefficient of its first-order term, b 3 is its constant term, and its vertex coordinates are:

[0111]

[0112]

[0113] The specific fitting method is to use the ordinate y of each point on the trajectory to be analyzed as the dependent variable, and the corresponding subscript, that is, the frame number f (time is counted from the front to the back, with the point farthest from the present saved by the tracker as the 0th frame) as the independent variable, to construct the following three independent variable sequences:

[0114] f 1 =[0 2 ,1 2 ,2 2 ,...,(len-1) 2 ]

[0115] f 2 =[0,1,2,...,len-1]

[0116] f 3 =[1,1,1,…,1](len)

[0117] The y sequence is f 1 ,f 2 ,f 3 Perform multiple linear regression on the sequence, using the built-in regress function in Matlab, and return the regression coefficients [b 1 ,b 2 ,b 3 ] and the coefficient of determination R 2 .

[0118] In this embodiment, the calculation of x in step S35 is 0 The specific method is:

[0119] Perform a function fit on the horizontal axis to the number of frames, and the linear equation of the horizontal axis obtained by fitting is:

[0120] x=c 1 f+c 2

[0121] c 1 Its first-order coefficient, c 2 is its constant term, and its coefficient of determination is In f 0 Department:

[0122] x 0 =c 1 f 0 +c 2

[0123] So far, all the information about the vertex of parabolic motion is obtained [f 0 ,x 0 ,y min ].

[0124] In this embodiment, the parabolic trajectory equation obtained by quadratic function fitting is y=1.0401f 2 +23.0760f+567.9726, the vertex coordinates are calculated as (y min =439.9849,f 0 =-11.0927). The linear equation of the horizontal coordinate obtained by first-order function fitting is: x = 1.4446f + 452.5120, and the calculated f 0 The corresponding horizontal coordinate x 0 =468.5364, the fitting curve is as follows Figure 2 shown.

[0125] In this embodiment, the method for obtaining the starting position and time of the parabolic motion in step S36 is specifically as follows:

[0126] If we accept the assumption that "there is no vertical initial velocity at the starting point of the parabola" (generally, unintentional parabolas all have this characteristic, let it be hyp1), then the vertex of the parabola is the source of the parabola, and the vertical coordinate y in the video frame is s =y min .

[0127] If we do not accept this assumption, we can at least assert that y s ≥y min .

[0128] In addition, the linear fitting coefficient of determination for the horizontal Set a threshold Rx2_thresh, it is recommended to take Rx2_thresh = 0.7. If the wind force is greater than the threshold, it is considered that the wind force has little effect on the lateral motion of the dropped object. 0 Available, the horizontal coordinate x of the parabola source in the video frame s =x 0 .

[0129] In summary, through the initial screening and R 2 >R2_thresh is used as the criterion for parabolic motion and is calculated from the fitting results [x 0 ,y min ], from which it can be deduced that [x s ,y s ]:

[0130]

[0131]

[0132] In addition, let t be the time when the trajectory is judged to be parabolic motion 1, the number of frames saved by the tracker is len, the frame rate of the camera is fps, and the time when the parabola occurs is:

[0133]

[0134] Finally, it should be noted that [x s ,y s ] is the coordinate of the parabola source in the video frame, which may be outside the picture range. To correspond to the actual position, it is necessary to combine the angle of the camera placement and the distance from the target building for specific calculation.

[0135] When determining responsibility, if hyp1 is accepted as the starting point of the parabolic motion, the vertical position of the source of the parabolic motion can be accurately determined, that is, the floor where it is located, and only the residents on that floor will be punished. Even if hyp1 is not accepted as the starting point of the parabolic motion, it can be determined that the source of the parabolic motion cannot be higher than a certain floor, and only the residents on that floor and below will be punished, thus avoiding blindly punishing all residents in the building once a parabolic accident occurs. If the parabolic lateral motion fitting effect is good, the horizontal position of the source of the parabolic motion in the floor can also be determined, that is, the window where it is located. For buildings with multiple residents on the same floor, the punishment can be accurate to each household.

[0136] The above specific examples are used to illustrate the present invention, which is only used to help understand the present invention and is not intended to limit the present invention. For technicians in the technical field to which the present invention belongs, some simple deductions, deformations or substitutions can also be made based on the ideas of the present invention. It is easy for technicians in this field to understand that the above is only a preferred embodiment of the present invention patent, and is not intended to limit the present invention patent. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention patent should be included in the protection scope of the present invention patent.

[0137] The contents not described in detail in this specification belong to the prior art known to professional and technical personnel in this field.

Claims

1. A method for detecting and tracing high-altitude objects by integrating kinematics and machine vision, characterized in that: The method includes the following steps: S1 Preprocess the building video captured by the high-definition camera for monitoring as the input image, perform background modeling on the preprocessed image, post-process the foreground image obtained from the background modeling to obtain the circumscribed rectangular frames of all foreground regions, i.e., foreground frames, expand all foreground frames, and fuse the foreground frames with intersecting regions to obtain the rectangular frames of each moving target; S2 Perform multi-target tracking on all the rectangular frames of the moving targets after fusion to obtain the sequence of trajectory points of each target's movement; S3 Initially screen out the quasi-parabolic motion trajectories from the tracked trajectories, select the quasi-parabolic motion point sequence, fit the ordinate of the quasi-parabolic motion point sequence to time as a quadratic function, and further determine whether it is a parabolic motion by the test statistic index. If so, the fitting result is the parabolic motion trajectory equation, save the relevant evidentiary video, and calculate the vertex of the parabolic trajectory by the quadratic function vertex coordinate formula, which is the parabolic source; The specific steps of step S3 include: S31 Initially screen out the quasi-parabolic motion trajectories that conform to the basic law of parabolic motion from each trajectory obtained by the multi-target tracker; S32 For the quasi-parabolic motion trajectory, the ordinates of each trajectory point are fitted with a quadratic function to obtain the analytical expression of the fitted parabola and the coefficient of determination R 2 ; S33 for the coefficient of determination R 2 Set the threshold R2_thresh. The trajectory greater than the threshold is considered as a parabolic motion trajectory, and save the evidence video; S34 calculates the vertex ordinate (f0, y min ); S35 Perform a linear fit on the abscissas of each trajectory point to obtain a linear function, and calculate the abscissa value x0 at the vertex of the trajectory point in the f0-th frame image; S36 is composed of (f0,x0,y min ) to obtain the time and specific location of the parabolic motion; The specific initial screening scheme in step S31 is: For the ordinate sequence [y1,y2,y3,...,y len2 ] Find [h1=y1-y2,h2=y2-y3,...,h len2-1 =y len2-1 -y len2 ], then the length of the h sequence len = len2-1, which represents the displacement of the target in the y direction between each frame. Since the time interval between each frame of the video shot by the camera is equal, the h sequence is equivalent to the speed sequence of the target moving in the y direction between each frame; Design two indicators: fall height judgment and acceleration judgment: 1) Fall height judgment: Define the fall height threshold. When the algebraic sum of the h sequence exceeds this threshold, the fall height judgment result is true; 2) Acceleration judgment: Define the acceleration point ratio threshold. When the ratio of acceleration points in the h sequence exceeds this threshold, the acceleration judgment result is true; Take the acceleration judgment and / or fall height judgment result as the preliminary screening result for parabola, and the trajectory passing through this round of parabola determination is the quasi-parabolic motion trajectory; The definition method of the acceleration point is a) For each point h[i] in the h sequence, define the point h[i] that satisfies h[i]>h[i + 1] as the acceleration point, where i <= len - 1; and / or b) Supplement the acceleration judgment threshold parameter, compare h[i] (i <= len - 1) with all h[j] that satisfy i < j <= len, and when the ratio of the points h[j] that satisfy h[i]>h[j] exceeds the threshold parameter, it is judged as an acceleration point; The specific calculation method of the vertex ordinate in step S34 is: Fit the ordinate to the number of frames as a quadratic function, and the obtained parabolic trajectory equation is: y=b1f 2 +b2f+b3 Then its vertex coordinates: In the formula, b1 is its quadratic term coefficient, b2 is its linear term coefficient, and b3 is its constant term; The specific method for calculating the abscissa value x0 at the vertex in step S35 is: Fit the abscissa to the number of frames as a linear function, and the obtained abscissa linear equation is: x = c1f + c2 Its determination coefficient is At f0: x0 = c1f0 + c2 In the formula, c1 is its linear term coefficient, and c2 is its constant term; Get all the information of the parabolic motion vertex [f0,x0,y min ]; In step S36, the time and specific location of the parabolic motion are determined to pass the preliminary screening and R 2 >R2_thresh is used as the criterion for parabolic motion and is calculated from the fitting results [x0,y min ], then the coordinates of the parabola source in the video frame [x s ,y s The calculation formula of ] is: Let the moment when the trajectory is judged as a parabolic motion be t1, the number of frames saved by the tracker at this time is len, and the shooting frame rate of the camera is fps, then the moment when the parabola occurs is:

2. According to claim 1, a method for detecting and tracing high-altitude objects by integrating kinematics and machine vision is characterized in that: The specific steps of step S1 include: S11 preprocesses each frame of input image to remove noise in the image and sharpen and enhance the image edge; S12 uses the kNN background subtraction method with shadow detection function to model the background of the preprocessed image sequence and obtain a grayscale image with pixel values ​​distributed in [0,255]; S13 performs binarization processing on the obtained grayscale image, taking 244 as a threshold value, and judging the pixel value greater than the threshold value as a foreground point, and setting the pixel value to 255, otherwise it is judged as a background point, and the pixel value is set to 0, thereby obtaining a binary image; S14 performs morphological filtering on the binary image; S15 calculates the area of ​​each foreground region obtained after morphological filtering, and the region whose area exceeds the set threshold is framed by its circumscribed rectangle; S16 expands all circumscribed rectangles to a set distance; S17 searches for common areas between the expanded rectangular frames, and if so, merges them to obtain a final foreground frame list.

3. According to claim 1, a method for detecting and tracing high-altitude objects by integrating kinematics and machine vision is characterized in that: The specific steps of step S2 include: S21 extracts the center of each moving object from the foreground frame list; S22 updates a multi-target tracker with the list of moving target centers. The multi-target tracker adopts a SORT algorithm, which is a fusion of a Hungarian algorithm and a Kalman filter. The Hungarian algorithm assigns target centers to existing trajectories. The Kalman filter establishes a motion model for each trajectory, takes the points newly assigned to the trajectory as observation values, and adds the predicted true position of the target center to the trajectory point sequence of the trajectory. The S23 multi-target tracker outputs a sequence of trajectory point coordinates for each trajectory.

4. According to claim 2, a method for detecting and tracing high-altitude objects by integrating kinematics and machine vision is characterized in that: The method for determining whether there is a common area between two rectangular frames in step S17 is as follows: Suppose the boundary positions of the two rectangular boxes are [left1, top1, right1, bottom1] and [left2, top2, right2, bottom2] respectively, inters indicates whether they intersect, and the expression is: inters=! (left2>right1||top2>bootom1||right2 <left1||bottom2<top1) When adding a rectangular box, compare it with all the rectangular boxes in the result and determine whether they intersect. If they intersect, update the rectangular box to be added to the minimum external rectangular box that can accommodate both rectangular boxes at the same time, delete the fused rectangular box in the result, compare and merge the fused rectangular box with all the rectangular boxes in the result again, until they cannot be merged, and add the fused rectangular box to the result.

5. A computer device comprising a memory, a processor and program instructions stored in the memory and executable by the processor, wherein the processor executes the program instructions to implement the method according to any one of claims 1 to 4.