Parabolic event detection method and device, equipment and storage medium

Abstract

According to the parabolic event detection method and device, the equipment and the storage medium provided by the invention, the to-be-processed video stream frame image is acquired, background modeling is carried out, and the foreground motion area is extracted; in the foreground motion area, extracting image blocks corresponding to the throwing object candidate target, synthesizing the image blocks into an image, and inputting the image into a throwing object classification model to obtain the probability that the throwing object candidate target is a throwing object; thirdly, a matching area is constructed according to the historical motion trail of the throwing object candidate target, cross-frame tracking is carried out, a target motion trail is obtained, and the target motion trail is input into the trail classification model to determine whether the target motion trail is a parabolic trail or not; according to the method, the image buffer area is constructed and the image blocks with continuous time are extracted, so that the complete appearance of the throwing object can be captured in different frames, and the classification accuracy is facilitated; in addition, the throwing object classification model and the track classification model are respectively used for intelligently judging the image content and the motion track, so that the false detection rate and the omission ratio are effectively reduced.

Description

Technical Field

[0001] This invention relates to the field of image recognition technology, and in particular to a method, apparatus, device, and storage medium for detecting parabolic events. Background Art

[0002] With the widespread application of video surveillance systems and intelligent analytics technologies, the demand for automated detection of abnormal events is increasing in fields such as public safety, traffic management, and campus surveillance. Especially in specific scenarios, such as prisons, schools, airports, and sports stadiums, there is a security risk of objects being thrown across monitored areas, necessitating automated methods to accurately identify and warn of "parabolic events."

[0003] Existing technologies for detecting projectiles often rely on manual monitoring or simple motion detection algorithms, which have the following shortcomings: Difficulty in accurately extracting projectile targets: Projectiles are usually small in size, move at high speed, and appear in complex backgrounds, making it difficult for traditional methods to accurately segment and identify them; Inaccurate trajectory recognition: Most existing motion analysis techniques cannot effectively model the complete trajectory of an object, especially for occluded or fast-moving targets, which can easily cause trajectory interruption or recognition errors. Lack of intelligent classification mechanism: Even if a moving target is detected, there is a lack of effective classification model to determine whether the target is a projectile, which can easily lead to false alarms; Lack of real-time alarm function: Some methods do not link the detection results with the real-time alarm mechanism, making it impossible to notify relevant personnel in the first instance when an incident occurs.

[0004] In summary, the problems existing in the current technology urgently need to be solved. Summary of the Invention

[0005] This invention provides a method, apparatus, device, and storage medium for detecting parabolic events, which addresses the deficiencies in the prior art and enables intelligent discrimination of image content and motion trajectory using projectile classification models and trajectory classification models, effectively reducing the false detection rate and false negative rate of projectiles.

[0006] This invention provides a method for detecting parabolic events, comprising: The process involves acquiring a video stream frame image to be processed, performing grayscale processing on the current frame image to obtain a grayscale image, and caching the grayscale image into an image buffer, which is used to store the grayscale processed frame image. Background modeling is performed on the grayscale image to extract the foreground motion region; In each frame of the image buffer, a corresponding image block is extracted based on the position of each projectile candidate target; The image patches are combined into a projectile candidate image corresponding to the projectile candidate target, and then input into a pre-trained projectile classification model to obtain the projectile confidence score. If the confidence level of the projectile is higher than a preset threshold, a matching region is constructed based on the historical motion trajectory of the projectile candidate target; Based on the matching region, the candidate projectile target is tracked across frames to obtain the target's motion trajectory, which is then input into a pre-trained trajectory classification model to determine whether the target's motion trajectory is a parabolic trajectory.

[0007] According to a method for detecting parabolic events provided by the present invention, the image buffer includes the current frame image and at least two frames preceding the current frame image, for providing temporally continuous image data during image patch extraction.

[0008] According to a method for detecting parabolic events provided by the present invention, after the step of performing background modeling on the grayscale image and extracting the foreground motion region, the method further includes: An erosion operation is performed on the foreground motion region to remove noise; An expansion operation is performed on the foreground motion region to repair the broken areas in the contour edge of the projectile candidate target.

[0009] According to a method for detecting parabolic events provided by the present invention, after the step of determining whether the target trajectory is a parabolic trajectory, the method further includes: If the trajectory classification result is a parabolic motion trajectory, then the event information of the target motion trajectory is output; Based on the event information, an alarm message is generated. According to a method for detecting parabolic events provided by the present invention, the step of synthesizing the image patch into a candidate image of the projectile corresponding to the candidate projectile target specifically includes: The image blocks in the image buffer corresponding to the positions of the projectile candidate targets are sorted in a temporal order. The multiple image blocks are stacked or stitched together in spatial order in time sequence to generate temporally consecutive candidate images of projectiles. The candidate images of projectiles are normalized in size to meet the input requirements of the projectile classification model.

[0010] According to a method for detecting parabolic events provided by the present invention, the step of constructing a matching region based on the historical motion trajectory of the candidate projectile target specifically includes: The historical trajectory of the projectile candidate target is fitted, and the predicted position and velocity information of the projectile candidate target in the current frame image are predicted using a Kalman filter, wherein the velocity information includes the direction of motion; Based on the prediction results, an elliptical matching region is generated in the current frame image with the predicted position as the center and the major axis direction consistent with the direction of motion. The elliptical matching region is used to limit the search range for cross-frame tracking.

[0011] According to a method for detecting parabolic events provided by the present invention, the step of performing cross-frame tracking of the candidate projectile target based on the matching region to obtain the target's motion trajectory specifically includes: Within the matching region, find the detection target in the current frame image that best matches the candidate target; A cost matrix is ​​constructed based on the normalized Wasserstein distance between the candidate projectile targets and their historical trajectories; The Hungarian algorithm is used to associate detected targets in different frames to obtain the complete target motion trajectory.

[0012] The present invention also provides a device for detecting parabolic events, comprising: The image acquisition module is used to acquire the video stream frame image to be processed, perform grayscale processing on the current frame image to obtain a grayscale image, and cache the grayscale image in the image buffer, which is used to store the grayscale processed frame image; The background modeling module is used to model the background of the grayscale image and extract the foreground motion region; The image extraction module is used to extract an image patch corresponding to each projectile candidate target in the foreground motion region from multiple frames of images in the image buffer, wherein one frame of image contains one image patch corresponding to the projectile candidate target; The target classification module is used to synthesize the image patches into projectile candidate images corresponding to the projectile candidate targets, and input them into a pre-trained projectile classification model to obtain the projectile confidence score. The region construction module is used to construct a matching region based on the historical motion trajectory of the candidate projectile if the confidence level of the projectile is higher than a preset threshold. The trajectory classification module is used to perform cross-frame tracking of the projectile candidate target based on the matching region, obtain the target motion trajectory, and input it into a pre-trained trajectory classification model to determine whether the target motion trajectory is a parabolic trajectory.

[0013] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the method for detecting parabolic events as described above.

[0014] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method for detecting parabolic events as described above.

[0015] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements a method for detecting parabolic events as described above.

[0016] The present invention provides a method, apparatus, device, and storage medium for detecting parabolic events. It acquires video stream frames to be processed, performs background modeling, and extracts foreground motion regions. Within the foreground motion regions, it extracts image patches corresponding to candidate projectile targets, synthesizes these image patches into an image, and inputs it into a pre-trained projectile classification model to obtain the probability that the candidate projectile target is a projectile. Next, it constructs a matching region based on the historical motion trajectory of the candidate projectile target, performs cross-frame tracking to obtain the target's motion trajectory, and inputs this trajectory into a pre-trained trajectory classification model to determine whether the target's motion trajectory is a parabolic trajectory. By constructing an image buffer and extracting temporally continuous image patches, the present invention can capture the complete appearance of the projectile in different frames, which is beneficial to classification accuracy. Furthermore, by using both the projectile classification model and the trajectory classification model to intelligently distinguish image content and motion trajectory, it effectively reduces the false detection rate and the false negative rate. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0018] Figure 1 This is a flowchart illustrating the parabolic event detection method provided by the present invention; Figure 2 This is a schematic diagram of the structure of the parabolic event detection device provided by the present invention; Figure 3 It is a structural schematic diagram of the electronic device provided by the present invention. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0020] To address the problems in existing technologies, this invention proposes a method for detecting parabolic events. This method uses projectile classification and trajectory classification models to intelligently distinguish image content and motion trajectories, effectively reducing the false positive and false negative rates of projectiles. The method for detecting parabolic events is described below. Figure 1 As shown, including but not limited to the following steps: Step 110: Obtain the video stream frame image to be processed, perform grayscale processing on the current frame image to obtain a grayscale image, and cache the grayscale image in the image buffer, which is used to store the grayscale processed frame image.

[0021] In step 110, after receiving the real-time video stream, the system extracts images frame by frame and performs grayscale processing on each frame, converting the color image into a single-channel grayscale image to reduce the complexity of subsequent processing. The grayscale image is stored in an image buffer, which can cache the current frame image and the two frames preceding it, thereby forming an image sequence containing at least three frames, providing support for subsequent multi-frame information fusion and candidate target modeling.

[0022] Step 120: Perform background modeling on the grayscale image and extract the foreground motion region.

[0023] In step 120, a background modeling method based on the Mixture of Gaussians (MOG) model is used to model the grayscale image sequence and extract the moving foreground. Specifically, after initializing the MOG model, the system models the image sequence frame by frame and outputs a foreground mask. For the extracted foreground mask, morphological operations such as erosion and dilation are further applied to remove isolated noise pixels and repair breaks in the foreground contour, enhancing the structural integrity of the target region.

[0024] Step 130: In each frame of the image buffer, based on the position of each projectile candidate target, extract the corresponding image block.

[0025] In step 130, contour extraction is performed on each connected component in the foreground region, and the minimum bounding rectangle is calculated as a candidate target region. In each frame of the image buffer, an image patch is cropped from the region corresponding to the position of the candidate target in the current frame. Each frame corresponds to one image patch, usually three image patches, from the current frame and the two frames before it, reflecting the evolution characteristics of the candidate target in the time dimension.

[0026] Step 140: Combine the image patches into a projectile candidate image corresponding to the projectile candidate target, and input it into a pre-trained projectile classification model to obtain the projectile confidence score.

[0027] In step 140, the aforementioned multi-frame image blocks are merged along the channel dimension in chronological order to form a multi-channel image, where each channel corresponds to the state image of the candidate projectile at different time points. After the image is normalized in size and pixel value, it is input into a pre-trained projectile classification network (ThrowObjectNet). ThrowObjectNet is a convolutional neural network model that can effectively extract the temporal features of multi-frame images and output the probability value of the candidate target being a projectile.

[0028] Step 150: When the confidence level of the projectile is higher than a preset threshold, a matching region is constructed based on the historical motion trajectory of the projectile candidate target.

[0029] In step 150, if the probability of a projectile output by the ThrowObjectNet network is greater than a preset threshold (e.g., 0.45), the system considers the candidate target a valid detection target and enters the tracking process. At this point, the system maintains the target's historical trajectory and estimates its predicted state in the current frame using a Kalman filter. Combining the target's velocity direction and historical path information, an elliptical matching region is constructed centered on the predicted position, with its major axis aligned with the velocity direction. This region is used to limit the tracking search range and improve the accuracy of data association.

[0030] Step 160: Perform cross-frame tracking of the projectile candidate target based on the matching region to obtain the target motion trajectory, and input it into a pre-trained trajectory classification model to determine whether the target motion trajectory is a parabolic trajectory.

[0031] In step 160, within the aforementioned matching region, a cost matrix is ​​constructed by calculating the normalized Wasserstein distance (NWD) between all detected targets in the current frame and existing target trajectories. The optimal matching path is then solved using the Hungarian Algorithm to achieve precise data association between the detection results and the tracking trajectory. For successfully tracked target instances, the system continuously updates their trajectory points and records their cross-frame motion history.

[0032] When the target's lifecycle ends (e.g., disappears or comes to rest), the system inputs its complete motion trajectory into the trajectory classification network (ThrowTrackNet). This network is a trajectory recognition model based on the Transformer structure, capable of distinguishing between parabolic and non-parabolic trajectories, and ultimately outputting a classification result indicating whether the target's motion trajectory belongs to parabolic motion.

[0033] As a further optional embodiment, the image buffer includes the current frame image and at least two frames preceding the current frame image, for providing temporally continuous image data during image patch extraction.

[0034] In step 130, to obtain more complete information on target state changes, the system extracts image patches corresponding to each foreground moving target based on its contour coordinates in the current frame image, respectively, in the current frame and the two previous frames. By extracting image patches at the same spatial location from the three frames, the system can capture the appearance changes and displacement trends of the target in a short time series, thereby improving the temporal perception capability of the subsequent projectile image discrimination model.

[0035] Furthermore, during the image patch synthesis process, the system assigns these three image patches to the three channels of the synthesized image, transforming the original grayscale image sequence into a pseudo-color 3-channel image, which is then input into the pre-trained ThrowObjectNet projectile classification network as a candidate image for projectiles. This design fully utilizes temporal continuity information to more accurately distinguish between projectiles and non-projectiles.

[0036] As a further optional embodiment, after the step of performing background modeling on the grayscale image and extracting the foreground motion region, the method further includes: An erosion operation is performed on the foreground motion region to remove noise; An expansion operation is performed on the foreground motion region to repair the broken areas in the contour edge of the projectile candidate target.

[0037] In this embodiment, an erosion operation is first performed on the foreground mask image to remove small-area pseudo-foreground noise introduced by factors such as background modeling errors or illumination changes. The erosion operation uses an elliptical structuring element (e.g., a cv2.MORPH_ELLIPSE structuring kernel), and the size of the erosion kernel or the number of erosions can be dynamically adjusted according to the number of contours or noise density in the image, thereby improving the adaptability of noise removal.

[0038] Subsequently, the system performs a dilation operation on the eroded foreground region to restore the edge information of the true foreground target and connect foreground fracture areas caused by occlusion or blurring during motion. The dilation operation also uses the same structuring element as the erosion stage, and further optimizes the edge repair effect with strategies such as median filtering and region connectivity analysis.

[0039] By combining the above-mentioned morphological operations of erosion and dilation, the overall coherence of the foreground region and the quality of the target contour can be significantly improved, thus laying a solid foundation for subsequent steps such as candidate target extraction, image patch cropping, and classification modeling.

[0040] As a further optional embodiment, after determining whether the target trajectory is a parabolic trajectory, the method further includes: If the trajectory classification result is a parabolic motion trajectory, then the event information of the target motion trajectory is output; Based on the event information, an alarm message is generated.

[0041] In this embodiment, when the trajectory classification result indicates that the target's motion trajectory is a parabolic trajectory, the system identifies the trajectory as a target trajectory that may involve a throwing event and outputs event information related to the target trajectory. This event information may include, but is not limited to, parameters such as: target ID, start frame number, end frame number, initial and end positions of the thrown object in the image coordinate system, trajectory length, and trajectory shape indicators (such as maximum height and curvature).

[0042] Furthermore, the system can generate corresponding alarm information based on the aforementioned event information. For example, it can push the event information to the upper-level platform in the form of structured data, trigger the alarm system, link the video screenshot or video recording archiving module, or output a prompt signal to the manual review terminal. The rules for generating this alarm information can be configured according to different scenarios (such as detention centers, campus security, border monitoring, etc.) to achieve an integrated closed loop of event detection and early warning linkage.

[0043] This implementation method enables the linkage between projectile trajectory recognition and intelligent alarm, improving the system's response capability to parabolic events and making it suitable for various intelligent video surveillance scenarios.

[0044] As a further optional embodiment, the step of synthesizing the image patches into a projectile candidate image corresponding to the projectile candidate target specifically includes: The image blocks in the image buffer corresponding to the positions of the projectile candidate targets are sorted in a temporal order. The multiple image blocks are stacked or stitched together in spatial order in time sequence to generate temporally consecutive candidate images of projectiles. The candidate images of projectiles are normalized in size to meet the input requirements of the projectile classification model.

[0045] Multiple image patches corresponding to the positions of the projectile candidate target in consecutive frames are extracted from the image buffer. Each image patch originates from the bounding box region of the candidate target in one frame. Subsequently, these image patches are temporally sorted according to the order in which the images were acquired.

[0046] Next, the above multiple image blocks are stacked or stitched together in spatial dimension according to time order. For example, the image blocks at three time points are combined into a three-channel image as three channels, so that the resulting projectile candidate image has good temporal continuity and spatial consistency, thereby better reflecting the dynamic appearance changes of the target between consecutive frames.

[0047] Finally, the synthesized candidate images of projectiles are normalized in size, for example, by adjusting them to a fixed size (such as 64×64 pixels or 128×128 pixels) to meet the input requirements of subsequent projectile classification models (such as ThrowObjectNet) and ensure the accuracy and stability of model inference.

[0048] By using the above processing methods, image information in the time dimension can be fully utilized to improve the recognition accuracy of projectile candidate targets and enhance the robustness of the system.

[0049] As a further optional embodiment, the step of constructing a matching region based on the historical motion trajectory of the candidate projectile target specifically includes: The historical trajectory of the projectile candidate target is fitted, and the predicted position and velocity information of the projectile candidate target in the current frame image are predicted using a Kalman filter, wherein the velocity information includes the direction of motion; Based on the prediction results, an elliptical matching region is generated in the current frame image with the predicted position as the center and the major axis direction consistent with the direction of motion. The elliptical matching region is used to limit the search range for cross-frame tracking.

[0050] In this embodiment, firstly, trajectory fitting processing is performed on the position information of the projectile candidate target in multiple historical frames prior to the current frame to obtain the target's motion trend. Preferably, the trajectory fitting employs polynomial fitting or Kalman filtering techniques to improve the accuracy of trajectory prediction.

[0051] Furthermore, using a Kalman filter based on the fitted historical trajectory information, the possible position (i.e., prediction center point) and velocity information of the projectile candidate target in the current frame are predicted. The velocity information includes the current velocity magnitude and motion direction angle.

[0052] Then, based on the above prediction results, an elliptical matching region is constructed in the current frame image with the predicted position as the center. The major axis of the ellipse is consistent with the predicted motion direction. The length of the major axis can be dynamically adjusted according to the speed or target size, and the minor axis is used to appropriately tolerate the drift of the target from the main motion direction.

[0053] This elliptical matching region serves as a candidate search area for the target in the current frame, limiting the range of subsequent cross-frame target matching. Specifically, the subsequent tracking process only searches for possible matching targets within this elliptical region, thereby reducing false matches and improving target tracking efficiency and accuracy.

[0054] Specifically, the Kalman filter is as follows: State vector definition: ,in The target center location, These are the x-axis velocity and the y-axis velocity, respectively. These are the x-axis acceleration and the y-axis acceleration, respectively. State transition equation: , It is a priori state estimation. It is the state transition matrix, where k represents the k-th frame.

[0055] For a 6-dimensional Kalman filter (assuming a time interval of...), )

[0056] Observation equation: ,in It is the observed target location. It is the observation matrix. It is observation noise. It refers to the state vector of the target (candidate target of the projectile) at time point k (i.e. the kth frame), which contains the target's position, velocity and acceleration information.

[0057] 6-dimensional observation matrix:

[0058] The Kalman filter prediction steps are as follows: 1. Prior state estimation: , Specifically: k-1 represents the previous time step. The superscript + (plus sign) indicates that this is a "posterior" estimate, which is the "optimal" or "corrected" estimate obtained at time k-1 after combining the measurements at that time. The ^ (hat symbol) above x indicates that this is an estimate, not the true value. A: is the state transition matrix, describing how the system state evolves from time k-1 to time k. The general meaning is that the predicted state at the current moment is obtained by applying the state transition matrix A to the final optimal state at the previous moment. 2. Prior error covariance: (in, It is the process noise covariance matrix. Specifically, this is the posterior error covariance at time k-1. It represents the optimal estimate at time k-1. The uncertainty is indicated by the superscript +, which represents the error covariance after correction for the measurement at time k-1.

[0059] The general meaning is that the uncertainty of the current moment's prediction stems from two aspects: firstly, the uncertainty of the state at the previous moment. Secondly, there are new uncertainties (Q) introduced by the system model itself during its evolution. Update steps: 1. Kalman gain: (in, (It is the observation noise covariance matrix) 2. Posterior state estimation:

[0060] 3. Posterior error covariance:

[0061] Data association (matching): Ellipse constraint: based on predicted location Predicting velocity direction and speed magnitude Define an elliptical region. A detection... The determination of whether something is inside an ellipse can be simplified to:

[0062] in It is the center of the ellipse. It is the ellipse rotation angle (related to the direction of velocity). It refers to the semi-major axis and the semi-minor axis (related to the speed and preset parameters).

[0063] The above method can effectively utilize historical trajectory information to predict and match the target's position in the current frame in a constrained manner, thereby enhancing the robustness of cross-frame tracking. It is particularly suitable for detecting small targets or projectiles with complex trajectories.

[0064] As a further optional embodiment, the step of performing cross-frame tracking of the candidate projectile target based on the matching region to obtain the target's motion trajectory specifically includes: Within the matching region, find the detected target in the current frame that best matches the candidate target; A cost matrix is ​​constructed based on the normalized Wasserstein distance between the candidate projectile targets and their historical trajectories; The Hungarian algorithm is used to associate detected targets in different frames to obtain the complete target motion trajectory.

[0065] First, in the current frame image, the search range for the target is limited based on the elliptical matching region generated in step 150. The system only searches for the target most likely corresponding to the candidate projectile within this matching region, thereby avoiding mismatches with other irrelevant moving targets and improving matching accuracy.

[0066] After the candidate matching targets are selected, a cost matrix is ​​constructed based on the normalized Wasserstein distance (NWD) between the historical trajectory data of each projectile candidate target and the candidate detected targets in the current frame. This normalized Wasserstein distance can comprehensively reflect the similarity between the predicted box and the detected box in multi-dimensional features such as center position and width and height, and is particularly suitable for similarity measurement between small targets or targets with deformation.

[0067] Subsequently, the Hungarian algorithm is used to solve the aforementioned cost matrix to obtain the optimal target matching relationship between the current frame and historical frames. This algorithm ensures that the global matching cost is minimized, thereby assigning the most suitable target for detection in the current frame to each tracker.

[0068] Finally, based on the matching results output by the Hungarian algorithm, the state of each tracker is updated, including motion state information such as position and velocity, and the continuous motion trajectory of the target is further generated.

[0069] Specifically, the cost matrix (Wasserstein Loss) is calculated as follows: For both predicted and detected bounding boxes (represented by their center point, width, and height), the calculation of NWD (the code implements a simplified form of exponential NWD similarity) can be summarized as follows: 1. Squared distance from the center point:

[0070] 2. Squared difference between width and height:

[0071] 3. The squared approximation of the Wasserstein distance:

[0072] 4. NWD Similarity: (in (This is a constant; specifically, it is 12.8) in this embodiment. 5. Cost (for minimization):

[0073] By combining the above methods with prediction region constraints and a highly robust trajectory similarity measurement mechanism, efficient, stable, and accurate cross-frame tracking of projectile targets can be achieved, providing continuous and complete temporal data support for subsequent trajectory recognition.

[0074] As an optional implementation, the data needs to be preprocessed before inputting into the trajectory classification model.

[0075] The trajectory feature extraction preprocessing is as follows: Width and height normalization: ,in and These are the mean and standard deviation of the width and height in the historical trajectory, respectively. It is a very small positive number (e.g., 1e-6), and its main function is to prevent the denominator from being zero. If all targets in the trajectory have the same width and height, the standard deviation σ will be 0, leading to a division-by-zero error. Adding ϵ ensures that the denominator is always a number greater than zero, thus increasing the stability of the calculation. This refers to the new width and height values ​​after normalization at time point t. This refers to the original width and height values ​​of the target at time point t.

[0076] Relative position:

[0077]

[0078] First-order difference (velocity approximation):

[0079]

[0080] Where x0 represents the x-coordinate of the starting point of the trajectory; Xt represents the x-coordinate at time t; Xt' represents the displacement of the target on the x-axis relative to its starting point; y0 represents the y-coordinate of the starting point of the trajectory; Yt represents the y-coordinate at time t; Yt' represents the displacement of the target on the y-axis relative to its starting point; Second-order difference (acceleration approximation):

[0081]

[0082] Final feature vector:

[0083] vxt: The approximate velocity of the object in the x-axis direction at time t.

[0084] xt′: The relative position of the object on the x-axis at time t (relative to the starting point).

[0085] xt−1′: The relative position of the object on the x-axis at time t-1 (i.e., the previous time).

[0086] vyt: The approximate velocity of the object in the y-axis direction at time t.

[0087] yt′: The relative position of the object on the y-axis at time t.

[0088] yt−1′: The relative position of the object on the y-axis at time t-1 (i.e., the previous time).

[0089] axt: The approximate acceleration of the object in the x-axis direction at time t.

[0090] vxt: The approximate velocity of the object in the x-axis direction at time t.

[0091] vx,t−1: The approximate velocity of the object in the x-axis direction at time t-1 (i.e., the previous time).

[0092] ayt: The approximate acceleration of the object in the y-axis direction at time t.

[0093] vyt: The approximate velocity of the object in the y-axis direction at time t.

[0094] vy,t−1: The approximate velocity of the object in the y-axis direction at time t-1 (i.e., the previous time).

[0095] The trajectory classification network is as follows: Positional Encoding:

[0096]

[0097] in, It is the position in the sequence. It is a dimensional index. It is the embedded dimension.

[0098] Multi-head attention mechanism:

[0099] in These are query, key, and value matrix. It is the dimension of the key vector.

[0100] The feedforward network is as follows:

[0101] After feature extraction and preprocessing, the trajectory needs to be preprocessed.

[0102] Specifically, Trajectory data: Trajectory data contains the object state of a target over a period of time, including x and y coordinates, width w, and height h.

[0103] Let the trajectory of a single target be ,in It is the unique ID of the target. The trajectory is... It consists of observation points arranged in chronological order.

[0104] In the Observation time points (of which) The state of the target can be represented as a vector. :

[0105] in: It is the x-coordinate of the center of the target (k) at its (j)th observation point.

[0106] It is the center y-coordinate of the target (k) at its (j)th observation point.

[0107] It is the width of the target (k) at its (j)th observation point.

[0108] It is the height of the target (k) at its (j)th observation point.

[0109] Then the complete historical trajectory data of this target (k) , can be represented as a matrix

[0110] This matrix Each row in the table represents the target's four primitive features (center x, center y, width, height) at a specific point in time.

[0111] Trajectory preprocessing: Width (w) and height (h) normalized: Assuming at time step In the original input features, the width is The height is For a segment containing The mean and standard deviation of the width and height of the trajectory sequence of points are calculated as follows: Mean:

[0112]

[0113] Standard deviation:

[0114]

[0115] Normalized width and height:

[0116]

[0117] Kinematic characteristic calculation Assuming the original trajectory has the first... The coordinates of the points are .

[0118] Displacement relative to the first point : make Let be the first point of the trajectory. Then, the... The points relative to The displacement is:

[0119] speed (First-order difference of displacement with respect to time) Represents the original coordinates:

[0120] It can also be written as:

[0121] acceleration : Acceleration is the first-order difference of velocity with respect to time (i.e., the second-order difference of displacement with respect to time).

[0122]

[0123]

[0124] Feature alignment and combination The preprocessing process ultimately constructs an 8-dimensional feature vector for each valid time step (starting from the third point of the original sequence, index 2). Let... Original coordinates For the first The width and height after normalization of each point.

[0125] For the first in the sequence Output feature points (corresponding to the original trajectory) Points, among which Its eigenvectors The definition is as follows: Relative coordinates (relative to) ):

[0126] Speed ​​(in) (place)

[0127] Acceleration (in) (place)

[0128] Normalized width and height (in) (place)

[0129] Therefore, eigenvectors for:

[0130] Sequence Construction After acquiring the target's raw time-series data (including position coordinates and optional morphological features) and calculating its kinematic derivatives (relative displacement, velocity, and acceleration), a composite feature sequence is constructed. Before construction, the length of the input trajectory is assessed. If the number of data points is less than a preset minimum threshold (e.g., 3 points), a valid composite feature vector cannot be generated. In this case, a zero-value tensor of a predetermined dimension is generated as the output to ensure data flow consistency. The composite feature vectors generated at all time steps are then processed. Arranged in chronological order, they form a composite feature sequence matrix. The dimension of this matrix is... ,in It is the number of time steps that can generate effective feature vectors. It is the dimension of the composite feature vector.

[0131] Sequence normalization To enable the model to handle input trajectories of arbitrary length, the composite feature sequence matrix needs to be... The timing length is standardized.

[0132] First, the matrix Timing length With a preset maximum sequence length Compare. If Then for the matrix Truncate the data, keeping only the data closest to the nearest node. The row vector corresponding to each time step. If Then in the matrix Append to the end (representing the end of the time series) Each dimension is The padding vector (e.g., the zero vector) is used until its timing length reaches [a certain value]. Finally, we obtain a dimension of Fixed-length characteristic matrix .

[0133] Finally, the trajectory classification model is as follows: 1. Input embedding layer: After preprocessing the trajectory data, a tensor with dimensions (batch, 8, 128) is obtained, where 8 is the dimension of the input features and 128 is the length of the input features.

[0134] The embedding layer "embeds" the trajectory data into a higher-dimensional feature space, preparing it for Transformer processing. Internally, it uses two one-dimensional convolutional layers. The first convolutional layer expands the feature dimension from 8 to 64. The second convolutional layer further expands it from 64 to 128. LayerNorm (layer normalization) and ReLU activation functions are applied between and after the convolutional layers to enhance the model's stability and non-linear expressiveness. After these layers, the output tensor has a shape of (batch_size, 128, 128).

[0135] Here we assume that the batch size is 1 and the max_length is 128, so the tensor dimension of the input model is (1, 8, 128).

[0136] First, this tensor goes through an embedding layer, which contains two 1D convolutional layers. Both convolutional layers have a kernel size of 3 and a padding size of 1. After the first layer, the dimensions become (1, 64, 128). After the second layer, the dimensions become (1, 128, 128). Finally, after passing through transpose and LayerNorm, the final dimensions are (1, 128, 128).

[0137] 2. Positional Encoding Because the Transformer model itself does not contain any information about the sequence order, a position encoding layer needs to be added to the model in order for the model to understand the temporal order of the trajectory points. Standard sine / cosine position encoding is used. This encoding method generates a unique, fixed vector for each position in the sequence. For each position in the sequence... and indexes in the embedding dimension Location coding The calculation is as follows:

[0138]

[0139] in This is the embedding dimension, which is 128 in this model. This positional encoding tensor will be directly added to the output of the embedding layer.

[0140] 3. CLS Token After adding positional encoding to the embedding output, a CLS token is appended to the beginning of the entire sequence. This CLS token interacts with tokens from all other time steps in the sequence during the Transformer Encoder process, continuously aggregating global information about the entire trajectory. Ultimately, the output of this CLS token can be used to classify whether the trajectory is a true projectile trajectory. 4. Transformer Encoder Module (TransformerBlock) It consists of multiple identical TransformerBlocks stacked together. Each TransformerBlock mainly consists of two sub-layers: multi-head self-attention and a feed-forward network.

[0141] Multi-Head Self-Attention (MHSA): Allows each token in the sequence (including the CLS token and the token at each time step) to pay attention to all other tokens in the sequence and compute a weighted representation based on their relevance.

[0142] The core of the attention mechanism is to compute the query. ), key ) and Value For the input sequence, , , They all originate from the same sequence (i.e., self-attention). The calculation formula is:

[0143] in This is the dimension of the key vector, used for scaling to prevent the gradient from becoming too small. Multi-head attention, on the other hand, divides this process into multiple "heads" that compute in parallel, and then concatenates the results, allowing the model to focus on information from different perspectives.

[0144] Feed-Forward Network (FFN): After the attention operation, a non-linear transformation is added to the representation of each token. It consists of two linear layers and a ReLU activation function.

[0145] Residual Connection & Layer Normalization: Each TransformerBlock's MHSA and FFN sub-layers adopt a Pre-LN structure, that is, layer normalization is performed before entering the sub-layer, and then the output of the sub-layer is added to the original input through residual connection.

[0146]

[0147]

[0148] The sublayer can be either MHSA or FFN. This structure helps stabilize the training of deep Transformers.

[0149] 5. Final classification head After the data flows through all TransformerBlocks, the following steps are performed for final classification: 1. Extract the CLS Token: From the output sequence (batch_size, max_length + 1, embed_dim), extract the representation of only the first token (CLSToken). Its shape is (batch_size, embed_dim). In this model, embed_dim is 128.

[0150] 2. Final processing: The representation of this CLS Token is passed through the last layer for normalization, a Dropout layer, and a linear layer in sequence.

[0151] 3. Output Logits: The linear layer maps the 128-dimensional (embedding dimension) vector to 2-dimensional (because there are two categories, projectiles and noise, resulting in raw scores for both categories). and The original score after SoftMax and It was converted into a probability value between 0 and 1. and ,and .

[0152]

[0153]

[0154] The following describes the parabolic event detection device provided by the present invention, such as... Figure 2As shown, the detection device for parabolic events described below and the detection method for parabolic events described above can be referred to in correspondence.

[0155] A device for detecting parabolic events, comprising: Image acquisition module 210 is used to acquire video stream frame images to be processed, perform grayscale processing on the current frame to obtain a grayscale image, and cache the grayscale image in an image buffer, wherein the image buffer is used to store the grayscale processed frame image; Background modeling module 220 is used to perform background modeling on the grayscale image and extract the foreground motion region; Image extraction module 230 is used to extract image blocks corresponding to each projectile candidate target in the foreground motion region from multiple frames of images in the image buffer, wherein one frame of image contains one image block corresponding to the projectile candidate target; The target classification module 240 is used to synthesize the image patches into a projectile candidate image corresponding to the projectile candidate target, and input it into a pre-trained projectile classification model to obtain the projectile confidence score. The region construction module 250 is used to construct a matching region based on the historical motion trajectory of the candidate projectile target when the confidence level of the projectile is higher than a preset threshold. The trajectory classification module 260 is used to perform cross-frame tracking of the projectile candidate target based on the matching region, obtain the target motion trajectory, and input it into a pre-trained trajectory classification model to determine whether the target motion trajectory is a parabolic trajectory.

[0156] Figure 3 An example of a physical structure diagram of an electronic device is shown below. Figure 3 As shown, the electronic device may include: a processor 310, a communications interface 320, a memory 330, and a communication bus 340, wherein the processor 310, the communications interface 320, and the memory 330 communicate with each other via the communication bus 340. The processor 310 can call logical instructions in the memory 330 to execute a method for detecting parabolic events, the method including: The process involves acquiring a video stream frame image to be processed, performing grayscale processing on the current frame to obtain a grayscale image, and caching the grayscale image into an image buffer, which is used to store the grayscale processed frame image. Background modeling is performed on the grayscale image to extract the foreground motion region; For each projectile candidate target within the foreground motion region, an image patch corresponding to the projectile candidate target is extracted from multiple frames in the image buffer, wherein one frame contains one image patch corresponding to the projectile candidate target; The image patches are combined into a projectile candidate image corresponding to the projectile candidate target, and then input into a pre-trained projectile classification model to obtain the projectile confidence score. When the confidence level of the projectile is higher than a preset threshold, a matching region is constructed based on the historical motion trajectory of the projectile candidate target. Based on the matching region, the candidate projectile target is tracked across frames to obtain the target's motion trajectory, which is then input into a pre-trained trajectory classification model to determine whether the target's motion trajectory is a parabolic trajectory.

[0157] Furthermore, the logical instructions in the aforementioned memory 330 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0158] On the other hand, the present invention also provides a computer program product, the computer program product comprising a computer program that can be stored on a non-transitory computer-readable storage medium, wherein when the computer program is executed by a processor, the computer is capable of executing the parabolic event detection method provided by the above methods, the method comprising: The process involves acquiring a video stream frame image to be processed, performing grayscale processing on the current frame to obtain a grayscale image, and caching the grayscale image into an image buffer, which is used to store the grayscale processed frame image. Background modeling is performed on the grayscale image to extract the foreground motion region; For each projectile candidate target within the foreground motion region, an image patch corresponding to the projectile candidate target is extracted from multiple frames in the image buffer, wherein one frame contains one image patch corresponding to the projectile candidate target; The image patches are combined into a projectile candidate image corresponding to the projectile candidate target, and then input into a pre-trained projectile classification model to obtain the projectile confidence score. When the confidence level of the projectile is higher than a preset threshold, a matching region is constructed based on the historical motion trajectory of the projectile candidate target. Based on the matching region, the candidate projectile target is tracked across frames to obtain the target's motion trajectory, which is then input into a pre-trained trajectory classification model to determine whether the target's motion trajectory is a parabolic trajectory.

[0159] In another aspect, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a method for detecting parabolic events provided by the methods described above, the method comprising: The process involves acquiring a video stream frame image to be processed, performing grayscale processing on the current frame to obtain a grayscale image, and caching the grayscale image into an image buffer, which is used to store the grayscale processed frame image. Background modeling is performed on the grayscale image to extract the foreground motion region; For each projectile candidate target within the foreground motion region, an image patch corresponding to the projectile candidate target is extracted from multiple frames in the image buffer, wherein one frame contains one image patch corresponding to the projectile candidate target; The image patches are combined into a projectile candidate image corresponding to the projectile candidate target, and then input into a pre-trained projectile classification model to obtain the projectile confidence score. When the confidence level of the projectile is higher than a preset threshold, a matching region is constructed based on the historical motion trajectory of the projectile candidate target. Based on the matching region, the candidate projectile target is tracked across frames to obtain the target's motion trajectory, which is then input into a pre-trained trajectory classification model to determine whether the target's motion trajectory is a parabolic trajectory.

[0160] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0161] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0162] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for detecting parabolic events, characterized in that, include: The process involves acquiring a video stream frame image to be processed, performing grayscale processing on the current frame image to obtain a grayscale image, and then caching the grayscale image into an image buffer, which is used to store the grayscale image after grayscale processing. Background modeling is performed on the grayscale image to extract the foreground motion region; In each frame of the image buffer, a corresponding image block is extracted based on the position of each projectile candidate target; The image patches are combined into a projectile candidate image corresponding to the projectile candidate target, and the projectile candidate image is input into a pre-trained projectile classification model to obtain the projectile confidence score, which is used to indicate the probability that the projectile candidate target is a projectile. When the confidence level of the projectile is higher than a preset threshold, a matching region is constructed based on the historical motion trajectory of the projectile candidate target. Based on the matching region, the candidate projectile target is tracked across frames to obtain the target's motion trajectory, which is then input into a pre-trained trajectory classification model to determine whether the target's motion trajectory is a parabolic trajectory.

2. The method for detecting parabolic events according to claim 1, characterized in that, The image buffer includes the current frame image and at least two frames preceding the current frame image, used to provide temporally continuous image data during image patch extraction.

3. The method for detecting parabolic events according to claim 1, characterized in that, After the step of performing background modeling on the grayscale image and extracting the foreground motion region, the method further includes: An erosion operation is performed on the foreground motion region to remove noise; An expansion operation is performed on the foreground motion region to repair the broken areas in the contour edge of the projectile candidate target.

4. The method for detecting parabolic events according to claim 1, characterized in that, After determining whether the target trajectory is a parabolic trajectory, the method further includes: If the trajectory classification result is a parabolic motion trajectory, then the event information of the target motion trajectory is output; Based on the event information, an alarm message is generated.

5. The method for detecting parabolic events according to claim 1, characterized in that, The step of synthesizing the image patches into a projectile candidate image corresponding to the projectile candidate target specifically includes: The image blocks in the image buffer corresponding to the positions of the projectile candidate targets are sorted in a temporal order. The multiple image blocks are stacked or stitched together in spatial order in time sequence to generate temporally consecutive candidate images of projectiles. The candidate images of projectiles are normalized in size to meet the input requirements of the projectile classification model.

6. The method for detecting parabolic events according to claim 1, characterized in that, The step of constructing a matching region based on the historical trajectory of the candidate projectile target specifically includes: The historical trajectory of the projectile candidate target is fitted, and the predicted position and velocity information of the projectile candidate target in the current frame image are predicted using a Kalman filter, wherein the velocity information includes the direction of motion; Based on the prediction results, an elliptical matching region is generated in the current frame image with the predicted position as the center and the major axis direction consistent with the direction of motion. The elliptical matching region is used to limit the search range for cross-frame tracking.

7. The method for detecting parabolic events according to claim 1, characterized in that, The step of performing cross-frame tracking of the candidate projectile target based on the matching region to obtain the target's motion trajectory specifically includes: Within the matching region, find the detection target in the current frame image that best matches the candidate target; A cost matrix is ​​constructed based on the normalized Wasserstein distance between the candidate projectile targets and their historical trajectories; The Hungarian algorithm is used to associate detected targets in different frames to obtain the complete target motion trajectory.

8. A device for detecting parabolic events, characterized in that, include: The image acquisition module is used to acquire the video stream frame image to be processed, perform grayscale processing on the current frame image to obtain a grayscale image, and cache the grayscale image in the image buffer, which is used to store the grayscale processed frame image; The background modeling module is used to model the background of the grayscale image and extract the foreground motion region; The image extraction module is used to extract corresponding image blocks from each frame of the image buffer based on the position of each projectile candidate target; The target classification module is used to synthesize the image patches into projectile candidate images corresponding to the projectile candidate targets, and input them into a pre-trained projectile classification model to obtain the projectile confidence score. The region construction module is used to construct a matching region based on the historical motion trajectory of the candidate projectile when the confidence level of the projectile is higher than a preset threshold. The trajectory classification module is used to perform cross-frame tracking of the projectile candidate target based on the matching region, obtain the target motion trajectory, and input it into a pre-trained trajectory classification model to determine whether the target motion trajectory is a parabolic trajectory.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the method for detecting parabolic events as described in any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method for detecting parabolic events as described in any one of claims 1 to 7.

Images