Intelligent behavior detection method and system based on motion feature enhancement

Abstract

The application discloses a kind of intelligent behavior detection method and system based on motion feature enhancement, construct based on adaptive mixture Gaussian model, the Mahalanobis distance of pixel point and background distribution is calculated and model weight is dynamically updated, realize the accurate decoupling and separation of foreground motion region in video frame;Then, the foreground mask obtained initially is based on the connected domain analysis of area threshold to filter out discrete noise, and the morphological closing operation of structure element adaptation is used to fill target internal cavity and repair edge fracture, to generate the high signal-to-noise ratio of motion region binary graph;Again, the purified binary graph is mapped as high visual saliency color heat map, it is weighted linearly fused with original RGB video frame, the fusion feature map is used to guide deep learning model to pay attention to motion region, and through double threshold detection strategy, the perception and identification ability of system to small target and hidden uncivilized behavior under complex dynamic scene is greatly improved.

Description

Technical Field

[0001] This invention belongs to the technical field of computer vision and digital image processing, and mainly relates to an intelligent behavior detection method and system based on motion feature enhancement. Background Technology

[0002] In the field of video surveillance and behavior analysis, accurate and robust extraction of moving targets is one of the core technologies. Currently, the mainstream methods in intelligent behavior detection fall into two main categories: one is deep learning-based target detection models, such as YOLO and SSD; the other is traditional computer vision-based motion detection algorithms, such as inter-frame difference, optical flow, and background subtraction.

[0003] While deep learning-based behavior detection methods can identify target semantic categories, their performance is heavily dependent on training data. They are prone to missed detections for targets with motion blur, drastic scale changes, or tiny targets that are rarely seen in the training data, preventing the system from issuing timely warnings. Traditional behavior detection methods, such as the Gaussian Mixture Model (GOG2), can effectively respond to motion changes, but the initial foreground mask generated directly often contains a lot of noise. This includes false detections caused by changes in lighting, holes due to similar target color to the background or slow motion, and fragmented target contours. As a result, the quality of the generated motion feature maps is poor, making it difficult to provide effective and clean motion information support for subsequent high-level analysis tasks, thus hindering the system from making accurate behavior detection.

[0004] Therefore, there is a lack of a dedicated method in the existing technology that can generate high-quality, robust motion feature maps to make up for the shortcomings of the above two types of technologies and provide more reliable input for subsequent behavior detection tasks. Summary of the Invention

[0005] This invention addresses the problems of high noise in the intermediate representations generated by existing technologies, such as high noise in the foreground mask, high speed of moving targets, and small target size, which easily lead to misjudgment and missed judgment. It provides an intelligent behavior detection method and system based on motion feature enhancement that can output high visual saliency, clear contours, and low noise. First, an adaptive Gaussian mixture model is constructed. By calculating the Mahalanobis distance between pixels and the background distribution and dynamically updating the model weights, accurate decoupling and separation of the foreground motion region in the video frame is achieved. Then, a mask purification strategy based on spatiotemporal geometric constraints is used to perform connected component analysis based on area thresholds on the initially acquired foreground mask to filter out discrete noise. Adaptive morphological closing operations of the structuring element are used to fill internal holes in the target and repair edge breaks, thereby generating a high signal-to-noise ratio binary map of the motion region. The purified binary map is then mapped to a color heatmap with high visual saliency, and weighted linearly fused with the original RGB video frame. The fused feature map guides the deep learning model to focus on the motion region, and a dual-threshold detection strategy significantly improves the system's ability to perceive and recognize small targets and hidden uncivilized behaviors in complex dynamic scenes. The method of this invention can be widely applied to smart community security, video surveillance, behavior recognition, target tracking and other fields. It has a wide range of applications and excellent performance.

[0006] To achieve the above objectives, the technical solution adopted by the present invention is: an intelligent behavior detection method based on motion feature enhancement, comprising the following steps:

[0007] S1, Data Preprocessing: Preprocess the input behavioral video data to convert the video into a video frame sequence, and obtain a group of N video frame segments with each frame as the anchor center.

[0008] S2, Video frame-level motion region detection: Motion regions are obtained by differentially analyzing adjacent selected frames, and the motion regions are detected by the MOG2 background subtraction algorithm;

[0009] S3, Basic Object Detection at the Video Frame Level: For the video sequence filtered in step S2, images are acquired in groups of N frames. Preliminary object detection is performed on the images using a deep learning object detection model to obtain a first detection result set. The deep learning object detection model is a single-stage object detection model based on a convolutional neural network, such as the YOLO series or SSD. The model includes a backbone network, a neck network, and a head network, used to extract image features and regress the location and category of the target. The first detection result set includes the bounding box coordinates and confidence scores.

[0010] S4, Motion Feature Enhancement and Generation: If the current frame number is an integer multiple of the preset processing interval N, the current frame image is converted into a grayscale image and processed to generate an initial foreground mask. The initial foreground mask is then post-processed to obtain a purified binary image of the motion region. The post-processing includes: thresholding, connected component analysis based on area constraints, and morphological closing operations. If the current frame number is not an integer multiple of the preset processing interval N, proceed directly to step S6.

[0011] S5, Feature Fusion Enhancement Detection: The original video frames in step S1 are fused with the color heatmap obtained in step S4, and the generated fused feature map is input into the target detection model again to obtain the second detection result set;

[0012] S6, Detection result fusion and optimization: merge the first detection result set obtained in step S3 and the second detection result set obtained in step S5, and process them through the non-maximum suppression algorithm to generate the final detection result set;

[0013] S7, Result Visualization and Judgment: Based on the final detection result set, the target behavior is judged according to logical rules; the corresponding discrimination rule library is loaded for different application scenarios. After the judgment is completed, the behavior category, confidence level and target border are drawn on the original video frame. If it is judged to be uncivilized behavior, an alarm is output or a log is recorded.

[0014] As an improvement of the present invention, step S4 specifically includes the following steps:

[0015] S41, determine whether the current frame number is an integer multiple of the preset processing interval N; if yes, proceed to step S42; otherwise, do not perform motion feature enhancement processing, directly proceed to step S6, reuse the motion feature map generated by the previous key frame or directly enter the next frame processing flow to save computing resources.

[0016] S42, convert the current frame image into a grayscale image; S43, process the grayscale image using a background subtractor to generate an initial foreground mask; S44, perform image post-processing on the initial foreground mask to obtain a purified motion region binary map; S45, convert the motion region binary map into a color heatmap as the motion feature map.

[0017] As an improvement of the present invention, step S44 specifically includes the following steps:

[0018] S441, Threshold the initial foreground mask to obtain a binary mask; S442, Calculate the area of ​​all connected components in the binary mask and remove connected components with an area smaller than the preset threshold Area_min to filter noise; S443, Perform a morphological closing operation on the filtered binary mask.

[0019] As another improvement of the present invention, step S5 specifically includes the following steps:

[0020] S51, normalize the original video frame image and the motion feature map;

[0021] S52, The normalized image is weighted and linearly superimposed according to the preset fusion weight W_fusion and motion weight W_motion to generate a fusion feature map; S53 inputs the fused feature map into the target detection model and performs detection using a confidence threshold lower than that of the basic detection step to obtain a second detection result set: since the motion region in the fused feature map has been enhanced, even weak target responses have high confidence. Therefore, using a lower Conf_low can effectively recall small moving targets that were missed under Conf_high.

[0022] To achieve the above objectives, the present invention also adopts the following technical solution: an intelligent behavior detection system based on motion feature enhancement, comprising a computer program, wherein the computer program, when executed by a processor, implements the steps of any of the methods described above.

[0023] Compared with the prior art, the present invention has the following beneficial effects:

[0024] (1) This invention significantly reduces noise in the original foreground mask by using the MOG2 background subtraction algorithm and a specific post-processing process. The generated motion feature map has a cleaner background, which effectively reduces interference in subsequent processing and is beneficial for detecting uncivilized behavior, especially small objects.

[0025] (2) The present invention effectively fills the holes inside the moving target and connects the broken edges through morphological closing operation, making the outline of the moving target more complete and clearer, improving the integrity of the moving area and reducing the possibility of false detection and missed detection.

[0026] (3) The color heat map output by the present invention utilizes the high sensitivity of human vision to color, especially warm colors, to greatly enhance the visual salience of the motion area, which is more conducive to manual observation and feature extraction of subsequent algorithms.

[0027] (4) As an upstream preprocessing technology, this invention has universality. Its output enhanced motion feature map can be combined with various downstream target detection and behavior recognition models to effectively improve the perception performance of these models in dynamic scenes and show a trend of improving the accuracy of the final task. Attached Figure Description

[0028] Figure 1 This is a flowchart of the steps of the method of the present invention;

[0029] Figure 2 This is a flowchart illustrating the motion feature enhancement in step S4 and the feature fusion in step S5 of the present invention. Detailed Implementation

[0030] The present invention will be further illustrated below with reference to the accompanying drawings and specific embodiments. It should be understood that the following specific embodiments are for illustrative purposes only and are not intended to limit the scope of the invention.

[0031] Example 1

[0032] Intelligent behavior detection methods based on motion feature enhancement, such as Figure 1 As shown, it includes the following steps:

[0033] Step S1, Data Preprocessing: Preprocess the input behavioral video data to convert the video into a video frame sequence, and obtain a group of N video frame segments with each frame as the anchor center.

[0034] Given a 1080p community surveillance video, decode it into an image sequence of 30 frames per second. Take each frame as the center and extract the five frames before and after it (i.e., [t-2, t-1, t, t+1, t+2]) as a processing segment.

[0035] Step S2, motion region detection at the video frame level: By performing differential and thresholding on adjacent selected video frames, a binary mask image is obtained, and motion regions are detected using the MOG2 background subtraction algorithm. The MOG2 background subtraction algorithm characterizes background changes by establishing a mixture model composed of K Gaussian distributions for each pixel, thus obtaining motion regions with eliminated background. Differentiation and thresholding aim to quickly capture pixel regions that change in the scene. The system continuously reads the video sequence in chronological order. Let the current time be t, and its corresponding video frame be image I_t, and the video frame at the previous time be image I{t-1}. First, images It and I{t-1} are converted from the RGB color image space to the grayscale image space to reduce computational complexity. Then, the absolute value difference operation is performed on the two grayscale images to obtain the difference image D_t, which is mathematically expressed as D_t = |I_t - I_{t-1}|. The pixels with higher brightness in the difference image D_t correspond to regions with significant changes between the two frames. Next, thresholding is applied to D_t, that is, an empirical grayscale threshold T1 is set, and pixels in D_t with grayscale values ​​greater than T1 are set to 255, i.e. white, representing the foreground / moving region, while pixels with grayscale values ​​less than or equal to T1 are set to 0, i.e. black, representing the background, thus obtaining a binary motion mask image M_frame_diff.

[0036] To further eliminate noise and small voids, morphological operations can be performed on M_frame_diff, such as first performing an opening operation (i.e., erosion followed by dilation) to remove noise points, and then performing a closing operation (i.e., dilation followed by erosion) to fill the small holes inside the moving area.

[0037] For example, in a surveillance scenario, if a pedestrian moves from frame t-1 to frame t, the pixel value at their location will change significantly. Through the inter-frame differencing and thresholding described above, a binary mask roughly outlining the pedestrian's silhouette can be obtained. However, this method may produce "holes," such as minimal variation within the same-color area of ​​the pedestrian's clothing, and it's difficult to detect complete moving objects. Therefore, we continue to use a combination of the MOG2 background subtraction algorithm to detect moving regions.

[0038] The MOG2 background subtraction algorithm is used to build and maintain an adaptive background model to more accurately separate moving objects in the foreground. The MOG2 algorithm characterizes the complex variations in the background by building a mixture model consisting of K Gaussian distributions for each pixel. The process is as follows:

[0039] Background model initialization: For each pixel in the first frame of the video, its pixel value, such as grayscale or RGB value, is used as the mean of the first Gaussian distribution, and is given a large variance and a small weight.

[0040] Model Update and Foreground Judgment: For each pixel X_t in each subsequent frame I_t, it is sorted in descending order of its weights and standard deviations among the K Gaussian distributions corresponding to the current pixel. X_t is then matched against each Gaussian distribution sequentially. This matching process essentially measures similarity by calculating the Mahalanobis distance between the current pixel's feature value X_t and the centers of each Gaussian distribution. Specifically, if the value of X_t falls within ±2.5 standard deviations of the mean of a distribution (i.e., the Mahalanobis distance is less than 2.5), a match is considered successful. Successfully matched Gaussian distributions are dynamically updated by increasing their weights and adjusting their mean and variance according to the learning rate, while the weights of unmatched distributions are decreased.

[0041] Subsequently, a preset background ratio threshold T is set. The weights of the Gaussian distributions arranged in descending order are accumulated sequentially. The first B Gaussian distributions whose accumulated weights are first greater than or equal to the threshold T are selected as the background model. If the Gaussian distribution that X_t successfully matches belongs to the first B Gaussian distributions, the pixel is determined to be the background; otherwise, it is determined to be the foreground. The background ratio T can be set to 0.7.

[0042] Foreground mask generation: After traversing the entire frame of the image, all pixels identified as foreground pixels constitute the foreground mask M_MOG2. The advantage of the MOG2 algorithm is that it can automatically adapt to slow changes in lighting, such as sunrise and sunset, and can also automatically adapt to slight background disturbances, such as swaying leaves. It can also ignore the shadows cast by objects by setting the "shadow detection" parameter, thereby obtaining a cleaner moving target area.

[0043] To combine the advantages of high sensitivity of the inter-frame difference method and complete contour of the MOG2 method, a logical AND operation is performed between the preliminary motion mask M_frame_diff obtained from the difference and thresholding steps and the precise motion mask M_MOG2 obtained from the MOG2 algorithm, i.e., M_final = M_frame_diff ∩ M_MOG2. This operation ensures that pixels ultimately identified as moving regions must be pixels detected as moving by both algorithms simultaneously. This effectively reduces false detections caused by noise, sudden changes in illumination, etc., and improves the robustness of the detection results. The final binary image M_final represents the moving region detected by this method.

[0044] Step S3, basic object detection at the video frame level: For the video sequence filtered in step S2, images are acquired in groups of N frames. The images are then subjected to preliminary object detection using a deep learning object detection model to obtain the first detection result set.

[0045] The deep learning-based object detection model employs a single-stage object detection architecture based on convolutional neural networks (CNNs) (such as the YOLOv8 series). This model mainly consists of three parts:

[0046] Backbone network: Responsible for extracting multi-scale feature maps of the input image, capturing semantic information such as texture and shape of the image;

[0047] Neck Network: Employs a Feature Pyramid Network (FPN) and a Path Aggregation Network (PAN) structure to fuse features at different scales, enhancing the model's ability to perceive targets of different sizes;

[0048] Detection Head: Based on the decoupled head structure, it predicts the target's class probability and bounding box coordinates, and finally outputs a detection result set containing the target's location, class, and confidence level.

[0049] The first detection result set includes the detection box coordinates (x, y, w, h) and confidence scores. In this embodiment, the convolutional neural network uses the YOLOv8 series, and the original frames are input into the YOLOv8s model with a confidence threshold of 0.25.

[0050] Step S4, motion feature enhancement and generation: If the current frame number is an integer multiple of the preset processing interval N, the current frame image is converted into a grayscale image and processed to generate an initial foreground mask. The initial foreground mask is then post-processed to obtain a purified binary image of the motion region. The post-processing includes: thresholding, connected component analysis based on area constraints, and morphological closing operation.

[0051] S41, determine if the current frame number is an integer multiple of the preset processing interval N; if yes, execute the following sub-step; otherwise, proceed to step S6. S42, convert the current frame image to a grayscale image, using the weighted average method recommended by the International Commission on Illumination (CIE) for the conversion: Where R, G, and B represent the pixel values ​​of the red, green, and blue channels in a color image, respectively. These are the pixel values ​​of the converted grayscale image. The coefficients 0.299, 0.587, and 0.114 reflect the physiological characteristic that the human eye is most sensitive to green, followed by red, and least sensitive to blue. S43, the grayscale image is processed using a background subtractor to generate an initial foreground mask; the background subtractor uses the MOG2 algorithm based on a Gaussian mixture model, with its historical frame count parameter set to 100 and its variance threshold parameter set to 16.

[0052] The history frame count is set to 100. This parameter controls the number of frames the background model "memorizes," directly affecting the model's speed of adapting to scene changes. A value of 100 means the model primarily references information from the most recent 100 frames. This ensures the model has sufficient adaptability to slow lighting changes and periodic disturbances, while also allowing the model to quickly "absorb" moving targets as background after they become stationary, effectively mitigating the "ghosting" problem. It achieves a good balance between model update speed and stability, making it suitable for most indoor and outdoor monitoring scenarios.

[0053] The variance threshold varThreshold is set to 16. This parameter determines the sensitivity in determining whether a pixel is in the foreground. The variance threshold is essentially the square of the Mahalanobis distance. A threshold of 16 corresponds to a difference of approximately 4 pixels in grayscale. This setting effectively filters out minute changes caused by camera noise and slight lighting jitter, while reliably detecting real object motion. It significantly reduces false detections caused by noise, and the generated initial foreground mask has a high signal-to-noise ratio, providing a better foundation for subsequent processing. S44, Post-process the initial foreground mask to obtain a cleaned binary image of the motion region;

[0054] First, perform thresholding on the initial foreground mask to obtain a binary mask: The initial foreground mask generated by the MOG2 algorithm may be a three - valued image, including background (0), foreground (255), and shadow (usually 127). To perform subsequent connected - component analysis and morphological operations, it must be converted into a standard binary image.

[0055] Use the fixed - threshold method for binarization: Set a determination threshold For each pixel value I in the initial foreground mask :

[0056]

[0057] Calculate the areas of all connected components in the binary mask, and remove the connected components with areas smaller than a preset threshold (Area_min) to filter out noise; the threshold can be taken as 800 pixels.

[0058] Real moving objects, such as pedestrians and vehicles, occupy relatively large connected areas in the image, while the connected areas generated by instantaneous noise, flying insects, or image - compression artifacts are usually very small. Based on this physical prior, filtering by area threshold is an efficient and reliable denoising method. Use the two - pass scan method or an algorithm based on contour tracking to perform connected - component analysis on the binary mask, identify and label all sets of connected foreground pixels. Then, for each labeled connected component, count the number of pixel points it contains, which is the area Ai of the connected component. Set an area threshold Amin. Traverse all connected components. If the area Ai of a connected component is less than Amin, determine that this area is noise and set all its pixel values to 0, that is, the background; if Ai≥Amin, keep it.

[0059] Perform a morphological closing operation on the filtered binary mask to fill the internal holes of the target and smooth the edges. The morphological closing operation uses a 5×5 - pixel rectangular structuring element. First, perform the dilation operation and then the erosion operation to fill the internal holes of the target and smooth the edges, while suppressing noise. The morphological closing operation is the combination of the dilation operation and the erosion operation, and its mathematical definition is , where A is the binary image and B is the structuring element. The order of dilation first and then erosion enables the dilation operation to fill the internal holes of the target smaller than the structuring element and bridge narrow breaks; the subsequent erosion operation can smooth the boundary of the target and basically restore the original size of the target, avoiding over - dilation.

[0060] In this embodiment, a 5×5 - pixel square rectangular structuring element is used. The structuring element of this size is sufficient to fill and connect the typical holes and breaks generated inside medium - and large - sized moving targets due to uniform texture, and at the same time, it will not over - smooth the details, resulting in the loss of small targets or unwanted merging between different targets.

[0061] S45, the binary motion map of the motion region is converted into a color heatmap as a motion feature map; the JET color mapping mode in the OpenCV library is used to convert the binary motion mask into a color heatmap, in which the motion region is represented by highly saliency colors such as red and yellow, and the non-motion region is represented by blue and black, finally obtaining a color motion heatmap aligned with the original frame.

[0062] Step S5, Feature fusion enhancement detection: such as Figure 2 As shown, the original video frames in step S1 are fused with the color heatmap obtained in step S4, and the generated fused feature map is input into the target detection model again to obtain the second detection result set.

[0063] First, the original video frame images and motion feature maps are normalized: the min-max normalization method is used to linearly map them to the interval [0, 1].

[0064] Find the minimum pixel value Mmin and the maximum pixel value Mmax in the motion feature map M, and transform each pixel in the map:

[0065]

[0066] If the motion feature map is a binary mask (0 / 255), then =0, =255, after normalization the result is a binary image (0 / 1) of [0, 1];

[0067] According to the preset fusion weight and motion weights The normalized images are weighted and linearly superimposed to generate a fused feature map: the generated fused feature map Ffused is calculated by the following formula:

[0068]

[0069] in: It is to assign normalized original frames Apparent weights It is to assign normalized motion characteristic map The motion weights satisfy + =1.0; the fusion weight W_fusion ranges from 0.5 to 0.9, and the motion weight W_motion ranges from 0.1 to 0.5.

[0070] Finally, the fused feature map is input into the target detection model, and detection is performed using a confidence threshold lower than that of the basic detection step to obtain a second detection result set.

[0071] Since the fused feature map has enhanced the region of interest with motion information, targets that were difficult to detect in the original image, such as small or blurry targets, will show a stronger response on this feature map. Therefore, this step aims to recall more true targets that were missed in the basic detection steps, especially those objects with weak features but which are indeed in motion, by lowering the detection confidence threshold.

[0072] In this embodiment, the original frame and the motion heatmap are fused with a weight of 0.7:0.3 to generate a fused feature map. The fused feature map is then input into the same YOLOv8s model, but the confidence threshold is reduced to 0.2, resulting in a second detection set. Lowering the confidence threshold can recall more targets that have become more prominent due to enhanced motion features.

[0073] Step S6, Detection result fusion and optimization: The first detection result set obtained in step S3 and the second detection result set obtained in step S5 are merged and processed by the non-maximum suppression algorithm to generate the final detection result set.

[0074] In this embodiment, a non-maximum suppression algorithm is applied, with a threshold value of 0.6 to remove redundant boxes and obtain merged boxes. Then, the YOLO model trained on the coco128 dataset and a self-made dataset can determine the type of object, thereby obtaining a preliminary behavior judgment. For example, in the detection of objects thrown from high altitudes, if the detected object is a balloon or a plastic bag, since its small mass and slow speed will not cause serious harm, no further steps are needed and it can be directly determined that it is not an object thrown from high altitudes.

[0075] Step S7, Result Visualization and Judgment: Based on the final detection result set, perform logical rule judgment on the target behavior.

[0076] For different application scenarios, corresponding discrimination rule libraries are loaded, such as rules for detecting objects thrown from heights and rules for detecting dogs on leashes. After the judgment is completed, the behavior category, confidence level, and target bounding box are drawn on the original video frame. If it is judged as uncivilized behavior, an alarm is output or a log is recorded. For the judgment of whether it is an object thrown from heights: if the movement trajectory is similar to a parabola and shows acceleration, it is determined to be an object thrown from heights, and the object is initially judged to be an object thrown from heights; if the trajectory speed is approximately uniform horizontal motion, it is not an object thrown from heights, and it is judged as normal behavior.

[0077] When judging whether a dog is being walked on a leash, if the leash, the person, and the dog are all present and their frames overlap, then the dog is considered to be on a leash, which is considered civilized behavior. If the leash, the person, and the dog are missing, or if there is a certain distance between them, then the person is not controlling the dog with a leash, which is considered walking the dog without a leash.

[0078] Based on the final detection results, the behavior type is determined and plotted on the original video frame for output.

[0079] Test case

[0080] To demonstrate the effectiveness and inventiveness of the method of this invention, a comparative experiment was conducted. This experiment aims to verify the improvement in the system's detection performance for small targets and uncivilized behaviors in complex backgrounds after introducing a motion feature enhancement mechanism, compared to the original target detection algorithm.

[0081] Experimental Environment and Configuration: The experiment was conducted on a unified hardware and software platform to ensure fairness in the comparison. The specific configuration is as follows:

[0082] Hardware environment: Processor is AMD Ryzen 9 7845HX with Radeon Graphics (3.00 GHz), graphics card is NVIDIA GeForce RTX 4060 Laptop GPU, and memory is 16GB (DDR5 4800MT / s).

[0083] Software environment: The operating system was Windows 11 Home Chinese Edition (64-bit), the deep learning framework was PyTorch 1.12.1, the parallel computing architecture was CUDA 11.8, and the programming language was Python 3.8. Experimental data was collected from real surveillance footage of the smart community and publicly available online datasets, covering scenarios including main roads, building entrances, and high-altitude surveillance views, and encompassing various lighting conditions such as daytime, nighttime, and rainy days.

[0084] Data composition: The dataset specifically selected 50 video clips containing three typical uncivilized behaviors: "throwing objects from high-rise buildings", "walking dogs without leashes", and "illegally bringing electric vehicles into households", as well as some background videos without abnormal behavior.

[0085] Dataset size: A total of 50 video clips containing uncivilized behavior were extracted. After frame extraction and cleaning, a total of 5250 experimental images were obtained, of which 3871 frames contained valid targets.

[0086] Dataset partitioning: The dataset is randomly divided into training, validation and test sets in a ratio of 7:2:1.

[0087] Definition of small target: Based on the MS COCO dataset definition and the requirements of actual scenarios, this experiment defines a target with a pixel area of ​​less than 32*32, i.e. less than 1024 pixels, but relaxes this to 2500 pixels in this experiment to adapt to monitoring distant scenes as a "small target".

[0088] Comparative Experiment Setup: To verify the effectiveness of this invention, the following two sets of comparative experiments were set up:

[0089] Control group (Baseline): The original YOLOv8s object detection model was used. This model only uses the original RGB images as input, without any motion feature enhancement, and the training parameters such as learning rate and batch size remain at their default settings.

[0090] Experimental group (Ours): Employed the intelligent behavior detection method based on motion feature enhancement described in this invention. This method, based on YOLOv8s, integrates motion heatmaps generated by MOG2 and morphological processing, and adopts a dual-threshold detection strategy. Except for the input features, the hyperparameters remain consistent with the control group.

[0091] Evaluation indicators: The experiment mainly selected the following indicators for quantitative evaluation:

[0092] Accuracy: The proportion of correctly predicted positive samples out of the total number of samples predicted as positive.

[0093] Recall: The proportion of correctly predicted positive samples out of the total number of actual positive samples. This metric is particularly sensitive to missed detections.

[0094] F1-Score: The harmonic mean of accuracy and recall, used to comprehensively evaluate model performance.

[0095] Based on the metrics, the above schemes were compared in experiments involving small target detection, i.e., objects smaller than 2500 pixels, and the results are shown in Table 1 below:

[0096] Table 1

[0097]

[0098] As shown in Table 1 above, after adopting the motion feature enhancement method provided by this invention, when processing the same 5250 frames of test data, although the total number of small targets is fixed at 1918, the detection count of the method of this invention increases from 553 in the control group to 773. Specifically, the recall rate of small targets significantly increases from 28.83% to 40.34%, an increase of more than 11 percentage points; the F1-Score also increases from 60.80% to 68.51%. This data difference directly proves that for small targets whose features are not obvious in the original static image and are easily buried by the background, this invention effectively "activates" these targets by enhancing motion features as attention guidance, making them easier for the detection model to capture, thereby significantly reducing the false negative rate and solving the technical problems proposed in the background art.

[0099] The above solutions were experimentally compared in real-world video scenarios. By comparing the behaviors detected by different methods, the comprehensive behaviors in complex real-world scenarios such as "throwing objects from high-rise buildings," "walking dogs without leashes," and "bringing electric vehicles upstairs" were classified, resulting in Table 2 below:

[0100] Table 2

[0101]

[0102] As shown in Table 2, both accuracy and recall are improved after adopting the motion feature enhancement method provided by this invention. Specifically, the number of false negatives decreased from 2520 with the pure YOLO method to 706, the recall increased from 45.73% to 63.25%, while the number of false positives decreased from 223 to 181, and the overall evaluation index F1-Score increased from 61.20% to 75.48%. This fully demonstrates that this invention, by enhancing motion features, has stronger adaptability in real-world scenarios.

[0103] More importantly, the value of this invention lies not only in improving accuracy and recall, but also in the qualitative leap it brings to discrimination capabilities. Unlike the pure YOLO model, which relies solely on static image semantics for judgment, this invention uses motion feature analysis to enable the model to prioritize "moving objects." This design significantly enhances the ability to identify suspected targets. For common non-threatening static or uniformly moving objects in community scenarios, such as balloons floating in the air or swaying trees, their motion patterns are fundamentally different from the motion trajectories and acceleration movements of risk events such as objects thrown from heights. This invention utilizes the inter-frame difference and background modeling techniques mentioned in step S2 to obtain motion trajectories, effectively distinguishing between vertically accelerating falling objects (parabolic objects) and horizontally moving objects (interference). For common interference objects in communities, such as swaying trees, whose motion patterns exhibit oscillating characteristics, the motion feature extraction algorithm described in this invention can effectively suppress them. This indicates that while improving the recall rate for real risks, this invention also inherently enhances robustness against specific interference objects. Its "false detections" are more concentrated in complex motion boundary cases that are difficult to distinguish, rather than misjudging simple interference objects.

[0104] In summary, the motion feature enhancement-based detection method described in this invention can effectively suppress background noise, highlight real moving targets, and significantly improve the accuracy and robustness of uncivilized behavior detection in smart community scenarios, especially the detection of small targets.

[0105] In the description of this specification, references to terms such as "an embodiment," "example," "specific example," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0106] It should be noted that the above content merely illustrates the technical concept of the present invention and should not be construed as limiting the scope of protection of the present invention. For those skilled in the art, various improvements and modifications can be made without departing from the principle of the present invention, and all such improvements and modifications fall within the scope of protection of the claims of the present invention.

Claims

1. An intelligent behavior detection method based on motion feature enhancement, characterized in that, Includes the following steps: S1, Data Preprocessing: Preprocess the input behavioral video data to convert the video into a video frame sequence, and obtain a group of N video frame segments with each frame as the anchor center. S2, Video frame-level motion region detection: By performing differential and thresholding on adjacent selected video frames, a binary mask image is obtained, and motion regions are detected by the MOG2 background subtraction algorithm; the MOG2 background subtraction algorithm establishes a mixture model composed of K Gaussian distributions for each pixel to characterize background changes and obtain motion regions with background removed. S3, Basic target detection at the video frame level: The video sequence detected in step S2 is processed into images in groups of N frames, which are then input into the target detection model to perform preliminary target detection, resulting in a first detection result set. The target detection model is based on a single-stage target detection architecture of a convolutional neural network, employing a feature pyramid network and a path aggregation network structure to fuse features at different scales and predict the target's class probability and bounding box coordinates. The first detection result set includes at least the target location and the class set confidence score. S4, Motion Feature Enhancement and Generation: If the current frame number is an integer multiple of the preset processing interval N, perform grayscale processing on the current frame image to generate an initial foreground mask. Perform image post-processing on the initial foreground mask to obtain a purified motion region binary map. The post-processing includes: thresholding, connected component analysis based on area constraints, and morphological closing operation. Convert the motion region binary map into a color heatmap to obtain a motion feature map. S5, Feature Fusion Enhancement Detection: After fusing the original video frames in step S1 with the motion feature maps obtained in step S4, the data is input into the target detection model again to obtain the second detection result set; S6, Detection result fusion and optimization: merge the first detection result set obtained in step S3 and the second detection result set obtained in step S5, and process them through the non-maximum suppression algorithm to generate the final detection result set; S7, Result Visualization and Judgment: Based on the final detection result set output in step S6, and using the discrimination rule base data, perform logical rule judgment on the target behavior to complete the behavior detection.

2. The intelligent behavior detection method based on motion feature enhancement as described in claim 1, characterized in that: In step S2, the method for differential and thresholding adjacent video frames is as follows: convert the adjacent video frame images to grayscale image space, and perform absolute value difference operation on the two grayscale images to obtain a difference image; set a grayscale threshold, set the pixels with grayscale values ​​greater than the grayscale threshold in the difference image to white to represent the foreground; set the pixels with grayscale values ​​less than the grayscale threshold in the difference image to black to represent the background, thereby obtaining a binarized motion mask image; The MOG2 background subtraction algorithm is as follows: For each pixel in the first frame of the video, its pixel value is used as the mean of the first Gaussian distribution, and a large variance and a small weight are assigned. For each pixel in each subsequent frame, the pixel is sorted in descending order with the K Gaussian distributions corresponding to the current pixel according to the ratio of weight to standard deviation. Each pixel is matched with each Gaussian distribution in turn. If the value of the pixel falls within the range of ±2.5 times the standard deviation of the mean of a Gaussian distribution, it is considered a successful match. The successfully matched Gaussian distributions are updated, and the weights of the unmatched Gaussian distributions are reduced. A background ratio threshold T is set, and the weights of the Gaussian distributions after descending order are accumulated sequentially. The first B Gaussian distributions whose accumulated weights are first greater than or equal to the threshold T are selected as the background model. If the Gaussian distribution that the pixel successfully matches belongs to the first B Gaussian distributions, the pixel is determined to be the background; otherwise, it is determined to be the foreground.

3. The intelligent behavior detection method based on motion feature enhancement as described in claim 1, characterized in that: In step S4, if the current frame number is not an integer multiple of the preset processing interval N, proceed to step S6 to fuse and optimize the first detection result set to obtain the final detection result set. If the current frame number is an integer multiple of the preset processing interval N, the grayscale image is processed using a background subtractor to generate an initial foreground mask; the background subtractor adopts the MOG2 algorithm based on the Gaussian mixture model, the historical frame number parameter is set to 100, and the variance threshold parameter is set to 16.

4. The intelligent behavior detection method based on motion feature enhancement as described in claim 3, characterized in that: In the post-processing of step S4, the thresholding process specifically involves setting a judgment threshold. For each pixel value I in the initial foreground mask The following judgments are made to distinguish between the foreground and background, resulting in a binary mask: ； in, These are the pixel coordinates; The area-constrained connected component analysis specifically involves: using a two-pass scanning method or a contour-tracking-based algorithm to perform connected component analysis on the binary mask, identifying and marking all interconnected foreground pixel sets; for each marked connected component, counting the number of pixels it contains, which is the area of ​​the connected component; setting an area threshold, traversing all connected components, if the area of ​​a connected component is less than the area threshold, then the region is determined to be noise, all its pixel values ​​are set to 0, and the region is considered background; if the area of ​​a connected component is greater than or equal to the area threshold, then it is considered foreground and is retained; The morphological closing operation specifically involves using a 5×5 pixel rectangular structural element, performing a dilation operation followed by an erosion operation to fill the internal holes of the target and smooth the edges.

5. The intelligent behavior detection method based on motion feature enhancement as described in claim 1, characterized in that: In step S5, the motion feature map is linearly mapped to [0, 1] using the min-max normalization method. The minimum pixel value Mmin and the maximum pixel value Mmax in the motion feature map M are found, and transformation normalization is performed on each pixel in the map. ； According to the preset fusion weight and motion weights The normalized images are weighted and linearly superimposed to generate a fused feature map, Ffused. ； in: It is to assign normalized original frames Apparent weights It is to assign normalized motion characteristic map The motion weights satisfy + =1.

0.

6. The intelligent behavior detection method based on motion feature enhancement as described in claim 5, characterized in that: When the target detection model performs secondary detection in step S5, it uses a confidence threshold lower than that used in the basic detection in step S3.

7. The intelligent behavior detection method based on motion feature enhancement as described in claim 1, characterized in that: In step S7, the detected target is tracked. If the target's trajectory is parabolic and its vertical velocity changes with acceleration, it is determined to be a high-altitude object throwing behavior. If the target's trajectory is approximately horizontal and uniform, it is determined to be a non-high-altitude object throwing behavior.

8. The intelligent behavior detection method based on motion feature enhancement as described in claim 7, characterized in that: The judgment in step S7 also includes performing correlation analysis on the categories in the detection result set; if the categories of people, dogs and leashes are detected at the same time, and the detection frames of the three overlap or are connected in space, it is judged as civilized dog walking behavior; if only people and dogs are detected, and the center distance between the two detection frames exceeds a preset threshold or no leash category is detected, it is judged as dog walking without a leash behavior.

9. An intelligent behavior detection system based on motion feature enhancement, comprising a computer program, characterized in that: Memory, used to store computer programs; A processor for executing the computer program to implement the steps of the method as described in any one of claims 1-8 above.

Images