Method for identifying abnormity of video monitoring data of waterlogging-prone point based on intelligent clustering algorithm

Abstract

The invention discloses a waterlogging-prone point video monitoring data abnormity identification method based on an intelligent clustering algorithm. The method comprises the following steps: S1, collecting video image data; s2, preprocessing each frame of image in the video image frame sequence; s3, based on the inter-frame distance matrix and the frame-level feature vector, outputting a final abnormal frame set by adopting an improved density peak clustering algorithm and combining a FixMatch semi-supervised classification model; s4, constructing a graph structure based on the fuzzy clustering boundary point set, inputting the graph neural network model, and outputting a final abnormal image frame set; s5, forming a complete abnormal frame sequence based on the combined abnormal judgment result set; and S6, when any image frame is judged to be an abnormal state frame, triggering an abnormal event judgment process. The method has the advantages of being high in anomaly recognition precision, high in response speed, high in anti-interference capacity, flexible in deployment and the like, and the automation and intelligence level of urban drainage scheduling can be remarkably improved.

Description

Technical Field

[0001] This invention relates to the fields of big data artificial intelligence and computer vision technology, and in particular to a method for identifying anomalies in video surveillance data of flood-prone areas based on intelligent clustering algorithms. Background Technology

[0002] In the current construction of refined urban governance and urban flood control systems, video surveillance-based methods for detecting water accumulation in flood-prone areas have been widely deployed. Traditional monitoring systems often rely on manual inspections or image processing algorithms based on simple thresholds for anomaly detection, such as identifying water areas based on image brightness changes, edge detection, or background modeling. While these methods are relatively inexpensive to implement, they are prone to misjudgments and missed detections when faced with complex environmental changes (such as rainy lighting, nighttime noise, and lens obstruction), failing to guarantee recognition accuracy and timeliness.

[0003] In recent years, some studies have attempted to apply clustering algorithms (such as K-means and DBSCAN) to anomaly identification in video image frame sequences, using inter-frame feature changes to cluster and analyze image states. However, these methods usually only build clustering models based on a single distance or density index, failing to fully consider the continuous evolution characteristics of image frames in the time dimension and the synergistic relationship between spatial features. This results in blurred cluster boundaries, unclear anomaly identification boundaries, and a lack of stable and reliable clustering judgment criteria.

[0004] In terms of further classification and screening of abnormal image frames, existing methods often rely on fully supervised training processes, requiring a large number of manually labeled samples, which increases costs and limits the generalization ability of the model in actual deployment. Regarding the utilization of clustering results, current methods mostly remain at the preliminary classification stage, lacking fine-grained modeling and re-judgment mechanisms for complex samples such as frames with blurred boundaries and frames showing trend changes, making it difficult to meet the practical monitoring needs for high robustness and high reliability.

[0005] Existing technologies for anomaly identification in image frames suffer from the following shortcomings: First, they fail to adequately utilize density and offset information, lacking effective structural modeling methods to characterize cluster boundaries and anomaly trends. Second, they fail to construct dynamic clustering mechanisms to address the complexity of inter-frame image state evolution, making it difficult to identify trending anomalies. Third, they utilize clustering results superficially, lacking mechanisms for pseudo-label construction, graph neural network learning, and multi-model fusion, thus failing to effectively improve the accuracy and intelligence level of anomaly identification. Therefore, there is an urgent need for an intelligent clustering algorithm that integrates density analysis, temporal modeling, and semi-supervised learning to construct a more robust anomaly detection mechanism and achieve efficient identification and reporting of anomaly image frames in video surveillance data of urban flood-prone areas. Summary of the Invention

[0006] One objective of this invention is to propose an anomaly identification method for video surveillance data of flood-prone areas based on an intelligent clustering algorithm. This invention integrates density estimation, inter-frame distance modeling, fuzzy clustering boundary recognition, trend anomaly analysis, and a semi-supervised learning mechanism to construct an anomaly detection process for video image frames in urban flooding scenarios. It details a multi-stage anomaly identification algorithm based on a density-offset bivariate function surface, density inertia map, FixMatch classification model, and graph neural network. This algorithm possesses advantages such as high anomaly detection accuracy, strong robustness, good model generalization ability, and the elimination of the need for extensive manual annotation.

[0007] The method for identifying anomalies in video surveillance data of flood-prone areas based on intelligent clustering algorithm according to an embodiment of the present invention includes the following steps:

[0008] S1. Collect video image data from video surveillance cameras deployed in flood-prone urban areas, and preprocess the video image data;

[0009] S2. Preprocess each frame of the video image frame sequence;

[0010] S3. By using the inter-frame distance matrix and frame-level feature vectors, introducing kernel density estimation and center offset distance calculation, constructing a density-offset bivariate function surface to extract saddle point candidate frames, forming a density split tree to identify fuzzy clustering boundaries, and constructing a density inertial map to capture trending abnormal states, an improved density peak clustering algorithm is formed, and combined with the FixMatch semi-supervised classification model to output the final abnormal frame set.

[0011] S4. Construct a graph structure based on the fuzzy clustering boundary point set and input it into the graph neural network model to output the final set of abnormal image frames;

[0012] S5. Based on the joint anomaly judgment result set, a complete anomaly frame sequence is constructed;

[0013] S6. When any image frame is determined to be an abnormal state frame, the abnormal event determination process is triggered.

[0014] Optionally, the preprocessing of the video image data includes frame extraction, image enhancement, and noise filtering to generate a standardized video image frame sequence;

[0015] Optionally, the preprocessing of each frame in the video image frame sequence includes extracting image features such as reflection brightness, edge gradient, grayscale mean, motion region boundary and pixel distribution variance, and constructing a set of frame-level feature vectors.

[0016] Optionally, S3 specifically includes:

[0017] S31. Construct an inter-frame distance matrix between each image frame within a sliding time window. The inter-frame distance matrix is ​​calculated based on the Euclidean distance between the frame-level feature vectors corresponding to each image frame. The frame-level feature vectors include reflectance brightness, edge gradient, grayscale mean, motion region boundary, and pixel distribution variance.

[0018] S32. The local density value of each frame image is calculated using the kernel density estimation method based on the inter-frame distance matrix. The local density values ​​of each image frame are arranged in chronological order to form a local density sequence. The local density value is obtained by counting the number of image frames whose distance from the current image frame is less than a set truncation distance. The truncation distance is quickly and adaptively adjusted by a parameter optimization model based on a meta-learning mechanism. In the local density sequence formed by arranging the local density values ​​of consecutive image frames in chronological order, the cumulative result of the density value increment within any time period is characterized as the density change trend.

[0019] S33. Determine the corresponding center offset distance based on the local density value of each image frame. The center offset distance is the minimum distance between the current image frame and all image frames with local density values ​​higher than the current image frame. When the current image frame is the image frame with the largest local density value within the sliding time window, the center offset distance is defined as the maximum distance between the current image frame and all other image frames.

[0020] S34. Construct a density-offset bivariate function surface by combining the local density value and the center offset distance. Identify regions where the function value changes erratically based on the density-offset bivariate function surface. Extract local minima from these regions as saddle point candidate frames to form an anomaly candidate point set.

[0021] S35. Using the frame with the highest local density as the cluster center, construct a density split tree structure along the density decreasing path to identify boundary nodes with weak density change continuity and abnormal inter-frame distance distribution, thus forming a fuzzy clustering boundary point set.

[0022] S36. Construct a density inertial map structure based on the local density sequence and cluster center position sequence of continuous image frames, analyze the density change trend of image frames on the time axis and the spatial drift trajectory of cluster centers. When the frame density change continues to rise or the cluster center position shifts continuously, determine that the current frame is in a trend abnormal state and mark it as a trend abnormal point set. The spatial drift trajectory of cluster centers consists of the two-dimensional coordinate points of cluster centers in the image space within continuous time steps, forming a cluster center position sequence. The cluster center position sequence is obtained by calculating the change of Euclidean distance between cluster centers of adjacent frames.

[0023] S37. Construct a pseudo-label dataset for the anomaly candidate point set, the fuzzy clustering boundary point set, and the trend anomaly point set. Use image frames from the anomaly candidate point set and the fuzzy clustering boundary point set as positive pseudo-label samples. Select image frames from the trend anomaly point set whose local density value is lower than the median local density of all image frames and whose corresponding center offset distance is higher than the 75th percentile of the center offset distance of all image frames as negative pseudo-label samples. Input the constructed pseudo-label dataset into the FixMatch semi-supervised classification model for iterative training and output the anomaly confidence score for each image frame. Reorder the anomaly candidate frames in descending order according to the confidence score, remove candidate frames whose confidence score is lower than the set threshold, and retain the final set of anomaly frames for outputting the water accumulation event results.

[0024] Optionally, the density decreasing path points from each non-center image frame to an image frame with a higher local density value and the smallest Euclidean distance from that image frame, forming a directed connection until the local density maximum frame is reached. Using the local density maximum frame as the cluster center frame, a density split tree structure is constructed along the density decreasing path; in the density split tree structure, if the following two conditions are satisfied:

[0025] Judgment condition 1: The local density difference between the current image frame and its corresponding density parent node is less than the preset density difference threshold;

[0026] Judgment condition 2: The Euclidean distance between the current image frame and its corresponding density parent node is greater than the set multiplier threshold of the average Euclidean distance between all image frames on the density decreasing path.

[0027] The current image frame is then identified as a boundary node with weak density change continuity and abnormal inter-frame distance distribution; all image frames that simultaneously satisfy condition 1 and condition 2 constitute a fuzzy clustering boundary point set.

[0028] Optionally, S4 specifically includes:

[0029] S41. For the fuzzy clustering boundary point set, construct a graph structure. Each node in the graph structure corresponds to a boundary image frame. The edges in the graph structure represent the feature similarity relationship between any two boundary image frames. The weight of the edges is calculated by the cosine similarity between the corresponding frame-level feature vectors of the image frames.

[0030] S42. Input the graph structure into the graph neural network model. The graph neural network model is constructed using a multi-layer graph convolutional structure. Each layer performs adjacent node feature aggregation and non-linear mapping operations, and updates the node embedding representation layer by layer.

[0031] S43. During the training phase of the graph neural network, the set of frames with the maximum local density obtained by the improved density peak clustering algorithm is used as pseudo-label samples. The node labels in the graph neural network are assigned as anomaly categories to guide the graph embedding feature propagation process. The graph neural network optimizes its parameters by minimizing the prediction error of the pseudo-label nodes.

[0032] S44. After the graph neural network is trained, an anomaly probability score is output for each image frame node in the graph structure to form a corresponding anomaly probability vector. Based on whether the anomaly probability score of each image frame node is higher than a set threshold, it is determined whether the current image frame node is judged as the final abnormal frame. All image frame nodes with anomaly probability scores higher than the set threshold form the final abnormal image frame set.

[0033] Optionally, S5 specifically includes:

[0034] S51. Merge the final set of abnormal frames, the set of candidate points of abnormality extracted based on the density-offset bivariate function surface in step S34, and the set of trend abnormality points marked in step S36 to construct a global set of candidate abnormal frames.

[0035] S52. Extract the frame number and corresponding density value of each image frame in the local density sequence from the anomaly candidate point set as a density anomaly reference basis, set a density recognition threshold, and filter frames with local density values ​​lower than the density recognition threshold to form a saddle point anomaly reference set.

[0036] S53. Based on each image frame in the trend anomaly point set, extract the spatial drift path within the corresponding time period from the cluster center position sequence and calculate the continuous drift distance; set a spatial drift distance threshold and select image frames whose continuous drift distance exceeds the spatial drift distance threshold to form a drift anomaly reference set.

[0037] S54. Perform a set merging operation based on the global candidate abnormal frame set, the saddle point abnormal reference set, and the drift abnormal reference set to form a joint abnormal judgment result set. Sort the joint abnormal judgment result set according to the image frame time order and output the complete abnormal frame sequence.

[0038] Optionally, S6 specifically includes:

[0039] S61. Perform frame-by-frame traversal detection on the complete abnormal frame sequence output in claim 6 to determine whether any image frame is marked as an abnormal frame; if there is an image frame marked as abnormal, trigger the abnormal event determination process.

[0040] S62. The abnormal event determination process specifically includes: extracting the timestamp and frame index number of the abnormal image frame in the video image frame sequence to form a set of image abnormal event tuples for the image abnormal event.

[0041] For each image anomaly event tuple, a corresponding water accumulation anomaly event tag is generated. The water accumulation anomaly event tag contains a unique event identifier, anomaly type, timestamp, and image frame index, and is encapsulated into a standardized anomaly event information structure according to a set format.

[0042] The standardized abnormal event information structure is sent to the flood control and dispatching platform through the communication interface;

[0043] Record the sending status feedback of abnormal event information. If the feedback is successful, the sending log is written to the event log file. If the sending fails, the retry mechanism is executed: try to resend within the set number of retries. If the sending still fails, the event information is cached in the local abnormal buffer pool and automatically resent in batches after communication is restored.

[0044] Optionally, the improved density peak clustering algorithm specifically includes the following steps:

[0045] The Euclidean distance between image frames is calculated based on the inter-frame distance matrix and frame-level feature vectors within the sliding time window, and the inter-frame distance matrix is ​​constructed.

[0046] The local density value of each image frame is calculated using the kernel density estimation method to generate a local density sequence. The center offset distance is determined based on the local density value and density ranking. A density-offset bivariate function surface is constructed, and local minima in the function surface are extracted as saddle point candidate frames to form an anomaly candidate point set.

[0047] Using the frame with the highest local density as the cluster center, a density split tree is constructed along the density decreasing path, and nodes with weak density change continuity or abnormal inter-frame distance distribution are identified as the fuzzy clustering boundary point set.

[0048] A density inertial map structure is constructed by combining the local density sequence of consecutive image frames with the cluster center location sequence.

[0049] Analyze the density change trend of image frames and the spatial drift trajectory of cluster centers to identify image frames with continuously increasing density or continuously shifting cluster centers, thus forming a set of trend-based outliers.

[0050] The pseudo-label dataset is constructed by combining the set of anomaly candidate points, the set of fuzzy clustering boundary points, and the set of trend anomalies. It is then input into the FixMatch semi-supervised classification model to train and output the anomaly confidence score for each frame. Based on the score results, the anomaly candidate frames are reordered, frames with confidence scores below a set threshold are removed, and the final set of anomaly frames is retained.

[0051] The beneficial effects of this invention are:

[0052] This invention constructs an improved density peak clustering algorithm, combining graph neural networks and a semi-supervised learning model, to achieve efficient identification of abnormal frames in video surveillance images of urban flood-prone areas. It achieves beneficial technical effects through multi-stage, multi-scale, and multi-strategy fusion. By introducing a local density value calculation method based on kernel density estimation and a center offset distance measurement method, a density-offset bivariate function surface is established, effectively characterizing the distribution pattern of image frames in the feature space. This accurately identifies regions with discontinuous function value changes and extracts saddle point candidate frames. A density splitting tree is constructed to identify fuzzy cluster boundaries, and density inertia maps are combined to analyze the density change trend of image frames and the continuous drift trajectory of cluster center positions, thereby enabling the identification of trending abnormal frames and improving response capabilities in scenarios with drastic or slowly evolving abnormal changes.

[0053] This invention introduces the FixMatch semi-supervised classification mechanism to construct a pseudo-label sample set from abnormal candidate frames, fuzzy boundary frames, and trend abnormal frames, guiding iterative training of the model and further enhancing the robustness and accuracy of anomaly detection results.

[0054] This invention employs a graph neural network to learn the graph structure constructed from a set of fuzzy clustering boundary points, and utilizes a graph embedding propagation mechanism to strengthen the interrelationships between features, thereby achieving anomaly probability scoring for frames with similar complex features. By combining multi-source anomaly judgment results and constructing a complete anomaly frame sequence, accurate detection and real-time reporting of waterlogging anomaly events are achieved, significantly improving the system's intelligence level in urban flooding early warning and response. Attached Figure Description

[0055] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0056] Figure 1 This is an overall flowchart of the method for identifying anomalies in video surveillance data of flood-prone areas based on intelligent clustering algorithm proposed in this invention;

[0057] Figure 2 This is a schematic diagram of the improved density peak clustering algorithm structure of the anomaly identification method for video surveillance data of flood-prone areas based on intelligent clustering algorithm proposed in this invention.

[0058] Figure 3 This is a flowchart illustrating the abnormal event determination process of the anomaly identification method for video surveillance data of flood-prone areas based on intelligent clustering algorithm proposed in this invention. Detailed Implementation

[0059] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0060] refer to Figure 1-3 A method for identifying anomalies in video surveillance data of flood-prone areas based on intelligent clustering algorithms includes the following steps:

[0061] S1. Collect video image data from video surveillance cameras deployed in flood-prone urban areas, and preprocess the video image data;

[0062] S2. Preprocess each frame of the video image frame sequence;

[0063] S3. By using the inter-frame distance matrix and frame-level feature vectors, introducing kernel density estimation and center offset distance calculation, constructing a density-offset bivariate function surface to extract saddle point candidate frames, forming a density split tree to identify fuzzy clustering boundaries, and constructing a density inertial map to capture trending abnormal states, an improved density peak clustering algorithm is formed, and combined with the FixMatch semi-supervised classification model to output the final abnormal frame set.

[0064] S4. Construct a graph structure based on the fuzzy clustering boundary point set and input it into the graph neural network model to output the final set of abnormal image frames;

[0065] S5. Based on the joint anomaly judgment result set, a complete anomaly frame sequence is constructed;

[0066] S6. When any image frame is determined to be an abnormal state frame, the abnormal event determination process is triggered.

[0067] This invention provides a method for anomaly identification of video surveillance data at flood-prone areas based on an intelligent clustering algorithm. The method involves preprocessing video surveillance image data deployed at flood-prone urban areas, including frame extraction, image enhancement, and noise filtering, to generate a standardized image frame sequence. Further, image frame-level feature vectors are extracted, an inter-frame distance matrix is ​​constructed, and a density-offset bivariate function surface is built using kernel density estimation and center offset distance to extract saddle point candidate frames. This forms a density splitting tree to identify fuzzy clustering boundaries. Simultaneously, a density inertial map is constructed to capture trending anomalies, and an improved density peak clustering algorithm is proposed. A set of credible anomaly frames is selected using a FixMatch semi-supervised classification model. Based on this, a graph structure is constructed based on the fuzzy clustering boundaries, and the graph neural network model further outputs a set of anomaly image frames. Finally, the results of various anomaly identifications are fused to form a complete anomaly frame sequence. If any frame is detected as an anomaly, an anomaly event judgment process is automatically triggered, achieving efficient and accurate image anomaly identification at flood-prone urban areas.

[0068] In this embodiment, the preprocessing of video image data includes frame extraction, image enhancement, and noise filtering to generate a standardized video image frame sequence.

[0069] In this embodiment, the preprocessing of each frame in the video image frame sequence includes extracting image features such as reflection brightness, edge gradient, grayscale mean, motion region boundary and pixel distribution variance, and constructing a set of frame-level feature vectors.

[0070] In this embodiment, S3 specifically includes:

[0071] S31. Construct an inter-frame distance matrix between each image frame within a sliding time window. The inter-frame distance matrix is ​​calculated based on the Euclidean distance between the frame-level feature vectors corresponding to each image frame. The frame-level feature vectors include reflectance brightness, edge gradient, grayscale mean, motion region boundary, and pixel distribution variance.

[0072] S32. The local density value of each frame image is calculated using the kernel density estimation method based on the inter-frame distance matrix. The local density values ​​of each image frame are arranged in chronological order to form a local density sequence. The local density value is obtained by counting the number of image frames whose distance from the current image frame is less than a set truncation distance. The truncation distance is quickly and adaptively adjusted by a parameter optimization model based on a meta-learning mechanism.

[0073] The meta-learning mechanism constructs a task meta set based on video image clustering tasks from multiple historical monitoring points, uses the Reptile meta-learning algorithm to train the initial parameters of the improved density peak clustering model, and performs a small number of clustering iterations before deployment at each monitoring point to achieve rapid parameter tuning.

[0074] S33. Determine the corresponding center offset distance based on the local density value of each image frame. The center offset distance is the minimum distance between the current image frame and all image frames with local density values ​​higher than the current image frame. When the current image frame is the image frame with the largest local density value within the sliding time window, the center offset distance is defined as the maximum distance between the current image frame and all other image frames.

[0075] S34. Construct a density-offset bivariate function surface by combining the local density values ​​and the center offset distance. The bivariate function is defined as follows:

[0076] ;

[0077] in, Indicates the first Local density values ​​of a frame image. Indicates the first Center offset distance of the frame image For the The function value of the frame image on the density-offset function surface.

[0078] Identifying unstable regions of function values ​​based on density-migrating bivariate function surfaces, through... The second derivative (i.e., the Laplace operator) is calculated to find the local minimum and the position of the surrounding function value change significantly as the saddle point candidate frame. These points are regarded as potential anomalous frames and are used to form an anomalous candidate point set.

[0079] S35. Using the frame with the highest local density as the cluster center, construct a density split tree structure along the density decreasing path to identify boundary nodes with weak density change continuity and abnormal inter-frame distance distribution, thus forming a fuzzy clustering boundary point set.

[0080] The density-decreasing path refers to establishing directed connections from each non-center frame to image frames with higher local density values ​​until reaching the frame with the highest local density, forming a unidirectional density flow path.

[0081] ;

[0082] in: Represents an image frame The local density of Represents an image frame With image frame The Euclidean distance between them; For image frames The density of the parent node, i.e., the density is higher than The closest one among all frames;

[0083] By all The mappings form a directed tree with the frame of maximum local density as the root node, called a density-split tree. In this structure, if a certain node... Satisfy its and density difference Drastic changes, or the distance between its frames If the distance is significantly greater than the average distance of its preceding frames, the node is marked as a fuzzy cluster boundary point.

[0084] S36. Construct a density inertial map structure based on the local density sequence and cluster center position sequence of continuous image frames, analyze the density change trend of image frames on the time axis and the spatial drift trajectory of cluster centers, and determine that the current frame is in a trend abnormal state when the frame density change continues to rise or the cluster center position shifts continuously, and mark it as a trend abnormal point set.

[0085] S37. Construct a pseudo-label dataset for the anomaly candidate point set, the fuzzy clustering boundary point set, and the trend anomaly point set. Use the image frames in the anomaly candidate point set and the fuzzy clustering boundary point set as pseudo-label positive samples, and use the trend anomaly point set with low density value and large offset distance as pseudo-label negative samples. Iteratively train on the FixMatch semi-supervised classification model, output the anomaly confidence score result for each frame, re-rank the anomaly candidate frames according to the confidence score, remove candidate frames with confidence scores lower than the set threshold, and retain the final anomaly frame set for outputting the water accumulation event results.

[0086] This invention proposes an improved density peak clustering algorithm for anomaly detection in video images of urban flood-prone areas. Within a sliding time window, it constructs an inter-frame distance matrix between image frames and extracts frame-level feature vectors, including reflectance brightness and edge gradients. Local density values ​​for each frame are calculated using kernel density estimation, forming a time-series density distribution. To improve the adaptability of density estimation, this invention introduces a Reptile-based meta-learning mechanism, using historical clustering tasks to train the initial parameters of the clustering model and performing rapid parameter tuning before deployment. Subsequently, the center offset distance of image frames is calculated and combined with local density values ​​to construct a density-offset bivariate function surface. By calculating its second derivative, saddle point candidate frames are identified, constructing an anomaly candidate point set. Using the frame with the highest local density as the root node, a density split tree is constructed along a density decreasing path, identifying density breakpoints and distance anomalies as a fuzzy clustering boundary point set. Based on this, a density inertial map is constructed to capture the temporal density change trend and the spatial drift trajectory of the cluster centers, identifying trending anomaly frames. Finally, a pseudo-label dataset is constructed by fusing the three types of outliers, and a FixMatch semi-supervised classification model is trained. The confidence level of the candidate frames is then used for screening and sorting, and a final set of high-confidence outlier frames is output.

[0087] In this embodiment, the construction of the density inertial map structure includes:

[0088] Step 1: Construct local density time series:

[0089] The local density values ​​of each image frame within the sliding time window are arranged sequentially according to the time order of the frames to form a local density time series; where the local density value of each frame represents the statistical degree of feature similarity between that frame and its adjacent frames.

[0090] Step 2: Construct the time series of cluster center locations:

[0091] The coordinates of the cluster centers in the image space corresponding to each moment within the sliding time window are arranged in chronological order to form a time series of cluster center locations; where the coordinates of the cluster centers at each moment represent the center position of the density distribution of all image frames at that moment.

[0092] Step 3: Define the rate of density change:

[0093] The density change rate represents the difference in local density value between any image frame and its previous image frame, reflecting the dynamic trend of image content in density distribution.

[0094] Step 4: Define the cluster center drift distance:

[0095] Cluster center drift distance represents the Euclidean distance between cluster centers in the image space between two consecutive time steps. It is used to measure the spatial movement of cluster centers over time and reflects the stability or shift trend of the cluster structure.

[0096] Based on the constructed local density time series and cluster center location time series, a fixed-length sliding time window is set. The image frame corresponding to the current time step is determined to be a trend outlier when any of the following conditions are met:

[0097] Condition 1: Within the current sliding time window, the local density value of each image frame increases continuously, and the total cumulative density change exceeds the preset density change threshold.

[0098] Condition 2: Within the current sliding time window, the spatial position of the cluster center shifts continuously at each moment, and the total cumulative drift distance exceeds the preset spatial drift threshold.

[0099] This invention proposes a method for constructing a density inertial map structure during anomaly image recognition. By introducing local density time series and cluster center position time series, it characterizes the density evolution features of image frames in the time dimension and the spatial drift trend of cluster centers. Specifically, it includes: arranging the local density values ​​of each image frame in chronological order to form a local density time series to characterize the changes in inter-frame feature similarity; simultaneously, extracting the spatial center of the density distribution of image frames at each time moment to construct a cluster center position time series to track the dynamic changes of the cluster structure in image space. Based on this, the density change rate and cluster center drift distance are defined as dynamic feature metrics, and the continuous growth trend of the density distribution of image frames and the continuous shift of the cluster center position are analyzed based on a sliding time window. When the cumulative magnitude of local density change exceeds a preset threshold, or the continuous drift distance of the cluster center exceeds a spatial drift threshold, the image frame corresponding to that time point is determined to be a trend anomaly point, thereby effectively capturing the unstable behavior of the cluster structure caused by water accumulation changes.

[0100] In this embodiment, S4 specifically includes:

[0101] S41. For the fuzzy clustering boundary point set, construct a graph structure. Each node in the graph structure corresponds to a boundary image frame. The edges in the graph structure represent the feature similarity relationship between any two boundary image frames. The weight of the edges is calculated by the cosine similarity between the corresponding frame-level feature vectors of the image frames.

[0102] S42. Input the graph structure into the graph neural network model. The graph neural network model is constructed using a multi-layer graph convolutional structure. Each layer performs adjacent node feature aggregation and non-linear mapping operations, and updates the node embedding representation layer by layer.

[0103] S43. During the training phase of the graph neural network, the set of frames with the highest local density obtained by the improved density peak clustering algorithm is used as pseudo-label samples. The corresponding node labels in the graph neural network are assigned anomaly categories to guide the graph embedding feature propagation process. The graph neural network optimizes its parameters by minimizing the prediction error of the pseudo-label nodes. Here, the graph structure is assumed to be... Each node Represents an image frame, with the node's initial features being: , No. The graph convolution operation for a layer is as follows:

[0104] ;

[0105] in, For nodes The set of adjacent nodes, The normalization coefficient is... For the The trainable weight matrix of the layer, Let be a non-linear activation function. Let the set of pseudo-label nodes be . Each node The pseudo-tags are Then the training loss function is:

[0106] ;

[0107] in, The nodes output by the graph neural network The predicted value of the anomaly probability. This represents the pseudo-labeling results. The model minimizes... Implement supervised updates of node representations and classification probabilities.

[0108] S44. After the graph neural network is trained, an anomaly probability score is output for each image frame node in the graph structure to form a corresponding anomaly probability vector. Based on whether the anomaly probability score of each image frame node is higher than a set threshold, it is determined whether the current image frame node is judged as the final abnormal frame. All image frame nodes with anomaly probability scores higher than the set threshold form the final abnormal image frame set.

[0109] This invention introduces a graph neural network-based discrimination mechanism in the process of anomaly image frame recognition. A graph structure is constructed for fuzzy clustering boundary point sets, treating each image frame as a node in the graph. The weights of edges between nodes are calculated using the cosine similarity between frame-level feature vectors, accurately reflecting the feature relationships between image frames. This graph structure is input into a multi-layer graph convolutional network, achieving deep propagation and expression of information between image frames through layer-by-layer aggregation and nonlinear mapping of adjacent node features. During model training, frames with the highest local density selected by an improved density peak clustering algorithm are used as pseudo-label positive samples to guide the graph embedding features to evolve towards the anomaly category. The prediction error of the pseudo-label nodes is optimized to improve the model's generalization ability. After training, an anomaly probability score is output for each node, and the score results are judged according to a set threshold. Finally, all image frames with anomaly probability scores higher than the threshold are selected to form the final set of anomaly image frames. This process fully combines graph structure modeling capabilities with the clustering pseudo-label guidance mechanism, effectively improving the accuracy and robustness of anomaly detection.

[0110] In this embodiment, S5 specifically includes:

[0111] S51. Merge the final set of abnormal frames, the set of candidate points of abnormality extracted based on the density-offset bivariate function surface in step S34, and the set of trend abnormality points marked in step S36 to construct a global set of candidate abnormal frames.

[0112] S52. Extract the frame number and corresponding density value of each image frame in the local density sequence from the anomaly candidate point set as a density anomaly reference basis, set a density recognition threshold, and filter frames with local density values ​​lower than the density recognition threshold to form a saddle point anomaly reference set.

[0113] S53. Based on each image frame in the trend anomaly point set, extract the spatial drift path within the corresponding time period from the cluster center position sequence and calculate the continuous drift distance; set a spatial drift distance threshold and select image frames whose continuous drift distance exceeds the spatial drift distance threshold to form a drift anomaly reference set.

[0114] S54. Perform a set merging operation based on the global candidate abnormal frame set, the saddle point abnormal reference set, and the drift abnormal reference set to form a joint abnormal judgment result set. Sort the joint abnormal judgment result set according to the image frame time order and output the complete abnormal frame sequence.

[0115] After initial anomaly identification, this invention further designs a joint anomaly judgment mechanism to improve the accuracy and completeness of anomaly image frame screening. A global candidate anomaly frame set is constructed by fusing the final anomaly frame set, the anomaly candidate point set extracted based on the density-offset bivariate function surface, and the trend anomaly point set identified in the density inertial map. Based on this, the frame number and corresponding density value of the anomaly candidate points in the local density sequence are extracted, and a density recognition threshold is set to identify frames with significantly low density values, forming a saddle point anomaly reference set. Simultaneously, the spatial drift path of the cluster centers corresponding to the trend anomaly points is analyzed, the continuous drift distance is calculated, and a spatial drift distance threshold is set to filter out frames with large offsets, forming a drift anomaly reference set. Finally, by performing a set merging operation on the global candidate anomaly frame set and the two reference sets, a joint anomaly judgment result set is formed, and a complete anomaly frame sequence is output in chronological order, thereby achieving a more comprehensive and accurate identification of abnormal water accumulation events in video images.

[0116] In this embodiment, S6 specifically includes:

[0117] S61. Perform frame-by-frame traversal detection on the complete abnormal frame sequence output in claim 6 to determine whether any image frame is marked as an abnormal frame; if there is an image frame marked as abnormal, trigger the abnormal event determination process.

[0118] S62. The anomaly event determination process specifically includes: extracting the timestamp and frame index number of the abnormal image frame in the video image frame sequence, and constructing an image anomaly event tuple set. ,in This indicates the timestamp corresponding to the abnormal frame. This indicates the frame number of the frame in the original frame sequence;

[0119] For each image anomaly event tuple Generate corresponding water accumulation anomaly event tags. The water accumulation anomaly event tags include a unique event identifier, anomaly type, timestamp, and image frame index, and are encapsulated into a standardized anomaly event information structure according to a set format.

[0120] The standardized abnormal event information structure is sent to the flood control and drainage dispatching platform through a communication interface, which supports the HTTP communication protocol or the MQ transmission protocol based on message queues.

[0121] Record the sending status feedback of abnormal event information. If the feedback is successful, the sending log is written to the event log file. If the sending fails, the retry mechanism is executed: try to resend within the set number of retries. If the sending still fails, the event information is cached in the local abnormal buffer pool and automatically resent in batches after communication is restored.

[0122] After completing the identification and sequence output of abnormal image frames, this invention further constructs an event-driven reporting process for waterlogging anomalies. By traversing the complete sequence of abnormal frames frame by frame, it determines whether any image frames are marked as abnormal. Once an abnormal frame is detected, an anomaly event determination process is triggered. In this process, the timestamp and frame index number of the corresponding image frame in the video image frame sequence are extracted to form a set of image anomaly event tuples, and a waterlogging anomaly event tag is generated based on each tuple. The event tag includes a unique event identifier, anomaly type, timestamp, and image frame index, and is encapsulated into a standardized anomaly event information structure. This structure is sent to the flood control scheduling platform through a communication interface supporting HTTP or Message Queuing (MQ) protocols, and its sending status feedback is recorded. If the feedback fails, a retry mechanism is initiated. After exceeding the set number of retries, the event information is cached in a local anomaly buffer pool. Once communication is restored, batch resending is automatically performed to ensure the complete and reliable transmission of anomaly information and coordinated flood control response.

[0123] Example 1:

[0124] To verify the feasibility of this invention in practice, it was applied to a flood-prone area monitored by a municipal drainage management department. A fixed video surveillance camera was deployed at the entrance of an underground passage with a high risk of water accumulation. Monitoring image data was continuously collected for 20 days, and the video image anomaly detection method based on intelligent clustering algorithm proposed in this invention was verified by combining the records of artificial water accumulation events.

[0125] In actual deployment, the front-end camera captures an image stream at a rate of 2 frames per second and uploads it to the back-end server. The server first performs frame extraction and image standardization preprocessing on the video image stream to remove invalid image noise and unify the resolution to 640×480 pixels. After the image sequence is generated, the system automatically extracts the brightness information, edge gradient features, grayscale statistics, motion region contours, and pixel variance of each frame to construct a set of frame-level feature vectors and builds an inter-frame distance matrix within a 5-minute sliding window.

[0126] The system constructs a density-offset bivariate function surface based on local density estimation and center offset distance calculation, and extracts anomalous saddle point frames using the Laplacian operator. Fuzzy cluster boundary points are divided using a density splitting tree, and a trend-based anomaly detection mechanism is constructed by combining density inertial maps. Finally, the results are input into the FixMatch semi-supervised classifier, which outputs anomaly confidence scores for each frame, resulting in a candidate anomaly frame set. After subsequent graph neural network inference optimization and global anomaly set fusion and reordering, the system finally outputs a complete anomaly frame sequence and automatically generates waterlogging anomaly event tags with timestamps and image indexes, which are uploaded to the flood control and drainage dispatching platform via HTTP. Specific experimental data are shown in Table 1 below, which compares the anomaly detection results.

[0127] Table 1 Comparison of Anomaly Detection Results

[0128] Project Number Test period (date) Total number of frames captured Manually marked number of water accumulation events Number of abnormal events detected by the system Valid match count Matching accuracy Average recognition latency (seconds) Average resource utilization A Days 1-5 108000 3 3 3 100% 1.5 68% B Days 6-10 107200 4 4 3 75% 1.9 72% C Days 11-15 109800 4 5 4 80% 1.6 69% D Days 16-20 110400 3 4 3 75% 1.8 70% total - 435400 14 16 13 92.3% 1.7

[0129] During the test period, manual annotation recorded 14 water accumulation-related events, all occurring within 5-30 minutes after short-term heavy rainfall. Corresponding video images showed watermarks spreading at the contact points between vehicle tires and the road surface, slow-moving vehicles, and pedestrians lingering. Statistics showed that the system automatically labeled 314 abnormal image frames, detecting 12 complete abnormal frame sequences. Of these, 11 events highly overlapped with the manually annotated time periods, with a timing deviation within ±2 minutes, achieving a detection accuracy of 92.3%. The system's average time to identify each event was 1.7 seconds, significantly faster than traditional methods relying on manual inspection or backend image filtering.

[0130] By comparing the operational efficiency of image analysis systems with and without the algorithm of this invention, under the same server load, this invention can stably achieve real-time processing of 30-channel images on a 4-core CPU + 8GB memory, processing over 430,000 image frames per day with resource utilization controlled below 70%, and exhibiting no significant frame drops or latency issues, demonstrating excellent engineering deployability and real-time response capabilities. These experimental results show that this invention not only possesses high anomaly recognition accuracy and low-latency output capabilities, but also effectively supports the coordinated monitoring tasks of urban-level flood control events, providing crucial data support for intelligent urban flood management.

[0131] The experimental results above demonstrate that this invention possesses excellent real-time performance, accurate event response, and efficient system resource utilization, making it highly valuable for intelligent monitoring of urban drainage.

[0132] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A method for identifying anomalies in video surveillance data of flood-prone areas based on intelligent clustering algorithm, characterized in that, Includes the following steps: S1. Collect video image data from video surveillance cameras deployed in flood-prone urban areas, and preprocess the video image data; S2. Preprocess each frame of the video image frame sequence; S3. Based on the inter-frame distance matrix and frame-level feature vectors, kernel density estimation and center offset distance calculation are introduced to construct a density-offset bivariate function surface to extract saddle point candidate frames, form a density split tree structure to identify fuzzy clustering boundary point set, and capture trend abnormal states by constructing a density inertial map structure. Finally, an improved density peak clustering algorithm is formed, and the final abnormal frame set is output by combining it with the FixMatch semi-supervised classification model. S4. Construct a graph structure based on the fuzzy clustering boundary point set and input it into the graph neural network model to output the final set of abnormal image frames; S5. Based on the joint anomaly judgment result set, a complete anomaly frame sequence is constructed; S6. When any image frame is determined to be an abnormal state frame, the abnormal event determination process is triggered.

2. The method for identifying anomalies in video surveillance data of flood-prone areas based on intelligent clustering algorithm according to claim 1, characterized in that, The preprocessing of video image data includes frame extraction, image enhancement, and noise filtering to generate a standardized video image frame sequence.

3. The method for identifying anomalies in video surveillance data of flood-prone areas based on intelligent clustering algorithm according to claim 1, characterized in that, The preprocessing of each frame in the video image frame sequence includes extracting reflection brightness, edge gradient, grayscale mean, motion region boundary and pixel distribution variance to construct a set of frame-level feature vectors.

4. The method for identifying anomalies in video surveillance data of flood-prone areas based on intelligent clustering algorithm according to claim 1, characterized in that, S3 specifically includes: S31. Construct an inter-frame distance matrix between each image frame within a sliding time window. The inter-frame distance matrix is ​​calculated based on the Euclidean distance between the frame-level feature vectors corresponding to each image frame. S32. The local density value of each frame image is calculated using the kernel density estimation method based on the inter-frame distance matrix. The local density values ​​of each image frame are arranged in chronological order to form a local density sequence. The cumulative result of the density value increment within any time period represents the density change trend. The local density value is obtained by counting the number of image frames whose distance from the current image frame is less than a set truncation distance. The truncation distance is quickly and adaptively adjusted by a parameter optimization model based on a meta-learning mechanism. S33. Determine the corresponding center offset distance based on the local density value of each image frame. The center offset distance is the minimum distance between the current image frame and all image frames with local density values ​​higher than the current image frame. When the current image frame is the image frame with the largest local density value within the sliding time window, the center offset distance is defined as the maximum distance between the current image frame and all other image frames. S34. Construct a density-offset bivariate function surface by combining the local density value and the center offset distance. Identify regions where the function value changes erratically based on the density-offset bivariate function surface. Extract local minima from these regions as saddle point candidate frames to form an anomaly candidate point set. S35. Using the frame with the highest local density as the cluster center, construct a density split tree structure along the density decreasing path to identify boundary nodes with weak density change continuity and abnormal inter-frame distance distribution, thus forming a fuzzy clustering boundary point set. S36. Construct a density inertial map structure based on the local density sequence and cluster center position sequence of continuous image frames, analyze the density change trend of image frames on the time axis and the spatial drift trajectory of cluster centers. When the frame density change continues to rise or the cluster center position shifts continuously, determine that the current frame is in a trend abnormal state and mark it as a trend abnormal point set. The spatial drift trajectory of cluster centers consists of the two-dimensional coordinate points of cluster centers in the image space within continuous time steps, forming a cluster center position sequence. The cluster center position sequence is obtained by calculating the change of Euclidean distance between cluster centers of adjacent frames. S37. Construct a pseudo-label dataset for the anomaly candidate point set, the fuzzy cluster boundary point set, and the trend anomaly point set. Use the image frames in the anomaly candidate point set and the fuzzy cluster boundary point set as pseudo-label positive samples, and use the trend anomaly point set with low density value and large offset distance as pseudo-label negative samples. Iteratively train on the FixMatch semi-supervised classification model, output the anomaly confidence score result for each frame, re-rank the anomaly candidate frames according to the confidence score, remove candidate frames with confidence scores lower than the set threshold, and retain the final set of anomaly frames.

5. The method for identifying anomalies in video surveillance data of flood-prone areas based on intelligent clustering algorithm according to claim 1, characterized in that, S4 specifically includes: S41. For the fuzzy clustering boundary point set, construct a graph structure. Each node in the graph structure corresponds to a boundary image frame. The edges in the graph structure represent the feature similarity relationship between any two boundary image frames. The weight of the edges is calculated by the cosine similarity between the corresponding frame-level feature vectors of the image frames. S42. Input the graph structure into the graph neural network model. The graph neural network model is constructed using a multi-layer graph convolutional structure. Each layer performs adjacent node feature aggregation and non-linear mapping operations, and updates the node embedding representation layer by layer. S43. During the training phase of the graph neural network, the set of frames with the maximum local density obtained by the improved density peak clustering algorithm is used as pseudo-label samples. The node labels in the graph neural network are assigned as anomaly categories to guide the graph embedding feature propagation process. The graph neural network optimizes its parameters by minimizing the prediction error of the pseudo-label nodes. S44. After the graph neural network is trained, an anomaly probability score is output for each image frame node in the graph structure to form a corresponding anomaly probability vector. Based on whether the anomaly probability score of each image frame node is higher than a set threshold, it is determined whether the current image frame node is judged as the final abnormal frame. All image frame nodes with anomaly probability scores higher than the set threshold form the final abnormal image frame set.

6. The method for identifying anomalies in video surveillance data of flood-prone areas based on intelligent clustering algorithm according to claim 1, characterized in that, S5 specifically includes: S51. Merge the final set of abnormal frames, the set of candidate points of abnormality extracted based on the density-offset bivariate function surface in step S34, and the set of trend abnormality points marked in step S36 to construct a global set of candidate abnormal frames. S52. Extract the frame number and corresponding density value of each image frame in the local density sequence from the anomaly candidate point set as a density anomaly reference basis, set a density recognition threshold, and filter frames with local density values ​​lower than the density recognition threshold to form a saddle point anomaly reference set. S53. Based on each image frame in the trend anomaly point set, extract the spatial drift path within the corresponding time period from the cluster center position sequence and calculate the continuous drift distance; set a spatial drift distance threshold and select image frames whose continuous drift distance exceeds the spatial drift distance threshold to form a drift anomaly reference set. S54. Perform a set merging operation based on the global candidate abnormal frame set, the saddle point abnormal reference set, and the drift abnormal reference set to form a joint abnormal judgment result set. Sort the joint abnormal judgment result set according to the image frame time order and output the complete abnormal frame sequence.

7. The method for identifying anomalies in video surveillance data of flood-prone areas based on intelligent clustering algorithm according to claim 1, characterized in that, S6 specifically includes: S61. Perform frame-by-frame traversal detection on the complete abnormal frame sequence output in claim 6 to determine whether any image frame is marked as an abnormal frame; if there is an image frame marked as abnormal, trigger the abnormal event determination process. S62. The abnormal event determination process specifically includes: extracting the timestamp and frame index number of the abnormal image frame in the video image frame sequence to form a set of image abnormal event tuples for the image abnormal event. For each image anomaly event tuple, a corresponding water accumulation anomaly event tag is generated. The water accumulation anomaly event tag contains a unique event identifier, anomaly type, timestamp, and image frame index, and is encapsulated into a standardized anomaly event information structure according to a set format. The standardized abnormal event information structure is sent to the flood control and dispatching platform through the communication interface; Record the sending status feedback of abnormal event information. If the feedback is successful, the sending log is written to the event log file. If the sending fails, the retry mechanism is executed: try to resend within the set number of retries. If the sending still fails, the event information is cached in the local abnormal buffer pool and automatically resent in batches after communication is restored.

8. The method for identifying anomalies in video surveillance data of flood-prone areas based on intelligent clustering algorithm according to claim 1, characterized in that, The improved density peak clustering algorithm specifically includes the following steps: The Euclidean distance between image frames is calculated based on the inter-frame distance matrix and frame-level feature vectors within the sliding time window, and the inter-frame distance matrix is ​​constructed. The local density value of each image frame is calculated using the kernel density estimation method to generate a local density sequence. The center offset distance is determined based on the local density value and density ranking. A density-offset bivariate function surface is constructed, and local minima in the function surface are extracted as saddle point candidate frames to form an anomaly candidate point set. Using the frame with the highest local density as the cluster center, a density split tree is constructed along the density decreasing path, and nodes with weak density change continuity or abnormal inter-frame distance distribution are identified as the fuzzy clustering boundary point set. By combining the local density sequence of continuous image frames with the cluster center location sequence, a density inertial map structure is constructed. The density change trend of image frames and the spatial drift trajectory of cluster centers are analyzed to identify image frames with continuously increasing density or continuous shift of cluster centers, thus forming a set of trend anomalies. The pseudo-label dataset is constructed by combining the set of anomaly candidate points, the set of fuzzy clustering boundary points, and the set of trend anomalies. It is then input into the FixMatch semi-supervised classification model to train and output the anomaly confidence score for each frame. Based on the score results, the anomaly candidate frames are reordered, frames with confidence scores below a set threshold are removed, and the final set of anomaly frames is retained.

Images