River anomaly event recognition method and system based on drone inspection

Abstract

The present invention discloses a method and system for identifying river channel abnormal events based on drone inspections, including regularly and directionally inspecting the river channel by drones, capturing river channel image data, and performing noise reduction processing on the river channel image data; identifying corresponding river channel information and suspected abnormal events in the processed image data; performing multi-dimensional tracking and analysis on the suspected abnormal events, identifying the event characteristic trajectories, and obtaining event attributes; analyzing and judging the event attributes to obtain the identification results of abnormal events; by adopting multi-dimensional tracking and analysis means, the present invention realizes the refined real-time monitoring and tracking of river channel abnormal events, obtains images of the target object at the same position at different time points, deeply analyzes the position change of the target object, accurately identifies its movement trajectory, intelligently identifies the behavior state and activity rules of the target object, improves the accuracy of abnormal event identification, and solves the problems that manual inspections cannot achieve real-time monitoring and continuous tracking of abnormal events.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent analysis, and specifically to a method and system for identifying abnormal events in a river channel based on drone patrol. Background Art

[0002] In the traditional process of river channel patrol, relying on manual patrol not only has low efficiency, but is also easily affected by weather and lighting conditions, resulting in insufficient accuracy and consistency of patrol results. Manual patrol usually requires a large amount of manpower and time to cover the entire river channel area, which not only increases costs, but may also miss abnormal situations due to the negligence or fatigue of patrol personnel. At the same time, due to the limitation of patrol frequency, it is difficult for manual patrol to achieve real-time monitoring and continuous tracking of abnormal events, resulting in some potential environmental problems not being discovered and processed in a timely manner. For example, problems such as water pollution, illegal emissions, and river channel blockage may deteriorate rapidly if not discovered in a timely manner, posing a serious threat to the ecological environment and public health. Summary of the Invention

[0003] The purpose of the present invention is to provide a method and system for identifying abnormal events in a river channel based on drone patrol, to solve the problem of missing abnormal situations due to the negligence or fatigue of patrol personnel, and to improve the ability to identify abnormal events in the river channel.

[0004] The purpose of the present invention can be achieved through the following technical solutions:

[0005] The present application provides a method for identifying abnormal events in a river channel based on drone patrol, including the following steps:

[0006] Regularly and directionally patrol the river channel by drone, capture river channel image data, and perform noise reduction processing on the river channel image data;

[0007] Identify the corresponding river channel information and suspected abnormal events in the processed image data;

[0008] Perform multi-dimensional tracking and analysis on the suspected abnormal events, identify the event characteristic trajectory, and obtain the event attributes;

[0009] Among them, the multi-dimensional tracking and analysis of the suspected abnormal events includes: obtaining the target objects at the same position in the image data captured in multiple time periods; identifying the movement trajectory of the target objects according to the position change of the target objects; then identifying the behavior state of the target objects according to the movement trajectory to obtain the behavior trajectory of the target objects; identifying the activity pattern of the target objects according to the behavior trajectory of the target objects; and identifying whether the target objects are abnormal events according to the activity pattern of the target objects.

[0010] Identifying whether the target object is an abnormal event according to the activity pattern of the target object further includes: extracting the image features of the target object using a Convolutional Neural Network (CNN), and classifying similar images using the K-means clustering algorithm to form different types of image clusters;

[0011] Obtaining the timestamps of each image in each image cluster, sorting them in chronological order, and screening out M image clusters within consecutive N frames as samples;

[0012] Randomly selecting P pictures from each sample to form a subset;

[0013] For each subset, calculating the Euclidean distance between two adjacent pictures, and classifying the pictures corresponding to the distances less than the threshold into the same sequence;

[0014] Repeating the above process until the sequence length reaches the set maximum length Q, and then counting the frequencies of each sequence to obtain the characteristic trajectory of the abnormal event.

[0015] Further, the drone regularly conducts directional inspections of the river channel and captures river channel image data, including: performing timed cruise scans and captures on the river channel according to a pre-set route to obtain image data of each point on the route; when receiving an abnormal event reported by the user, adjusting the current heading to near the abnormal event reported by the user and capturing image data near the abnormal event;

[0016] Performing noise reduction processing on the image data, including:

[0017] Using a Gaussian filter to smooth the image data;

[0018] Using the threshold segmentation method to perform binary processing on the image data;

[0019] Using morphological operations to perform erosion and dilation processing on the image data.

[0020] Further, the threshold segmentation method divides the gray value of the image into object and background parts through the Otsu algorithm, including:

[0021] Calculating the histogram of the image, that is, the frequency of each gray level;

[0022] Dividing each element of the histogram by the total number of pixels of the image to obtain a normalized histogram;

[0023] Calculating the cumulative probability distribution of each gray level according to the normalized histogram, expressed as , where i is the gray level;

[0024] For each gray level of the threshold t, calculating the between-class variance of the foreground and the background, where the probability of the foreground Probability of foreground and background It is expressed as:

[0025] ;

[0026] Wherein, is the sum of the probabilities of all gray levels below the threshold t, is the sum of the probabilities of all gray levels at t and above;

[0027] According to the threshold t, calculate the means of the foreground and background and , which are expressed as:

[0028] ; ; wherein, is the weighted average gray value of the gray levels below the threshold t, is the weighted average gray value of the gray levels above t;

[0029] Calculate the between-class variance corresponding to each threshold t , and find the maximum value. The between-class variance is expressed as: , by determining the t value that makes maximum for segmenting the image into foreground and background.

[0030] Further, before the corresponding river channel information and suspected abnormal events in the image data after the recognition processing, it includes:

[0031] Perform edge detection on the original image data captured;

[0032] Perform target localization on the image data after edge detection to obtain the position coordinates of the object to be detected;

[0033] Determine whether the object to be detected is located within a preset river channel area; if so, regard the object to be detected as a suspected abnormal event.

[0034] Further, compare the suspected abnormal event with a preset rule. When the suspected abnormal event meets the preset rule, it is determined as the target object that needs multi-dimensional tracking analysis; wherein, the preset rule is used to indicate at least one of the following: abnormal event type, abnormal event occurrence time, abnormal event occurrence location.

[0035] Further, when performing multi-dimensional tracking analysis on a suspected abnormal event, it further includes: establishing a tracking model, where the tracking model includes a first sub-model and a second sub-model; wherein, the first sub-model is used to predict the location information of the suspected abnormal event at the next moment according to the historical location information and the current location information of the suspected abnormal event; the second sub-model is used to estimate the error information of the location information of the suspected abnormal event at the next moment according to the historical location information and the current location information of the suspected abnormal event; using the first sub-model and the second sub-model to perform tracking analysis on the suspected abnormal event until the suspected abnormal event disappears.

[0036] Further, performing tracking analysis on the suspected abnormal event includes:

[0037] Obtaining image data of the suspected abnormal event at multiple time points;

[0038] Calculating the motion vector of the suspected abnormal event according to the image data of the suspected abnormal event at two adjacent time points;

[0039] Predicting the image data of the suspected abnormal event at subsequent time points according to the motion vector to generate a predicted trajectory map;

[0040] If the error between the predicted trajectory map and the actual trajectory map is less than the threshold, it is considered that the suspected abnormal event is real, and continue to perform tracking analysis;

[0041] End the tracking analysis until the error between the predicted trajectory map and the actual trajectory map is greater than the threshold.

[0042] Further, the identifying the event feature trajectory and obtaining the event attribute is, during the tracking analysis process, continuously tracking multiple frames of the suspected abnormal events of the same category according to the time stamp and the spatial coordinates to form a trajectory sequence, extracting the feature vectors of the abnormal events to form the feature trajectory of the abnormal events, and obtaining the attributes of the abnormal events according to the feature trajectory.

[0043] Further, after obtaining the attributes of the suspected abnormal event, it further includes: analyzing and judging the event attributes to obtain the abnormal event recognition result.

[0044] A river channel abnormal event recognition system based on UAV inspection is specifically applied to a river channel abnormal event recognition method based on UAV inspection, including a data acquisition module, an image preprocessing module, an abnormal detection and tracking analysis module, and an event recognition and decision module.

[0045] The data acquisition module: The UAV conducts river channel inspection according to a predetermined route, uses a camera to capture river channel images, receives abnormal events reported by users in real time, and adjusts the route for targeted shooting.

[0046] An image preprocessing module that performs noise reduction on the collected image data, smooths the image using a Gaussian filter, performs threshold segmentation using the Otsu algorithm to distinguish the foreground and background in the image, and applies morphological operations for erosion and dilation processing;

[0047] An anomaly detection and tracking analysis module that performs edge detection and target localization, determines the position coordinates of the object to be detected, compares the suspected anomaly events with preset rules, filters out the targets that need to be tracked and analyzed, performs multi-dimensional tracking and analysis on the targets, identifies the movement trajectory and behavior state, and constructs a behavior trajectory model;

[0048] An event recognition and decision-making module that uses CNN and K-means algorithms to extract image features and perform clustering to form feature trajectories, establishes a tracking model to predict and evaluate the spatial position changes of suspected anomaly events, performs positioning and tracking on the suspected anomaly events, and analyzes and judges the attributes and authenticity of the anomaly events according to the feature trajectories and behavior patterns.

[0049] The beneficial effects of the present invention are:

[0050] By adopting multi-dimensional tracking and analysis means, the refined real-time monitoring and tracking of river channel anomaly events are realized. By obtaining images of the target object at the same position at different time points, the position changes of the target object are deeply analyzed, its movement trajectory is accurately identified, and then according to these movement trajectories, the behavior state and activity rules of the target object are intelligently identified. This multi-dimensional analysis not only greatly improves the accuracy of anomaly event recognition, but also through the integration of a prediction model and an error estimation model, can provide accurate tracking and analysis of abnormal situations until the anomaly event is properly handled or disappears, realizing the refined real-time monitoring and continuous tracking of river channel anomaly events, solving the problem that manual inspections cannot achieve real-time monitoring and continuous tracking of anomaly events, improving the accuracy of anomaly event recognition, and reducing misjudgments and missed judgments caused by human factors;

[0051] By using a convolutional neural network (CNN) and the K-means clustering algorithm, the present invention has made breakthrough progress in image feature extraction and anomaly event recognition. CNN effectively captures the local features of the image through its deep structure and constructs more complex and abstract feature representations layer by layer, greatly enhancing the recognition ability of the system. The K-means clustering algorithm effectively classifies these features in the feature space to form image clusters with the same or similar attributes. This process not only improves the accuracy of anomaly event recognition, but also can quickly respond and accurately locate abnormal behaviors by analyzing the behavior patterns in consecutive image frames, providing strong technical support for the timely handling of river channel anomaly events;

[0052] Through regular and directional river channel inspections by drones, the present invention realizes the automatic collection of river channel image data, greatly improving the efficiency and data quality of river channel inspections. Drone inspections avoid the interference of human factors, ensuring the consistency and reliability of data collection. Combined with image noise reduction processing technology, the system further purifies the image data, providing high-quality input for subsequent feature extraction and anomaly analysis. The present invention not only reduces the manpower burden, but also provides an efficient and reliable solution for real-time monitoring and anomaly response of river channels through refined management, solving the problems of low efficiency of manual inspections and unstable data quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] For a better understanding and implementation, the technical solutions of the present application will be described in detail below with reference to the accompanying drawings.

[0054] Figure 1 Schematic flow chart of the method for identifying river channel anomaly events based on drone inspections provided in Embodiment 1 of the present application;

[0055] Figure 2 Schematic flow chart of the method for identifying river channel anomaly events based on drone inspections provided in Embodiment 1 of the present application for noise reduction processing of image data;

[0056] Figure 3 Schematic flow chart of the method for identifying river channel anomaly events based on drone inspections provided in Embodiment 1 of the present application for dividing the gray value of an image into an object and a background;

[0057] Figure 4 Schematic flow chart of the method for identifying river channel anomaly events based on drone inspections provided in Embodiment 1 of the present application before identifying the river channel information and suspected anomaly events in image data;

[0058] Figure 5 Schematic flow chart of the method for identifying river channel anomaly events based on drone inspections provided in Embodiment 1 of the present application for identifying whether a target object is an anomaly event;

[0059] Figure 6 Schematic flow chart of the method for identifying river channel anomaly events based on drone inspections provided in Embodiment 1 of the present application for tracking and analyzing suspected anomaly events. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0060] To further elaborate on the technical means and effects adopted by the present invention to achieve the predetermined invention purpose, exemplary embodiments will be described in detail herein, and the examples are shown in the drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. On the contrary, they are merely examples of methods and systems consistent with some aspects of the present application as detailed in the appended claims.

[0061] The terms used in the present application are for the purpose of describing specific embodiments only and are not intended to limit the present application. The singular forms "a", "the", and "said" used in the present application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used herein refers to and includes any and all possible combinations of one or more of the associated listed items.

[0062] The following will, in conjunction with the accompanying drawings and preferred embodiments, elaborate in detail on the specific implementation manners, features, and effects of the present invention.

[0063] Embodiment 1

[0064] Please refer to Figures 1-6 , this embodiment provides a method and system for identifying river channel abnormal events based on drone inspections, which solves the problem of missed detection of abnormal situations due to the negligence or fatigue of inspection personnel and improves the ability to identify river channel abnormal events.

[0065] The present invention provides a method for identifying river channel abnormal events based on drone inspections, including the following steps:

[0066] S1. Regularly and directionally inspect the river channel by drone, capture river channel image data, and perform noise reduction processing on the river channel image data;

[0067] S2. Identify the corresponding river channel information and suspected abnormal events in the processed image data;

[0068] S3. Perform multi-dimensional tracking analysis on the suspected abnormal events, identify the event characteristic trajectories, and obtain the event attributes;

[0069] Further, the drone regularly and directionally inspects the river channel and captures river channel image data, including: performing timed cruise scanning and shooting on the river channel according to a preset flight path to obtain image data of each point on the flight path; when receiving an abnormal event reported by a user, adjusting the current heading to the vicinity of the abnormal event reported by the user and capturing image data in the vicinity of the abnormal event.

[0070] Further, perform noise reduction processing on the image data, including: S11. Smooth the image data using a Gaussian filter; specifically, for each pixel in the image, replace the pixel value with the weighted average value within the neighborhood;

[0071] S12. Binarize the image data using the threshold segmentation method;

[0072] The threshold segmentation method divides the gray values of the image into an object part and a background part through the Otsu algorithm, including:

[0073] S111. Calculate the histogram of the image, that is, the frequency of occurrence of each gray level;

[0074] S112. Divide each element of the histogram by the total number of pixels in the image to obtain a normalized histogram;

[0075] S113. Calculate the cumulative probability distribution of each gray level according to the normalized histogram, expressed as , where i is the gray level;

[0076] S114. For each gray level of the threshold t, calculate the between-class variance of the foreground and the background, where the probability of the foreground and the probability of the background are expressed as:

[0077] ;

[0078] Among them, is the sum of the probabilities of all gray levels below the threshold t, is the sum of the probabilities of all gray levels at t and above;

[0079] S115. According to the threshold t, calculate the means and of the foreground and the background, expressed as:

[0080] ; ; Among them, is the weighted average gray value of the gray levels below the threshold t, is the weighted average gray value of the gray levels above t;

[0081] S116. Calculate the between-class variance corresponding to each threshold t, and find the value that makes the largest. The between-class variance is expressed as: , and determine the t value that makes the largest for dividing the image into the foreground and the background.

[0082] S13. Perform erosion and dilation processing on the image data using morphological operations.

[0083] Among them, erosion is to convolve the image with a structuring element and replace the value of each pixel with the minimum value of the point and the corresponding position of the structuring element in its neighborhood. Dilation is to replace the value of each pixel with the maximum value of the point and the corresponding position of the structuring element in its neighborhood.

[0084] It should be explained that the drone conducts regular cruise scans and shootings of the river channel according to the preset route to ensure systematic collection of river channel image data. When receiving an abnormal event reported by the user, the drone can flexibly adjust its heading and quickly collect images of the abnormal area. The collected images will then undergo noise reduction processing. The Gaussian filter is used to smooth the image to reduce image noise and retain important information. Next, the image is subjected to threshold segmentation using the Otsu algorithm. This algorithm automatically calculates the optimal threshold and divides the gray value of the image into two parts: foreground (object) and background, so as to clearly distinguish abnormal objects in the river channel. Finally, erosion and dilation processing are performed on the image through morphological operations to further remove noise and details in the image and emphasize the structural features of the image, providing clear image data for feature extraction and subsequent analysis of abnormal events. This process improves the accuracy and efficiency of river channel abnormal event detection through automated and intelligent image processing technologies.

[0085] Further, before the river channel information and suspected abnormal events corresponding to the identified and processed image data, it includes:

[0086] S21. Perform edge detection on the original image data taken; use an edge detection algorithm to process the original image taken and extract the edges of the river channel and its surrounding areas in the image.

[0087] S22. Perform target localization on the image data after edge detection to obtain the position coordinates of the object to be detected; specifically, use a target detection algorithm to locate the position of the object to be detected and obtain its coordinates.

[0088] S23. Determine whether the object to be detected is located within the preset river channel area; if so, regard the object to be detected as a suspected abnormal event. Compare the position coordinates of the target object with the preset river channel area to judge whether it is within the river channel range. If the target object is located within the preset river channel area, mark it as a suspected abnormal event and conduct further analysis.

[0089] For example, the image captured by the drone shows an unknown floating object at a certain location in the river channel. Through edge detection and target positioning, the coordinates of the floating object are determined to be (X1, Y1). The preset river channel area is defined as a rectangular area through GIS data, with the coordinates of its lower left corner being (Xmin, Ymin) and the coordinates of its upper right corner being (Xmax, Ymax). If Xmin ≤ X1 ≤ Xmax and Ymin ≤ Y1 ≤ Ymax, the floating object is located within the river channel area and is marked as a suspected abnormal event.

[0090] Further, compare the suspected abnormal event with the preset rules. When the suspected abnormal event meets the preset rules, it is determined as the target object that requires multi-dimensional tracking and analysis; wherein, the preset rules are used to indicate at least one of the following: abnormal event type, abnormal event occurrence time, and abnormal event occurrence location.

[0091] Specific abnormal event types include:

[0092] Sewage discharge: Identify the discharge of unknown liquids appearing in the image, especially discharges with colors significantly different from the surrounding water bodies; Garbage dumping: Detect garbage piles or abnormal objects appearing in the river channel, such as plastic bags, bottles, etc.; Illegal swimming: Identify swimmers appearing in non-swimming areas; Person falling into the water: Detect whether there are people or animals accidentally falling into the water in the river channel.

[0093] The abnormal event occurrence time includes:

[0094] Specific time periods: Events occurring within certain time periods may be more concerned, such as illegal garbage dumping at night;

[0095] The abnormal event occurrence location includes:

[0096] Within the protected area: Any abnormal event occurring within the ecological protection area or other sensitive areas of the river channel; Historical problem areas: Events occurring again at locations where abnormal events have occurred multiple times in the past; Monitoring blind spots: Events occurring at the edges or dead corners of the river channel monitoring.

[0097] Further, the multi-dimensional tracking and analysis of the suspected abnormal event includes: obtaining the target object at the same position in the image data captured in multiple time periods; identifying the movement trajectory of the target object according to the position change of the target object;

[0098] Then, identify the behavior state of the target object according to the movement trajectory to obtain the behavior trajectory of the target object; identify the activity pattern of the target object according to the behavior trajectory of the target object; and identify whether the target object is an abnormal event according to the activity pattern of the target object.

[0099] It should be explained that when the river management department monitors an unknown object in the non-swimming section of the river through the UAV inspection system and initially judges that it may be illegally dumped garbage, the process of multi-dimensional tracking and analysis includes: analyzing the position change of the object in the time-series image, using image matching technology to determine whether the object is moving, and identifying the movement trajectory. According to the movement trajectory of the object, analyze its behavior state, such as the moving speed and direction of the floating object; then combine the movement speed, direction and duration of the object to construct a behavior trajectory model, and identify the activity pattern of the object through statistical analysis of the behavior trajectory, such as whether it appears at the same time every day, whether it moves with the water flow, etc. If the activity pattern of the object does not conform to the behavior of the object in the normal river, it means that if an object should move with the water flow but does not move, or continuously appears in a place where it should not appear, this may indicate that it is placed or dumped artificially; if it continuously appears in the non-swimming area and does not move with the water flow, it may be judged as a suspicious behavior, which means that if an object repeatedly appears in the area where swimming is prohibited and this area usually does not have such objects, this may indicate that illegal behavior has occurred, such as illegal garbage dumping.

[0100] In a river, most objects move with the direction and speed of the water flow. If an object remains in the same position for a period of time and does not drift with the water flow, this may indicate that the object is fixed to the riverbed or abandoned there. The management department can use the data collected by the UAV inspection system to identify and judge whether there is suspicious behavior by analyzing the movement trajectory and activity pattern of the object.

[0101] Furthermore, identifying whether the target object is an abnormal event according to the activity pattern of the target object further includes:

[0102] S31. Use the convolutional neural network CNN to extract the image features of the target object, and use the clustering algorithm K-means to classify similar images to form different types of image clusters;

[0103] S32. Obtain the timestamps of each image in each image cluster, sort them in chronological order, and select M image clusters within consecutive N frames as samples;

[0104] S33. Randomly select P pictures from each sample to form a subset;

[0105] S34. For each group of subsets, calculate the Euclidean distance between two adjacent pictures, and classify the pictures corresponding to the distances less than the threshold into the same sequence;

[0106] S35. Repeat the above process until the sequence length reaches the set maximum length Q; finally, count the frequencies of each sequence to obtain the characteristic trajectory of this abnormal event.

[0107] Further, when performing multi-dimensional tracking analysis on a suspected abnormal event, it further includes: establishing a tracking model, where the tracking model includes a first sub-model and a second sub-model; wherein, the first sub-model is used to predict the location information of the suspected abnormal event at the next moment based on the historical location information and the current location information of the suspected abnormal event; the second sub-model is used to estimate the error information of the location information of the suspected abnormal event at the next moment based on the historical location information and the current location information of the suspected abnormal event; using the first sub-model and the second sub-model to perform tracking analysis on the suspected abnormal event until the suspected abnormal event disappears.

[0108] Specifically, image feature extraction and clustering analysis provide an understanding of abnormal events in the visual and temporal dimensions, helping to identify the behavior patterns and development trends of events. The tracking model provides predictions and error evaluations of the spatial location changes of abnormal events, increasing the quantitative analysis of the development possibilities and uncertainties of events.

[0109] The results of image feature extraction and clustering analysis can be used as the input of the tracking model to help the model better understand the historical and current states of abnormal events, so as to make more accurate predictions. At the same time, the prediction results of the tracking model can also be fed back to the image analysis process to adjust and optimize the parameters of image feature extraction and clustering analysis, forming an iterative analysis process.

[0110] Further, performing tracking analysis on the suspected abnormal event includes:

[0111] S41. Obtain the image data of the suspected abnormal event at multiple time points;

[0112] S42. Calculate the motion vector of the suspected abnormal event based on the image data of the suspected abnormal event at two adjacent time points;

[0113] S43. Predict the image data of the suspected abnormal event at subsequent time points based on the motion vector to generate a predicted trajectory map;

[0114] S44. If the error between the predicted trajectory map and the actual trajectory map is less than the threshold, it is considered that the suspected abnormal event is real, and continue to perform tracking analysis;

[0115] S45. End the tracking analysis until the error between the predicted trajectory map and the actual trajectory map is greater than the threshold.

[0116] Specifically, first, image data of suspected abnormal events are obtained from multiple time points. Using this data, we calculate the motion vectors of the suspected abnormal events between adjacent time points, which reflect the moving direction and speed of the events in the river channel. Subsequently, based on these motion vectors, we predict the positions of the events at future time points and generate a predicted trajectory map. This process involves comparing with the actually observed trajectory map. If the error between the two is less than the threshold we set, we will continue the tracking analysis and consider that the event is real and continuous.

[0117] Furthermore, the step of identifying the characteristic trajectory of the event and obtaining the event attributes is to perform multi-frame continuous tracking on the suspected abnormal events of the same category according to the time stamp and spatial coordinates during the tracking analysis process, form a trajectory sequence, extract the feature vectors of the abnormal events, form the characteristic trajectory of the abnormal events, and obtain the attributes of the abnormal events according to the characteristic trajectory.

[0118] Among them, the characteristic trajectory of the abnormal event contains the spatio-temporal characteristics and visual characteristics of the event.

[0119] It should be explained that the method of first extracting and clustering image features through CNN and K-means, and obtaining the characteristic trajectory of abnormal events through sequence analysis; then analyzing abnormal situations by establishing a tracking model, including prediction and error estimation; finally, determining the authenticity of the event through the calculation of motion vectors and trajectory prediction, and extracting feature vectors from it to obtain event attributes, deeply analyzes abnormal events from different angles to ensure a comprehensive understanding and accurate identification of the events.

[0120] During the tracking analysis process, we first perform multi-frame continuous tracking on the suspected abnormal events of the same category according to the time stamp and spatial coordinates to form a trajectory sequence. This step involves precise analysis of the image data to ensure that we can capture the continuous changes of the event over time. Then, key spatio-temporal characteristics and visual characteristics are extracted from the trajectory sequence. These characteristics, as the attributes of the event, help to understand the essence of the event more deeply.

[0121] Specifically, the spatio-temporal characteristics include the type of the event, moving speed, direction change, duration, occurrence frequency, etc., while the visual characteristics may cover visual patterns such as the color, shape, texture of the event. Through these characteristics, the characteristic trajectory of the abnormal event can be constructed, which not only reflects the moving path of the event in the physical space but also reveals its evolution law over time.

[0122] Furthermore, after obtaining the attributes of the suspected abnormal events, it also includes: S4. Analyzing and judging the event attributes to obtain the recognition result of the abnormal event.

[0123] The specific abnormal event recognition results include at least one or more of events such as sewage discharge, garbage dumping, swimming in non-swimming river sections, and people falling into the water.

[0124] Embodiment 2

[0125] This embodiment is a river channel abnormal event recognition system based on UAV inspection, specifically applied to a river channel abnormal event recognition method based on UAV inspection, including:

[0126] Data acquisition module: The UAV conducts river channel inspection according to a predetermined route, uses a camera to capture river channel images, receives abnormal events reported by users in real time, and adjusts the route for targeted shooting;

[0127] Image preprocessing module: Denoise the collected image data, smooth the image using a Gaussian filter, perform threshold segmentation using the Otsu algorithm to distinguish the foreground and background in the image, and apply morphological operations for erosion and dilation processing;

[0128] Abnormal detection and tracking analysis module: Perform edge detection and target localization, determine the position coordinates of the object to be detected, compare the suspected abnormal events with preset rules, screen out the targets that need to be tracked and analyzed, perform multi-dimensional tracking analysis on the targets, identify the movement trajectory and behavior state, and construct a behavior trajectory model;

[0129] Event recognition and decision-making module: Use CNN and K-means algorithms to extract image features and perform clustering to form feature trajectories, establish a tracking model to predict and evaluate the error of the spatial position change of the suspected abnormal events, perform positioning and tracking on the suspected abnormal events, and analyze and judge the attributes and authenticity of the abnormal events according to the feature trajectories and behavior rules.

[0130] Embodiment 3

[0131] The difference between this embodiment and Embodiment 1 is that it uses a deep learning denoising autoencoder and an image processing method of multi-scale analysis to denoise the river channel image data; by introducing a deep learning denoising autoencoder, the noise in the river channel image can be effectively removed, especially under complex lighting or weather conditions, improving the clarity of the image. Then, through multi-scale analysis, the subtle river channel features in the image are further enhanced, making the inspection and monitoring more accurate. The combination of these two methods provides higher-quality image data for UAV river channel inspection, thereby improving the accuracy of abnormal event recognition and analysis.

[0132] The autoencoder is a neural network structure for unsupervised learning, mainly used for dimensionality reduction and data denoising. The specific content includes:

[0133] Data preprocessing: Standardize the captured river channel image data and divide it into a training set and a test set;

[0134] Construct a denoising autoencoder model: Construct a deep neural network consisting of several convolutional layers (for feature extraction) and fully connected layers (for feature compression); During training, add random noise (e.g., random Gaussian noise) to the input image, and through the autoencoder network, the model learns to recover the noise-free image from the noisy input;

[0135] Model training: Use the mean squared error (MSE) as the loss function, optimize the model parameters, and perform iterative training through backpropagation and gradient descent until the model can better remove noise;

[0136] Image denoising: Input the river channel image data taken under complex lighting or weather conditions during actual inspections into the trained autoencoder model to generate high-quality noise-free image data.

[0137] Then, use multi-scale analysis for image enhancement. Multi-scale analysis extracts the subtle features of the image by decomposing the image at different scales. This method is particularly suitable for capturing the changes in details in river channel images, such as the twists and turns of the riverbank line, vegetation cover, and water flow characteristics.

[0138] Specifically, it includes:

[0139] Image decomposition: Adopt multi-scale decomposition methods such as wavelet transform or Laplacian pyramid to decompose the original image into sub-images at different scales, and each sub-image contains the feature information at a specific scale;

[0140] Feature extraction and enhancement: For each sub-image at each scale, apply techniques such as contrast enhancement and edge sharpening to enhance the subtle features, and recombine the enhanced sub-images to form an enhanced high-quality image;

[0141] Image reconstruction: Reconstruct the enhancement results at all scales to generate the final image. This image is optimized at different scales and can more clearly display the detailed features of the river channel.

[0142] The above description is only a preferred embodiment of the present invention and does not impose any form of limitation on the present invention. Although the present invention has been disclosed as above with a preferred embodiment, it is not intended to limit the present invention. Any person skilled in the art can make some changes or modifications to the equivalent embodiments by using the above-disclosed technical content within the scope of the technical solution of the present invention. However, as long as it does not depart from the content of the technical solution of the present invention, any brief modifications, equivalent changes, and modifications made to the above embodiments based on the technical essence of the present invention still fall within the scope of the technical solution of the present invention.

Claims

1. A method for identifying abnormal events in a river channel based on drone patrol, characterized in that: It includes the following steps: Regularly conduct directional inspections of the river channel by drones, capture image data of the river channel, and perform noise reduction processing on the river channel image data; Identify the corresponding river channel information and suspected abnormal events in the processed image data; Conduct multi-dimensional tracking and analysis on the suspected abnormal events, identify the characteristic trajectories of the events, and obtain the event attributes; Among them, conducting multi-dimensional tracking and analysis on the suspected abnormal events includes: obtaining the target objects at the same position in the image data captured in multiple time periods; identifying the movement trajectories of the target objects according to the position change situations of the target objects; then identifying the behavior states of the target objects according to the movement trajectories to obtain the behavior trajectories of the target objects; identifying the activity rules of the target objects according to the behavior trajectories of the target objects; and identifying whether the target objects are abnormal events according to the activity rules of the target objects; Identifying whether the target objects are abnormal events according to the activity rules of the target objects further includes the steps of: S1. Use the convolutional neural network CNN to extract the image features of the target objects, and use the clustering algorithm K-means to classify similar images to form different types of image clusters; S2. Obtain the timestamps of each image in each image cluster, sort them in chronological order, and select M image clusters within N consecutive frames as samples; S3. Randomly select P pictures from each sample to form a subset; S4. For each group of subsets, calculate the Euclidean distance between two adjacent pictures, and classify the pictures corresponding to the distances less than the threshold into the same sequence; S5. Repeat steps S1 - S4 until the sequence length reaches the set maximum length Q, and then count the frequencies of each sequence occurrence to obtain the characteristic trajectories of abnormal events; When conducting multi-dimensional tracking and analysis on the suspected abnormal events, it further includes: establishing a tracking model, where the tracking model includes a first sub-model and a second sub-model; among them, the first sub-model is used to predict the position information of the suspected abnormal event at the next moment according to the historical position information and the current position information of the suspected abnormal event; the second sub-model is used to estimate the error information of the position information of the suspected abnormal event at the next moment according to the historical position information and the current position information of the suspected abnormal event; and use the first sub-model and the second sub-model to conduct tracking and analysis on the suspected abnormal event until the suspected abnormal event disappears.

2. The method for identifying abnormal events in a river based on drone inspection according to claim 1, wherein: The drones regularly conduct directional inspections of the river channel and capture image data of the river channel, including: conducting timed cruise scans of the river channel according to a pre-set route to obtain image data of each point on the route; when receiving an abnormal event reported by a user, adjusting the current heading to near the abnormal event reported by the user and capturing image data near the abnormal event; Performing noise reduction processing on the image data includes: Using a Gaussian filter to perform smoothing processing on the image data; Using the threshold segmentation method to perform binarization processing on the image data; Using morphological operations to perform erosion and dilation processing on the image data.

3. The method for identifying abnormal events in a river based on drone patrol according to claim 2, wherein: The threshold segmentation method divides the grayscale values of an image into an object and a background through the Otsu algorithm, including: Calculating the histogram of the image, that is, the frequency of each grayscale level; Dividing each element of the histogram by the total number of pixels in the image to obtain a normalized histogram; Calculate the cumulative probability distribution of each gray level according to the normalized histogram, expressed as , where i is the gray level; For each gray level of the threshold t, calculate the between-class variance of the foreground and the background, where the probability of the foreground and the probability of the background are expressed as: ； ； Among them, is the sum of the probabilities of all gray levels below the threshold t, is the sum of the probabilities of all gray levels at t and above; Calculate the means of the foreground and the background according to the threshold t and , expressed as: ; ; wherein, is the weighted average gray value of gray levels below the threshold t, is the weighted average gray value of gray levels above t; Calculate the between-class variance corresponding to each threshold t , and find the maximum value. The between-class variance is expressed as: , by determining the t value that makes maximum for segmenting the image into foreground and background.

4. The method for identifying abnormal events in a river channel based on drone inspection according to claim 1, wherein: Before the corresponding river channel information and suspected abnormal events in the processed image data of the recognition, including: Performing edge detection on the original image data taken; Performing target localization on the image data after edge detection to obtain the position coordinates of the object to be detected; Determining whether the object to be detected is located within a preset river channel area; if so, regarding the object to be detected as a suspected abnormal event.

5. The method for identifying abnormal events in a river channel based on drone patrol according to claim 4, wherein: Comparing the suspected abnormal event with a preset rule, and when the suspected abnormal event meets the preset rule, determining it as a target object that requires multi-dimensional tracking analysis; wherein, the preset rule is used to indicate at least one of the following: abnormal event type, abnormal event occurrence time, abnormal event occurrence location.

6. The method for identifying abnormal events in a river channel based on drone patrol according to claim 1, wherein: Performing tracking analysis on the suspected abnormal event, including: Obtaining image data of the suspected abnormal event at multiple time points; Calculating the motion vector of the suspected abnormal event according to the image data of the suspected abnormal event at two adjacent time points; Predicting the image data of the suspected abnormal event at subsequent time points according to the motion vector to generate a predicted trajectory map; If the error between the predicted trajectory map and the actual trajectory map is less than the threshold, it is considered that the suspected abnormal event is real, and continue to perform tracking analysis; Until the situation where the error between the predicted trajectory map and the actual trajectory map is greater than the threshold appears, end the tracking analysis.

7. The method for identifying abnormal events in a river channel based on drone patrol according to claim 1, characterized in that: Identifying the characteristic trajectory of the event and obtaining the event attribute is, during the tracking analysis process, performing multi-frame continuous tracking on the suspected abnormal events of the same category according to the time stamp and spatial coordinates to form a trajectory sequence, extracting the feature vector of the abnormal event to form the characteristic trajectory of the abnormal event, and obtaining the attribute of the abnormal event according to the characteristic trajectory.

8. The method for identifying abnormal events in a river channel based on UAV inspection according to claim 7, wherein: After obtaining the attribute of the suspected abnormal event, it further includes: analyzing and judging the event attribute to obtain the abnormal event recognition result.

9. A river channel anomaly event recognition system based on UAV inspection, which is applied to the river channel anomaly event recognition method based on UAV inspection as described in any one of claims 1-8, and is characterized in that: Including a data acquisition module, an image preprocessing module, an abnormal detection and tracking analysis module, and an event recognition and decision-making module, The data acquisition module, the drone conducts river channel inspections according to a predetermined route, uses a camera to take pictures of the river channel, receives abnormal events reported by users in real time, and adjusts the route for targeted shooting; The image preprocessing module performs noise reduction processing on the collected image data, smooths the image using a Gaussian filter, performs threshold segmentation using the Otsu algorithm to distinguish the foreground and background in the image, and applies morphological operations for erosion and dilation processing; The abnormal detection and tracking analysis module performs edge detection and target localization, determines the position coordinates of the object to be detected, compares the suspected abnormal event with a preset rule, screens out the targets that require tracking analysis, performs multi-dimensional tracking analysis on the targets, identifies the motion trajectory and behavior state, and constructs a behavior trajectory model; The event recognition and decision-making module uses CNN and K-means algorithms to extract image features and perform clustering, forming feature trajectories, establishing a tracking model to predict and evaluate the error of the spatial position change of suspected abnormal events, performing positioning and tracking on suspected abnormal events, and analyzing and judging the attributes and authenticity of abnormal events according to the feature trajectories and behavior rules.

Images