Unmanned aerial vehicle patrol-based river channel abnormal event identification method and system

Abstract

The invention discloses a river channel abnormal event identification method and system based on unmanned aerial vehicle patrol, and the method comprises the steps: carrying out the regular and directional patrol of a river channel through an unmanned aerial vehicle, shooting river channel image data, and carrying out the noise reduction of the river channel image data; identifying corresponding river channel information and suspected abnormal events in the processed image data; performing multi-dimensional tracking analysis on the suspected abnormal event, identifying an event feature trajectory, and obtaining an event attribute; the event attributes are analyzed and judged, and an abnormal event recognition result is obtained; by adopting a multi-dimensional tracking analysis means, refined real-time monitoring and tracking of a river channel abnormal event are realized, and by obtaining target object images at the same position at different time points, the position change condition of a target object is deeply analyzed, the movement track of the target object is accurately identified, and the behavior state and the activity rule of the target object are intelligently identified, so that the accuracy of the target object is improved. The accuracy of abnormal event recognition is improved, and the problem that real-time monitoring and continuous tracking of abnormal events cannot be achieved through manual patrol is solved.

Description

Technical Field

[0001] The present invention relates to the field of intelligent analysis technology, and in particular to a method and system for identifying abnormal events in a river channel based on drone inspection. Background Art

[0002] In the traditional river inspection process, manual inspection is not only inefficient, but also easily affected by weather and lighting conditions, resulting in insufficient accuracy and consistency of inspection results. Manual inspection usually requires a lot of manpower and time to cover the entire river area, which not only increases costs, but may also miss abnormal situations due to negligence or fatigue of inspectors. At the same time, due to the limitation of inspection frequency, manual inspection is difficult to achieve real-time monitoring and continuous tracking of abnormal events, resulting in some potential environmental problems not being discovered and handled in time. For example, water pollution, illegal discharges, river blockages and other problems may deteriorate rapidly if they are not discovered in time, posing a serious threat to the ecological environment and public health. Summary of the invention

[0003] The purpose of the present invention is to provide a method and system for identifying abnormal events in a river channel based on drone inspection, so as to solve the problem of missing abnormal situations due to negligence or fatigue of inspectors and improve the ability to identify abnormal events in a river channel.

[0004] The purpose of the present invention can be achieved through the following technical solutions: This application provides a method for identifying abnormal river events based on drone inspection, including the following steps: Regularly inspect the river channel with a drone, capture river channel image data, and perform noise reduction on the river channel image data; Identify the corresponding river channel information and suspected abnormal events in the processed image data; Conduct multi-dimensional tracking and analysis on the suspected abnormal events, identify event feature trajectories, and obtain event attributes; The multi-dimensional tracking and analysis of suspected abnormal events includes: obtaining target objects at the same position in the image data taken in multiple time periods; identifying the motion trajectory of the target object according to the position change of the target object; then identifying the behavior state of the target object according to the motion trajectory to obtain the behavior trajectory of the target object; identifying the activity pattern of the target object according to the behavior trajectory of the target object; and identifying whether the target object is an abnormal event according to the activity pattern of the target object; Identifying whether the target object is an abnormal event according to the activity pattern of the target object also includes: extracting image features of the target object using a convolutional neural network (CNN), and classifying similar images using a clustering algorithm (K-means) to form image clusters of different types; Get the timestamp of each image in each image cluster, sort them in chronological order, and select M image clusters within N consecutive frames as samples; Randomly select P images from each sample to form a subset; For each subset, calculate the Euclidean distance between two adjacent images, and classify the images corresponding to the distance less than the threshold into the same sequence; The above process is repeated until the sequence length reaches the set maximum length Q, and then the frequency of occurrence of each sequence is counted to obtain the characteristic trajectory of the abnormal event.

[0005] Furthermore, the drone regularly patrols the river in a directional manner and captures river image data, including: performing a scheduled cruise scanning and capturing of the river according to a pre-set route, and acquiring image data of each point on the route; when receiving an abnormal event reported by a user, adjusting the current heading to the vicinity of the abnormal event reported by the user, and capturing image data near the abnormal event; Performing noise reduction processing on the image data, comprising: Using a Gaussian filter to smooth the image data; Using a threshold segmentation method to perform binarization processing on the image data; The image data is subjected to erosion and dilation processing using morphological operations.

[0006] Furthermore, the threshold segmentation method is to divide the gray value of the image into two parts, object and background, by Otsu algorithm, including: Calculate the histogram of the image, that is, the frequency of each gray level; Divide each element of the histogram by the total number of pixels in the image to obtain a normalized histogram; The cumulative probability distribution of each gray level is calculated according to the normalized histogram, which is expressed as , where i is the gray level; For each gray level threshold t, the inter-class variance of foreground and background is calculated, where the probability of foreground and background probability It is expressed as: ; in, is the sum of the probabilities of all gray levels below the threshold t, is the sum of the probabilities of all gray levels t and above; According to the threshold t, calculate the mean of the foreground and background and , expressed as: ; ;in, is the weighted average gray value of gray levels below threshold t, is the weighted average gray value of gray levels above t; Calculate the inter-class variance corresponding to each threshold t , and find the maximum value, the between-class variance is expressed as: , by determining The largest t value is used to segment the image into foreground and background.

[0007] Furthermore, before the corresponding river channel information and suspected abnormal events in the image data after the recognition processing are obtained, the method includes: Perform edge detection on the captured original image data; Perform target positioning on the image data after edge detection to obtain the position coordinates of the object to be detected; Determine whether the object to be detected is located in the preset river channel area; if so, treat the object to be detected as a suspected abnormal event.

[0008] Furthermore, the suspected abnormal event is compared with preset rules. When the suspected abnormal event meets the preset rules, it is determined as a target object that requires multi-dimensional tracking and analysis; wherein the preset rules are used to indicate at least one of the following: the type of abnormal event, the time when the abnormal event occurred, and the location where the abnormal event occurred.

[0009] Furthermore, when performing multi-dimensional tracking and analysis on suspected abnormal events, it also includes: establishing a tracking model, the tracking model including a first sub-model and a second sub-model; wherein the first sub-model is used to predict the location information of the suspected abnormal event at the next moment based on the historical location information and the current location information of the suspected abnormal event; the second sub-model is used to estimate the error information of the location information of the suspected abnormal event at the next moment based on the historical location information and the current location information of the suspected abnormal event; the first sub-model and the second sub-model are used to track and analyze the suspected abnormal event until the suspected abnormal event disappears.

[0010] Furthermore, the suspected abnormal event is tracked and analyzed, including: Acquire image data of suspected abnormal events at multiple time points; Calculate the motion vector of the suspected abnormal event based on the image data of the suspected abnormal event at two adjacent time points; According to the motion vector, the image data of the suspected abnormal event at the subsequent time point is predicted to generate a predicted trajectory map; If the error between the predicted trajectory map and the actual trajectory map is less than the threshold, the suspected abnormal event is considered to be true and the tracking analysis continues; The tracking analysis ends when the error between the predicted trajectory map and the actual trajectory map is greater than a threshold.

[0011] Furthermore, the identification of event feature trajectories and the acquisition of event attributes are performed during the tracking and analysis process by continuously tracking multiple frames of suspected abnormal events of the same category according to timestamps and spatial coordinates to form a trajectory sequence, extracting feature vectors of the abnormal events, forming feature trajectories of the abnormal events, and obtaining the attributes of the abnormal events based on the feature trajectories.

[0012] Furthermore, after obtaining the attributes of the suspected abnormal event, the method further includes: analyzing and judging the event attributes to obtain an abnormal event identification result.

[0013] The river abnormal event recognition system based on drone inspection is specifically applied to the river abnormal event recognition method based on drone inspection, including data acquisition module, image preprocessing module, abnormal detection and tracking analysis module and event recognition and decision module. In the data collection module, the drone patrols the river according to the scheduled route, uses the camera to capture images of the river, receives abnormal events reported by users in real time, and adjusts the route for targeted shooting; The image preprocessing module performs noise reduction on the collected image data, uses a Gaussian filter to smooth the image, uses the Otsu algorithm to perform threshold segmentation, distinguishes the foreground and background in the image, and applies morphological operations to perform corrosion and expansion processing; The anomaly detection and tracking analysis module performs edge detection and target positioning, determines the position coordinates of the object to be detected, compares suspected abnormal events with preset rules, screens out the targets that need to be tracked and analyzed, performs multi-dimensional tracking analysis on the targets, identifies the motion trajectory and behavior status, and builds a behavior trajectory model; The event recognition and decision-making module uses CNN and K-means algorithms to extract image features and cluster them to form feature trajectories. It establishes a tracking model to predict the spatial position changes and error assessment of suspected abnormal events, locates and tracks suspected abnormal events, and analyzes and determines the attributes and authenticity of abnormal events based on feature trajectories and behavior patterns.

[0014] The beneficial effects of the present invention are: By adopting multi-dimensional tracking and analysis methods, we can achieve refined real-time monitoring and tracking of abnormal events in the river. By acquiring images of target objects at the same location at different time points, we can deeply analyze the position changes of the target objects, accurately identify their movement trajectories, and then intelligently identify the behavior status and activity patterns of the target objects based on these movement trajectories. This multi-dimensional analysis not only greatly improves the accuracy of abnormal event identification, but also provides accurate tracking and analysis of abnormal situations through integrated prediction models and error estimation models until the abnormal events are properly handled or disappear. This achieves refined real-time monitoring and continuous tracking of abnormal events in the river, solves the problem that manual inspections cannot achieve real-time monitoring and continuous tracking of abnormal events, improves the accuracy of abnormal event identification, and reduces misjudgments and missed judgments caused by human factors. By using convolutional neural network (CNN) and K-means clustering algorithm, the present invention has made breakthrough progress in image feature extraction and abnormal event recognition. CNN effectively captures local features of images through its deep structure and builds more complex and abstract feature representation layer by layer, which greatly enhances the recognition ability of the system. K-means clustering algorithm effectively classifies these features in feature space to form image clusters with the same or similar attributes. This process not only improves the accuracy of abnormal event recognition, but also can quickly respond to and accurately locate abnormal behaviors by analyzing the behavior patterns in continuous image frames, providing strong technical support for the timely handling of abnormal events in rivers. Through regular directional river inspections conducted by drones, the present invention realizes the automatic collection of river image data, greatly improving the efficiency and data quality of river inspections. Drone inspections avoid interference from human factors and ensure the consistency and reliability of data collection. Combined with image noise reduction processing technology, the system further purifies the image data and provides high-quality input for subsequent feature extraction and anomaly analysis. The present invention not only reduces the manpower burden, but also provides an efficient and reliable solution for real-time monitoring and abnormal response of rivers through refined management, solving the problems of low efficiency of manual inspections and unstable data quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] For better understanding and implementation, the technical solution of the present application is described in detail below with reference to the accompanying drawings.

[0016] Figure 1 A schematic diagram of the flow of a method for identifying abnormal river events based on drone inspections provided in Example 1 of the present application; Figure 2 A schematic diagram of a process for performing noise reduction processing on image data using a method for identifying abnormal river events based on drone inspections provided in Example 1 of the present application; Figure 3A schematic diagram of a process for dividing the grayscale value of an image into an object and a background in a method for identifying abnormal river events based on drone inspection provided in Example 1 of the present application; Figure 4 A schematic diagram of the process before the river channel information and suspected abnormal events are identified in the image data of the river channel abnormal event identification method based on drone inspection provided in Example 1 of the present application; Figure 5 A schematic diagram of a process for identifying whether a target object is an abnormal event by a method for identifying abnormal events in a river channel based on drone inspection provided in Example 1 of the present application; Figure 6 A schematic diagram of the process of tracking and analyzing suspected abnormal events in the river channel abnormal event identification method based on drone inspection provided in Example 1 of the present application. DETAILED DESCRIPTION

[0017] In order to further explain the technical means and effects taken by the present invention to achieve the predetermined invention purpose, exemplary embodiments will be described in detail here, and examples thereof are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are only examples of methods and systems consistent with some aspects of the present application as detailed in the attached claims.

[0018] The terms used in this application are for the purpose of describing specific embodiments only and are not intended to limit this application. The singular forms of "a", "said" and "the" used in this application and the appended claims are also intended to include plural forms unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0019] The specific implementation methods, features and effects of the present invention are described in detail below in conjunction with the accompanying drawings and preferred embodiments.

[0020] Example 1 See also Figure 1-Figure 6 ,This embodiment provides a method and system for identifying abnormal events in a river channel based on drone inspection, which solves the problem of missing abnormal situations due to the negligence or fatigue of the inspectors, and improves the ability to identify abnormal events in the river channel.

[0021] The present invention provides a method for identifying abnormal events in a river channel based on drone inspection, comprising the following steps: S1. Regularly inspect the river channel with a drone, capture river channel image data, and perform noise reduction on the river channel image data; S2, identifying the corresponding river channel information and suspected abnormal events in the processed image data; S3, performing multi-dimensional tracking analysis on the suspected abnormal event, identifying the event feature trajectory, and obtaining event attributes; Furthermore, the drone regularly patrols the river channel in a directional manner and captures image data of the river channel, including: performing scheduled cruise scanning and capturing of the river channel according to a pre-set route, and acquiring image data of each point on the route; when receiving an abnormal event reported by a user, adjusting the current heading to the vicinity of the abnormal event reported by the user, and capturing image data near the abnormal event.

[0022] Further, the image data is subjected to noise reduction processing, including: S11, smoothing the image data using a Gaussian filter; specifically, for each pixel in the image, replacing the pixel value with a weighted average value in a neighborhood; S12, performing binarization processing on the image data by using a threshold segmentation method; The threshold segmentation method is to divide the gray value of the image into two parts: object and background through the Otsu algorithm, including: S111, calculating the histogram of the image, that is, the frequency of occurrence of each gray level; S112, dividing each element of the histogram by the total number of pixels of the image to obtain a normalized histogram; S113, calculate the cumulative probability distribution of each gray level according to the normalized histogram, expressed as , where i is the gray level; S114, for each gray level of threshold t, calculate the inter-class variance of foreground and background, where the probability of foreground and background probability It is expressed as: ; in, is the sum of the probabilities of all gray levels below the threshold t, is the sum of the probabilities of all gray levels t and above; S115. Calculate the mean of foreground and background according to threshold t and , expressed as: ; ;in, is the weighted average gray value of gray levels below threshold t, is the weighted average gray value of gray levels above t; S116. Calculate the inter-class variance corresponding to each threshold t , and find The maximum value, the between-class variance is expressed as: , by determining The largest t value is used to segment the image into foreground and background.

[0023] S13, using morphological operations to perform corrosion and dilation processing on the image data.

[0024] Among them, erosion is to convolve the image with the structural element, replacing the value of each pixel with the minimum value of the corresponding position of the structural element in its neighborhood, and dilation is to replace the value of each pixel with the maximum value of the corresponding position of the structural element in its neighborhood.

[0025] It should be explained that the drone conducts scheduled cruise scanning and photography of the river according to the preset route to ensure that the river image data can be systematically collected. When receiving abnormal events reported by users, the drone can flexibly adjust its course and quickly collect images of the abnormal area. The collected images will then undergo noise reduction processing and use a Gaussian filter to smooth the images to reduce image noise and retain important information. Next, the images are threshold segmented using the Otsu algorithm, which automatically calculates the optimal threshold and divides the grayscale value of the image into foreground (object) and background, thereby clearly distinguishing abnormal objects in the river. Finally, the image is corroded and expanded through morphological operations to further remove noise and details in the image, emphasize the structural features of the image, and provide clear image data for feature extraction and subsequent analysis of abnormal events. This process improves the accuracy and efficiency of abnormal event detection in rivers through automated and intelligent image processing technology.

[0026] Furthermore, before the corresponding river channel information and suspected abnormal events in the image data after the recognition processing are obtained, the method includes: S21, performing edge detection on the captured original image data; using an edge detection algorithm to process the captured original image, and extracting the edge of the river channel and its surrounding area in the image; S22, performing target positioning on the image data after edge detection to obtain the position coordinates of the object to be detected; specifically, using a target detection algorithm to locate the position of the object to be detected to obtain its coordinates; S23, determine whether the object to be detected is located in the preset river channel area; if so, treat the object to be detected as a suspected abnormal event. Compare the position coordinates of the target object with the preset river channel area to determine whether it is located within the river channel range. If the target object is located in the preset river channel area, mark it as a suspected abnormal event for further analysis.

[0027] For example, the image captured by the drone shows an unidentified floating object somewhere in the river. Through edge detection and target positioning, the coordinates of the floating object are determined to be (X1, Y1). The preset river area is defined as a rectangular area through GIS data, and its lower left corner coordinates are (Xmin, Ymin) and the upper right corner coordinates are (Xmax, Ymax). If Xmin≤X1≤Xmax and Ymin≤Y1≤Ymax, the floating object is located in the river area and is marked as a suspected abnormal event.

[0028] Furthermore, the suspected abnormal event is compared with preset rules. When the suspected abnormal event meets the preset rules, it is determined as a target object that requires multi-dimensional tracking and analysis; wherein the preset rules are used to indicate at least one of the following: the type of abnormal event, the time when the abnormal event occurred, and the location where the abnormal event occurred.

[0029] Specific abnormal event types include: Sewage discharge: Identify unknown liquid discharges appearing in images, especially those with colors that are significantly different from the surrounding water. Garbage dumping: Detect garbage piles or abnormal objects in rivers, such as plastic bags and bottles. Illegal swimming: Identify swimmers in non-swimming areas. People falling into water: Detect whether people or animals accidentally fall into the water in the river.

[0030] Abnormal events occur when: Specific time periods: Incidents that occur during certain time periods may be of greater concern, such as illegal dumping at night; Locations where unusual events occurred include: Within the protected area: any abnormal incident occurring in the ecological protection area or other sensitive areas of the river; historical problem area: incidents occurring again at locations where multiple abnormal incidents have occurred in the past; monitoring blind spot: incidents occurring at the edge or blind spot of river monitoring.

[0031] Furthermore, the multi-dimensional tracking and analysis of the suspected abnormal event includes: obtaining the target object at the same position in the image data taken in multiple time periods; identifying the movement trajectory of the target object according to the position change of the target object; Then, the behavior state of the target object is identified according to the motion trajectory to obtain the behavior trajectory of the target object; the activity pattern of the target object is identified according to the behavior trajectory of the target object; and whether the target object is an abnormal event is identified according to the activity pattern of the target object.

[0032] It needs to be explained that: when the river management department monitors unidentified objects in non-swimming sections of the river through the drone patrol system and preliminarily judges that they may be illegally dumped garbage, the multi-dimensional tracking and analysis process includes: analyzing the position changes of objects in time series images, image matching technology can be used to determine whether the object is moving, and identify the motion trajectory, and analyze its behavior state according to the object's motion trajectory, such as the movement speed and direction of floating objects; then, combining the object's movement speed, direction and duration, construct a behavior trajectory model, and through statistical analysis of the behavior trajectory, identify the object's activity pattern, such as whether it appears at the same time every day, whether it moves with the water flow, etc. If the object's activity pattern is inconsistent with the behavior of normal objects in the river, it means that an object should move with the water flow but does not move, or continues to appear in places where it should not appear, which may indicate that it was placed or dumped by humans; if it continues to appear in a non-swimming area and does not move with the water flow, it may be judged as suspicious behavior, which means that if an object repeatedly appears in an area where swimming is prohibited, and there are usually no such objects in this area, this may indicate that illegal behavior has occurred, such as illegal dumping of garbage.

[0033] In a river, most objects will move with the direction and speed of the water flow. If an object remains in the same position for a period of time and does not drift with the water flow, this may indicate that the object is fixed to the riverbed or abandoned there. The management department can use the data collected by the drone patrol system to analyze the movement trajectory and activity patterns of the object to identify and determine whether suspicious behavior has occurred.

[0034] Further, identifying whether the target object is an abnormal event according to the activity pattern of the target object further includes: S31, using convolutional neural network CNN to extract the image features of the target object, and using clustering algorithm K-means to classify similar images to form different types of image clusters; S32, obtaining the timestamp of each image in each image cluster, sorting them in chronological order, and selecting M image clusters within N consecutive frames as samples; S33, randomly select P pictures from each sample to form a subset; S34, for each subset, calculating the Euclidean distance between two adjacent images, and classifying the images corresponding to the distances less than a threshold into the same sequence; S35, repeat the above process until the sequence length reaches the set maximum length Q; finally, count the frequency of occurrence of each sequence to obtain the characteristic trajectory of the abnormal event.

[0035] Furthermore, when performing multi-dimensional tracking and analysis on suspected abnormal events, it also includes: establishing a tracking model, the tracking model including a first sub-model and a second sub-model; wherein the first sub-model is used to predict the location information of the suspected abnormal event at the next moment based on the historical location information and the current location information of the suspected abnormal event; the second sub-model is used to estimate the error information of the location information of the suspected abnormal event at the next moment based on the historical location information and the current location information of the suspected abnormal event; the first sub-model and the second sub-model are used to track and analyze the suspected abnormal event until the suspected abnormal event disappears.

[0036] Specifically, image feature extraction and cluster analysis provide an understanding of the visual and temporal dimensions of abnormal events, helping to identify the behavior patterns and development trends of events. The tracking model provides predictions and error assessments of the spatial location changes of abnormal events, increasing the quantitative analysis of the possibility and uncertainty of event development. The results of image feature extraction and cluster analysis can be used as input to the tracking model to help the model better understand the history and current status of abnormal events, thereby making more accurate predictions. At the same time, the prediction results of the tracking model can also be fed back to the image analysis process to adjust and optimize the parameters of image feature extraction and cluster analysis, forming an iterative analysis process.

[0037] Furthermore, the suspected abnormal event is tracked and analyzed, including: S41, acquiring image data of suspected abnormal events at multiple time points; S42, calculating a motion vector of the suspected abnormal event based on the image data of the suspected abnormal event at two adjacent time points; S43, predicting the image data of the suspected abnormal event at a subsequent time point according to the motion vector, and generating a predicted trajectory map; S44, if the error between the predicted trajectory map and the actual trajectory map is less than the threshold, the suspected abnormal event is considered to be true, and the tracking analysis is continued; S45, until the error between the predicted trajectory map and the actual trajectory map is greater than a threshold, the tracking analysis is terminated.

[0038] Specifically, we first obtain image data of suspected abnormal events from multiple time points. Using this data, we calculate the motion vectors of suspected abnormal events between adjacent time points, which reflects the direction and speed of the event in the river. Then, based on these motion vectors, we predict the location of the event at future time points and generate a predicted trajectory map; this process involves comparing it with the actual observed trajectory map. If the error between the two is less than the threshold we set, we will continue tracking and analysis and believe that the event is real and ongoing.

[0039] Furthermore, the identification of event feature trajectories and the acquisition of event attributes are performed during the tracking and analysis process by continuously tracking multiple frames of suspected abnormal events of the same category according to timestamps and spatial coordinates to form a trajectory sequence, extracting feature vectors of the abnormal events, forming feature trajectories of the abnormal events, and obtaining the attributes of the abnormal events based on the feature trajectories.

[0040] Among them, the characteristic trajectory of abnormal events includes the spatiotemporal and visual characteristics of the events. What needs to be explained is that we first use CNN and K-means to extract and cluster image features, and use sequence analysis to obtain the characteristic trajectory of abnormal events; then we build a tracking model, including prediction and error estimation, to analyze the process of abnormal situations; finally, we determine the authenticity of the event through motion vector calculation and trajectory prediction, and extract feature vectors from them to obtain event attributes, so as to conduct in-depth analysis of abnormal events from different angles and ensure a comprehensive understanding and accurate identification of the events.

[0041] In the tracking analysis process, we first continuously track multiple frames of suspected abnormal events of the same category according to timestamps and spatial coordinates to form a trajectory sequence. This step involves precise analysis of image data to ensure that we can capture the continuous changes of events over time. Then, we extract key spatiotemporal and visual features from the trajectory sequence. These features serve as attributes of the event and help us understand the nature of the event more deeply.

[0042] Specifically, spatiotemporal features include the type of event, speed of movement, direction change, duration, and frequency of occurrence, while visual features may include visual patterns such as color, shape, texture, etc. Through these features, the characteristic trajectory of abnormal events can be constructed, which not only reflects the movement path of the event in physical space, but also reveals its evolution over time.

[0043] Furthermore, after obtaining the attributes of the suspected abnormal event, the method further includes: S4, analyzing and judging the event attributes to obtain an abnormal event identification result.

[0044] The specific abnormal event identification results include at least one or more of sewage discharge, garbage dumping, swimming in a river section not suitable for swimming, and people falling into water.

[0045] Example 2 This embodiment is a river abnormal event recognition system based on drone inspection, which is specifically applied to a river abnormal event recognition method based on drone inspection, including: In the data collection module, the drone patrols the river according to the scheduled route, uses the camera to capture images of the river, receives abnormal events reported by users in real time, and adjusts the route for targeted shooting; The image preprocessing module performs noise reduction on the collected image data, uses a Gaussian filter to smooth the image, uses the Otsu algorithm to perform threshold segmentation, distinguishes the foreground and background in the image, and applies morphological operations to perform corrosion and expansion processing; The anomaly detection and tracking analysis module performs edge detection and target positioning, determines the position coordinates of the object to be detected, compares suspected abnormal events with preset rules, screens out the targets that need to be tracked and analyzed, performs multi-dimensional tracking analysis on the targets, identifies the motion trajectory and behavior status, and builds a behavior trajectory model; The event recognition and decision-making module uses CNN and K-means algorithms to extract image features and cluster them to form feature trajectories. It establishes a tracking model to predict the spatial position changes and error assessment of suspected abnormal events, locates and tracks suspected abnormal events, and analyzes and determines the attributes and authenticity of abnormal events based on feature trajectories and behavior patterns.

[0046] Example 3 This embodiment differs from Embodiment 1 in that the river image data is denoised by introducing a deep learning denoising autoencoder and a multi-scale analysis image processing method. By introducing the deep learning denoising autoencoder, the noise in the river image can be effectively removed, especially under complex lighting or weather conditions, to improve the clarity of the image. The subtle river features in the image are further enhanced through multi-scale analysis, making inspections and monitoring more accurate. The combination of these two methods provides higher quality image data for UAV river inspections, thereby improving the accuracy of abnormal event recognition and analysis.

[0047] Autoencoder is a neural network structure for unsupervised learning, mainly used for dimensionality reduction and data denoising, including: Data preprocessing: standardize the captured river image data and divide them into training set and test set; Build a denoising autoencoder model: Build a deep neural network consisting of several convolutional layers (for feature extraction) and fully connected layers (for feature compression); during training, add random noise (for example, random Gaussian noise) to the input image. Through the autoencoder network, the model learns to recover a noise-free image from the noisy input; Model training: Use mean square error (MSE) as the loss function to optimize model parameters, and perform iterative training through back propagation and gradient descent until the model can effectively remove noise; Image denoising: The river image data taken under complex lighting or weather conditions during actual inspections are input into the trained autoencoder model to generate noise-free, high-quality image data.

[0048] Multi-scale analysis is then used for image enhancement. Multi-scale analysis extracts subtle features of the image by decomposing the image at different scales. This method is particularly suitable for capturing changes in details in river images, such as the tortuosity of the river bank, vegetation coverage, and water flow characteristics.

[0049] Specifically include: Image decomposition: Use multi-scale decomposition methods such as wavelet transform or Laplace pyramid to decompose the original image into sub-images of different scales, each of which contains feature information at a specific scale; Feature extraction and enhancement: For each scale sub-image, apply contrast enhancement, edge sharpening and other techniques to enhance subtle features, and reassemble the enhanced sub-images to form an enhanced high-quality image; Image reconstruction: The enhancement results at all scales are reconstructed to generate the final image, which is optimized at different scales and can show the detailed features of the river channel more clearly.

[0050] The above description is only a preferred embodiment of the present invention and does not limit the present invention in any form. Although the present invention has been disclosed as a preferred embodiment as above, it is not used to limit the present invention. Any technical personnel in this field can make some changes or modify the technical contents disclosed above into equivalent embodiments without departing from the scope of the technical solution of the present invention. However, any brief modifications, equivalent changes and modifications made to the above embodiments based on the technical essence of the present invention without departing from the content of the technical solution of the present invention are still within the scope of the technical solution of the present invention.

Claims

1. A method for identifying abnormal river events based on drone inspection, characterized by: The steps include: Regularly inspect the river channel with a drone, capture river channel image data, and perform noise reduction on the river channel image data; Identify the corresponding river channel information and suspected abnormal events in the processed image data; Conduct multi-dimensional tracking and analysis on the suspected abnormal events, identify event feature trajectories, and obtain event attributes; The multi-dimensional tracking and analysis of suspected abnormal events includes: obtaining target objects at the same position in the image data taken in multiple time periods; identifying the motion trajectory of the target object according to the position change of the target object; then identifying the behavior state of the target object according to the motion trajectory to obtain the behavior trajectory of the target object; identifying the activity pattern of the target object according to the behavior trajectory of the target object; and identifying whether the target object is an abnormal event according to the activity pattern of the target object; Identifying whether the target object is an abnormal event according to the activity pattern of the target object further includes: The convolutional neural network (CNN) is used to extract the image features of the target object, and the clustering algorithm K-means is used to classify similar images to form different types of image clusters. Get the timestamp of each image in each image cluster, sort them in chronological order, and select M image clusters within N consecutive frames as samples; Randomly select P images from each sample to form a subset; For each subset, calculate the Euclidean distance between two adjacent images, and classify the images with a distance less than the threshold into the same sequence; The above process is repeated until the sequence length reaches the set maximum length Q, and then the frequency of occurrence of each sequence is counted to obtain the characteristic trajectory of the abnormal event.

2. The method for identifying abnormal river events based on drone inspection according to claim 1 is characterized in that: The drone regularly patrols the river in a directional manner and captures river image data, including: performing a scheduled cruise scanning and capturing of the river according to a pre-set route, and acquiring image data of each point on the route; when receiving an abnormal event reported by a user, adjusting the current heading to the vicinity of the abnormal event reported by the user, and capturing image data near the abnormal event; Performing noise reduction processing on the image data, comprising: Using a Gaussian filter to smooth the image data; Using a threshold segmentation method to perform binarization processing on the image data; The image data is subjected to erosion and dilation processing using morphological operations.

3. The method for identifying abnormal river events based on drone inspection according to claim 2 is characterized in that: The threshold segmentation method is to divide the gray value of the image into two parts: object and background through the Otsu algorithm, including: Calculate the histogram of the image, that is, the frequency of each gray level; Divide each element of the histogram by the total number of pixels in the image to obtain a normalized histogram; The cumulative probability distribution of each gray level is calculated according to the normalized histogram, which is expressed as , where i is the gray level; For each gray level threshold t, the inter-class variance of foreground and background is calculated, where the probability of foreground and background probability It is expressed as: ； ； in, is the sum of the probabilities of all gray levels below the threshold t, is the sum of the probabilities of all gray levels t and above; According to the threshold t, calculate the mean of the foreground and background and , expressed as: ; ;in, is the weighted average gray value of gray levels below threshold t, is the weighted average gray value of gray levels above t; Calculate the inter-class variance corresponding to each threshold t , and find the maximum value, the between-class variance is expressed as: , by determining The largest t value is used to segment the image into foreground and background.

4. The method for identifying abnormal river events based on drone inspection according to claim 1 is characterized in that: Before the corresponding river channel information and suspected abnormal events in the image data after the recognition processing, it includes: Perform edge detection on the captured original image data; Perform target positioning on the image data after edge detection to obtain the position coordinates of the object to be detected; Determine whether the object to be detected is located in the preset river channel area; if so, treat the object to be detected as a suspected abnormal event.

5. The method for identifying abnormal river events based on drone inspection according to claim 4 is characterized in that: The suspected abnormal event is compared with the preset rules. When the suspected abnormal event meets the preset rules, it is determined as a target object that requires multi-dimensional tracking and analysis; wherein the preset rules are used to indicate at least one of the following: the type of abnormal event, the time when the abnormal event occurred, and the location where the abnormal event occurred.

6. The method for identifying abnormal river events based on drone inspection according to claim 1 is characterized in that: When performing multi-dimensional tracking and analysis on suspected abnormal events, it also includes: establishing a tracking model, the tracking model includes a first sub-model and a second sub-model; wherein the first sub-model is used to predict the location information of the suspected abnormal event at the next moment based on the historical location information and the current location information of the suspected abnormal event; the second sub-model is used to estimate the error information of the location information of the suspected abnormal event at the next moment based on the historical location information and the current location information of the suspected abnormal event; and the first sub-model and the second sub-model are used to track and analyze the suspected abnormal event until the suspected abnormal event disappears.

7. The method for identifying abnormal river events based on drone inspection according to claim 6 is characterized in that: Track and analyze the suspected abnormal events, including: Acquire image data of suspected abnormal events at multiple time points; Calculate the motion vector of the suspected abnormal event based on the image data of the suspected abnormal event at two adjacent time points; According to the motion vector, the image data of the suspected abnormal event at the subsequent time point is predicted to generate a predicted trajectory map; If the error between the predicted trajectory map and the actual trajectory map is less than the threshold, the suspected abnormal event is considered to be true and the tracking analysis continues; The tracking analysis ends when the error between the predicted trajectory map and the actual trajectory map is greater than a threshold.

8. The method for identifying abnormal river events based on drone inspection according to claim 1 is characterized in that: The method of identifying event feature trajectories and obtaining event attributes is to continuously track suspected abnormal events of the same category for multiple frames according to timestamps and spatial coordinates during the tracking and analysis process to form a trajectory sequence, extract feature vectors of the abnormal events, form feature trajectories of the abnormal events, and obtain the attributes of the abnormal events based on the feature trajectories.

9. The method for identifying abnormal river events based on drone inspection according to claim 8 is characterized in that: After obtaining the attributes of the suspected abnormal event, the method further includes: analyzing and judging the event attributes to obtain an abnormal event identification result.

10. A river abnormal event recognition system based on drone inspection, applied to a river abnormal event recognition method based on drone inspection as claimed in any one of claims 1 to 9, characterized in that: It includes data acquisition module, image preprocessing module, anomaly detection and tracking analysis module and event recognition and decision-making module. In the data acquisition module, the drone patrols the river according to the predetermined route, uses the camera to capture images of the river, receives abnormal events reported by users in real time, and adjusts the route for targeted shooting; The image preprocessing module performs noise reduction processing on the collected image data, smoothes the image using a Gaussian filter, performs threshold segmentation using the Otsu algorithm, distinguishes the foreground and background in the image, and performs corrosion and expansion processing using morphological operations; The anomaly detection and tracking analysis module performs edge detection and target positioning, determines the position coordinates of the object to be detected, compares suspected abnormal events with preset rules, screens out targets that need to be tracked and analyzed, performs multi-dimensional tracking analysis on the targets, identifies motion trajectories and behavior states, and constructs a behavior trajectory model; The event recognition and decision module uses CNN and K-means algorithms to extract image features and perform clustering to form feature trajectories, establishes a tracking model to predict spatial position changes and error assessment of suspected abnormal events, locates and tracks suspected abnormal events, and analyzes and determines the attributes and authenticity of abnormal events based on feature trajectories and behavior patterns.

Images