An unmanned aerial vehicle intelligent patrol early warning method and system based on river semantic segmentation and abnormal heat map linkage

Abstract

The application discloses a kind of unmanned aerial vehicle intelligent inspection early warning method and system based on riverway semantic segmentation and abnormal heat map linkage.The method comprises the following steps: unmanned aerial vehicle collects riverway inspection image, performs semantic segmentation and target detection in parallel at edge, generates abnormal water body mask and discrete abnormal target detection frame;Multi-modal fusion and spatial alignment are carried out at 5G-MEC edge node, and abnormal type, area, confidence information is extracted to construct multi-dimensional abnormal heat map;Cloud platform triggers early warning according to hierarchical early warning threshold, and calculates area change rate by comparing time and space of historical heat map, judges abnormal evolution trend to generate enhanced early warning.The system adopts three-level collaborative architecture of unmanned aerial vehicle end, 5G-MEC edge node and cloud management platform.The application realizes full-element identification, structured quantification and dynamic evolution early warning of riverway anomaly, effectively improves the comprehensiveness, objectivity and early warning timeliness of riverway inspection.

Description

Technical Field

[0001] This invention relates to the field of smart water conservancy technology, specifically to a method and system for intelligent inspection and early warning of unmanned aerial vehicles (UAVs) based on the linkage of river semantic segmentation and anomaly heat map. Background Technology

[0002] With the comprehensive advancement of smart water conservancy construction, drone-based river inspection technology has gradually replaced traditional manual foot patrols, becoming a core means of watershed water environment monitoring. However, current mainstream drone-based river inspection solutions still have many unresolved technical pain points and cannot meet the actual needs of modern river management. Existing technologies mostly adopt single-modal visual analysis techniques, either using only object detection algorithms to identify discrete point anomalies or only using semantic segmentation algorithms to extract continuous areal regions. This either-or single-modal architecture cannot simultaneously cover the complex scenario of discrete and continuous anomalies coexisting in river inspection, resulting in significant blind spots in anomaly identification. Schemes based on object detection algorithms can only output rectangular bounding boxes, suitable for identifying discrete targets with clear geometric boundaries, but cannot perform pixel-level precise segmentation of continuous areal anomalies without clear boundaries. While schemes based on semantic segmentation algorithms can extract continuous regions, they mostly remain at the level of basic land cover classification and cannot accurately locate discrete point anomalies, nor can they define the specific category of abnormal water bodies. Meanwhile, existing technologies lack a unified mechanism for quantifying anomalies. Most only output binary mask images or detection boxes, providing only qualitative results indicating the presence or absence of anomalies. They cannot automatically extract precise quantitative information about abnormal areas, leaving managers unable to directly ascertain the scale of pollution. This often requires manual review of video footage frame by frame for estimation, which is not only inefficient but also suffers from inconsistent judgment standards among different personnel, resulting in insufficient objectivity in inspection results. Furthermore, existing inspection logic is mostly based on instantaneous judgment of single-frame images, lacking the ability to deeply mine and dynamically compare historical data. The system cannot automatically correlate current inspection results with historical data, calculate the area change rate of abnormal areas, or determine whether the problem is a newly emerging, continuously worsening, or naturally regressing issue. This prevents tiered early warning based on the evolution trend of anomalies, often resulting in detection only after the anomaly has caused serious impact, missing the optimal window for remediation. Summary of the Invention

[0003] The technical problem to be solved by this invention is that existing UAV river inspection technologies suffer from single-modal perception blind spots, lack of spatial quantification capabilities, and insufficient dynamic evolution tracking. They cannot simultaneously cover discrete point-like and continuous area-like anomalies, cannot provide objective quantitative data on the scale of anomalies, and cannot predict the development trend of anomalies based on historical data, resulting in poor comprehensiveness, objectivity, and timeliness of early warning.

[0004] The technical solution of this invention to solve the above-mentioned technical problems is as follows: A UAV intelligent inspection and early warning method based on the linkage of river semantic segmentation and anomaly heatmap, comprising the following steps: S1, collecting river inspection images through a UAV to obtain raw image data; S2, executing semantic segmentation and target detection tasks in parallel on the edge computing unit of the UAV, wherein the semantic segmentation task performs pixel-level classification on the preprocessed raw image data, generates a pixel-level classification mask containing the abnormal water body category, extracts the pixels of the abnormal water body category from the pixel-level classification mask, and obtains the abnormal water body mask; the target detection task performs bounding box regression on the preprocessed raw image data to obtain discrete abnormal target detection boxes, and outputs the corresponding confidence scores; S3, performing multimodal fusion at the edge nodes, combining the abnormal water body mask with the discrete abnormal target detection... S4. The edge nodes construct a multidimensional anomaly heatmap, including at least a type channel, an area channel, and a confidence channel, and determine the anomaly region based on the multidimensional anomaly heatmap; S5. The edge nodes transmit the multidimensional anomaly heatmap to the cloud management platform, and the cloud management platform triggers different levels of basic early warning information according to preset graded early warning thresholds; S6. The cloud management platform automatically retrieves historical multidimensional anomaly heatmaps of the same river section, performs a spatiotemporal comparison between the current multidimensional anomaly heatmap and the historical multidimensional anomaly heatmap, calculates the area change trend of the anomaly region, and if the anomaly region shows a significant expansion compared to the same period in history, it generates enhanced trend information; the enhanced trend information is attached to the basic early warning information to form the final early warning information output to the terminal device.

[0005] The beneficial effects of this invention are: by constructing a complete technical chain of dual-modal parallel sensing, multi-modal fusion quantification and historical time series comparison, it realizes the full-element identification, structured quantification and dynamic evolution early warning of river anomalies, comprehensively solves the core defects of existing technologies, and significantly improves the intelligence level and early warning capability of river inspection.

[0006] Based on the above technical solution, the present invention can be further improved as follows.

[0007] Furthermore, the semantic segmentation task in S2 specifically includes: the edge computing unit on the UAV scales the collected river inspection image to a first pixel size, inputs it into a lightweight semantic segmentation model, and outputs four masks: 0=background, 1=normal water body, 2=shoreline, and 3=abnormal water body; the target detection task in S2 specifically includes: the edge computing unit on the UAV scales the same frame of river inspection image to a second pixel size, inputs it into a lightweight target detection model, and outputs N bounding boxes. Each bounding box contains the target category, confidence value, and the coordinate information of the bounding box in the image. The target categories include sewage outlets, floating objects, and illegal buildings.

[0008] The beneficial effects of adopting the above-mentioned further scheme are: by combining the optimized lightweight semantic segmentation model with the target detection model, it is ensured that the model can run in real time on the edge computing unit of the UAV and has sufficient detection accuracy. At the same time, the input and output format of the model is unified, laying a solid foundation for subsequent multimodal fusion processing.

[0009] Further, S4 specifically includes: performing the following steps at the edge node: S41, extracting anomaly type information, anomaly area information, and model confidence information for each connected region in the spatially aligned anomalous water body mask and each detection box in the discrete anomaly target detection box; S42, creating a three-channel floating-point matrix with the same resolution as the original image, denoted as... ,in Represents a multidimensional anomaly heatmap matrix. Represents the real number field. The height and width of the multidimensional anomaly heatmap are represented, and the values ​​are consistent with the height and width of the original image; S43, the anomaly type information is assigned to the first channel, which is a type channel and uses integer value encoding to identify the anomaly type of each pixel position; S44, the anomaly area information is normalized and assigned to the second channel, which is an area channel. The specific calculation method is as follows: for each connected anomaly region in the multidimensional anomaly heatmap, the number of pixels contained in the anomaly region is counted and recorded as... And set the area channel value of all pixels in the abnormal region to equal Divide by the number of pixels corresponding to the preset maximum area threshold. S45. The model confidence information is smoothed and then assigned to the third channel, which is the confidence channel.

[0010] The beneficial effects of adopting the above-mentioned further scheme are: by defining a standardized three-channel floating-point matrix form of multidimensional anomaly heatmap data structure, the three core information of anomaly type, area, and confidence level are digitally encoded, realizing the structured and quantitative expression of anomaly information, which facilitates subsequent storage, transmission, analysis and application.

[0011] Furthermore, the basic early warning information in S5 is triggered by the cloud management platform according to the following rules: when the area of ​​the abnormal area is greater than the first area threshold and the confidence level is greater than the first confidence threshold, a red warning is triggered; when a suspected sewage outlet or the area of ​​the abnormal area is detected to be between the second area threshold and the first area threshold, a yellow warning is triggered, wherein the first area threshold is greater than the second area threshold.

[0012] The beneficial effects of adopting the above-mentioned further scheme are: by using a graded early warning mechanism based on the abnormal area and confidence level, different early warning levels are set according to the severity of the abnormality, which realizes differentiated alarms, effectively filters invalid early warning information, improves the pertinence and effectiveness of early warning information, and facilitates the rational allocation of disposal resources by managers.

[0013] Furthermore, the cloud platform in S6 performs the following steps: S61, extracting the current multidimensional anomaly heatmap. Historical multidimensional anomaly heatmap Binary masks for anomalous water body regions, denoted as follows: and ,in The time interval between historical data and current data; S62, if If so, the current abnormal region is directly determined to be a newly added abnormality, and enhanced trend information including the newly added abnormality marker is generated; S63, if The growth rate of the abnormal region area is calculated using the following expression: ,in This indicates the area of ​​the current abnormal water body. Indicates the area of ​​historically abnormal water bodies. This indicates the change in the area of ​​the abnormal region. Indicates the area of ​​historical anomalous regions; S64, if >When the change in the area of ​​the second area threshold or the area of ​​the abnormal region exceeds the preset growth rate threshold, it is determined that the abnormal region has significantly expanded, and enhanced trend information including a significant expansion marker is generated; S65, the enhanced trend information is attached to the basic warning information to form the final warning information and output to the terminal device.

[0014] The beneficial effects of adopting the above-mentioned further scheme are: through the spatiotemporal comparison mechanism of historical heat maps, it is possible to automatically track the dynamic evolution process of anomalies, calculate the area change rate of anomaly regions, distinguish different anomaly development states, enhance early warning for significantly deteriorating anomalies, realize the transformation from post-event handling to pre-event prevention, and significantly improve the foresight and accuracy of early warning.

[0015] Furthermore, the multimodal fusion and multidimensional anomaly heatmap generation operations in S3 are both completed at the edge nodes, while the early warning judgment in S5 and the historical comparison and enhanced trend information generation in S6 are both completed on the cloud management platform.

[0016] The beneficial effects of adopting the above-mentioned further solution are as follows: by adopting a three-level collaborative architecture of drone terminal, 5G-MEC edge node and cloud management platform, computing tasks of different complexities are distributed to nodes at different levels for execution, which reduces transmission bandwidth and latency, improves system reliability and security, and ensures the stability and real-time performance of data transmission by adopting standardized communication protocols.

[0017] Furthermore, this application also provides an intelligent UAV inspection and early warning system based on the linkage of river semantic segmentation and anomaly heatmap, including a UAV terminal, edge nodes, and a cloud management platform; the UAV terminal is equipped with a visual recognition unit and an edge artificial intelligence computing unit, wherein the edge artificial intelligence computing unit simultaneously deploys a lightweight semantic segmentation model and a lightweight target detection model, for parallel operation of the lightweight semantic segmentation model and the lightweight target detection model, generating abnormal water body masks and discrete anomaly target detection boxes respectively, and sending preliminary analysis results through the uplink; the MEC edge nodes are deployed along the river. On the communication base station side, a communication connection is established with the UAV terminal via the network to receive the preliminary analysis results sent by the UAV terminal, perform multimodal fusion calculations and generate a multidimensional anomaly heatmap, and process the generated multidimensional anomaly heatmap; the cloud management platform is deployed in a government cloud or private cloud environment, establishes a communication connection with the edge node via the network, receives and stores the multidimensional anomaly heatmap, performs hierarchical early warning to generate basic early warning information, retrieves historical heatmaps for comparison to generate enhanced trend information, attaches the enhanced trend information to the basic early warning information to form the final early warning information, and outputs it to the terminal device.

[0018] Furthermore, the edge nodes are configured to perform the following operations to generate a multidimensional anomaly heatmap: extracting anomaly type information, anomaly area information, and model confidence information for each connected region in the spatially aligned anomaly water body mask and each detection box in the discrete anomaly target detection box; creating a three-channel floating-point matrix with the same resolution as the original image, denoted as... ,in Represents a multidimensional anomaly heatmap matrix. Represents the real number field. The height and width of the multidimensional anomaly heatmap are represented, and the values ​​are consistent with the height and width of the original image. The anomaly type information is assigned to the first channel, which is a type channel and uses integer value encoding to identify the anomaly type of each pixel location. The anomaly area information is normalized and assigned to the second channel, which is an area channel. The specific calculation method is as follows: for each connected anomaly region in the multidimensional anomaly heatmap, the number of pixels contained in the anomaly region is counted and recorded as... And set the area channel value of all pixels in the abnormal region to equal Divide by the number of pixels corresponding to the preset maximum area threshold. The model confidence information is smoothed and then assigned to the third channel, which is the confidence channel.

[0019] Furthermore, the cloud management platform is specifically configured to perform the following operations: store historical multidimensional anomaly heatmaps of the same river segment identified by GPS coordinate segments, with a time granularity of once per day; trigger basic early warning information according to preset rules, the basic early warning information including red and yellow warnings, the red warning being triggered when the area of ​​the anomaly region is greater than a first area threshold and the confidence level is greater than the first confidence threshold, the yellow warning being triggered when a suspected sewage outlet is detected or the area of ​​the anomaly region is between a second area threshold and a first area threshold; retrieve historical multidimensional anomaly heatmaps, if... If so, the current abnormal region is directly determined to be a newly added anomaly, and enhanced trend information including the newly added anomaly marker is generated; if The growth rate of the abnormal region area is calculated using the following expression: ,in This indicates the area of ​​the current abnormal water body. Indicates the area of ​​historically abnormal water bodies. This indicates the change in the area of ​​the abnormal region. Indicates the area of ​​the historical anomalous region; if When the change in the area of ​​the second area threshold or the area of ​​the abnormal region exceeds the preset growth rate threshold, it is determined that the abnormal region has significantly expanded, and enhanced trend information including a significant expansion marker is generated; the enhanced trend information is attached to the basic warning information to form the final warning information and pushed to the terminal device.

[0020] Furthermore, this application also provides an electronic device, including: a processor; a memory for storing executable instructions of the processor; wherein the processor is configured to execute the instructions to realize the aforementioned UAV intelligent inspection and early warning method based on river semantic segmentation and anomaly heat map linkage.

[0021] Furthermore, this application also provides a computer-readable storage medium storing at least one computer program, which is loaded and executed by a processor to enable the computer to implement the aforementioned UAV intelligent inspection and early warning method based on the linkage of river semantic segmentation and anomaly heat map. Attached Figure Description

[0022] Figure 1 This is a flowchart illustrating an intelligent UAV inspection and early warning method based on the linkage of river semantic segmentation and anomaly heatmap according to the present invention. Figure 2 This is a schematic diagram of the structure of an intelligent UAV inspection and early warning system based on the linkage of river semantic segmentation and anomaly heat map according to the present invention. Figure 3 This is a schematic diagram of the structure of an electronic device according to the present invention. Detailed Implementation

[0023] The principles and features of the present invention are described below. The examples given are only for explaining the present invention and are not intended to limit the scope of the present invention.

[0024] Example 1: Existing drone-based river inspection methods mostly employ single-modal sensing technology, which cannot simultaneously cover discrete point-like and continuous area-like anomalies. Furthermore, they lack a unified quantitative expression and dynamic early warning mechanism, resulting in the comprehensiveness, objectivity, and timeliness of inspections failing to meet the actual needs of smart water conservancy.

[0025] Reference Figure 1This embodiment provides an intelligent UAV inspection and early warning method based on the linkage of river semantic segmentation and anomaly heatmap, including the following steps: S1, collecting river inspection images through a UAV to obtain raw image data; S2, executing semantic segmentation and target detection tasks in parallel on the edge computing unit of the UAV, wherein the semantic segmentation task performs pixel-level classification on the preprocessed raw image data, generates a pixel-level classification mask containing the abnormal water body category, extracts the pixels of the abnormal water body category from the pixel-level classification mask to obtain the abnormal water body mask; the target detection task performs bounding box regression on the preprocessed raw image data to obtain discrete abnormal target detection boxes, and outputs the corresponding confidence scores; S3, performing multimodal fusion at the edge nodes, and combining the abnormal water body mask and the discrete abnormal target detection boxes in the raw image. S4. The edge node constructs a multidimensional anomaly heatmap, including at least a type channel, an area channel, and a confidence channel, and determines the anomaly area based on the multidimensional anomaly heatmap; S5. The edge node transmits the multidimensional anomaly heatmap from the edge node to the cloud management platform, and the cloud management platform triggers basic early warning information of different levels according to the preset graded early warning threshold; S6. The cloud management platform automatically retrieves historical multidimensional anomaly heatmaps of the same river section, performs spatiotemporal comparison between the current multidimensional anomaly heatmap and the historical multidimensional anomaly heatmap, calculates the area change trend of the anomaly area, and if the anomaly area shows a significant expansion compared to the same period in history, it generates enhanced trend information; the enhanced trend information is attached to the basic early warning information to form the final early warning information output to the terminal device.

[0026] In another possible embodiment, river inspection images are first acquired using a visual recognition unit mounted on the drone. The visual recognition unit refers to an image acquisition device mounted on the drone, including a visible light camera or a multispectral camera, capable of acquiring high-definition visual information of the river surface, obtaining raw image data, and storing it in the drone's local storage unit. Subsequently, semantic segmentation and object detection tasks are executed in parallel on the edge computing unit at the drone's end. The edge computing unit refers to an embedded artificial intelligence computing device integrated on the drone, capable of processing multiple artificial intelligence models in parallel. The semantic segmentation task refers to computer vision technology that classifies each pixel in the image, and it is used to classify images by size. The original image data, after preprocessing operations such as scaling and normalization, undergoes pixel-level classification to generate pixel-level classification masks containing categories such as background, normal water bodies, shorelines, and abnormal water bodies. All pixels labeled as abnormal water body categories are extracted from these masks to obtain an abnormal water body mask containing only the abnormal water body region. Abnormal water bodies include algal bloom water bodies, high-turbidity water bodies, or oily / smoky water bodies. Target detection is a computer vision technique for locating and identifying specific targets in an image. It performs bounding box regression on the same original image data after the same preprocessing operations to obtain discrete abnormal target detection boxes containing the location and range of discrete abnormal targets, and outputs the model confidence corresponding to each detection box. The confidence level represents the model's certainty about the detection result. Next, multimodal fusion is performed at edge nodes. Edge nodes refer to multi-access edge computing nodes deployed near communication base stations along the river channel, providing low-latency computing services close to the data generation location. Abnormal water body masks and discrete abnormal target detection boxes are uniformly mapped to the pixel coordinate system of the original image for spatial alignment, eliminating coordinate deviations between different model outputs. Then, a multidimensional anomaly heatmap is constructed, including at least a type channel, an area channel, and a confidence level channel. The location and extent of all abnormal regions are determined based on the pixel values ​​in the multidimensional anomaly heatmap. Finally, the multidimensional anomaly heatmap is transmitted via a wireless communication network from... Edge nodes transmit data to the cloud management platform. The cloud management platform judges abnormal areas based on pre-set hierarchical early warning thresholds, triggering basic early warning information of different levels. Finally, the cloud management platform automatically retrieves historical multidimensional anomaly heat maps of the same river section at corresponding time intervals, performs spatiotemporal comparison between the current multidimensional anomaly heat map and the historical multidimensional anomaly heat map, calculates the area change trend of the abnormal area, and if the abnormal area shows a significant expansion compared to the same period in history, it generates enhanced trend information containing the anomaly evolution, attaches the enhanced trend information to the basic early warning information, forms the final early warning information containing basic anomaly information and evolution trend, and outputs it to the terminal devices of the river chief office and other relevant departments.

[0027] By constructing a complete technical chain of dual-modal parallel sensing, multi-modal fusion quantification, and historical time series comparison, we have achieved full-element identification, structured quantification, and dynamic evolution early warning of river anomalies. This has comprehensively solved the core defects of existing technologies and significantly improved the intelligence level and early warning capabilities of river inspection.

[0028] Preferably, in this embodiment, the semantic segmentation task in S2 specifically includes: the edge computing unit on the UAV scales the collected river inspection image to a first pixel size, inputs it into a lightweight semantic segmentation model, and outputs four masks: 0=background, 1=normal water body, 2=shoreline, and 3=abnormal water body; the target detection task in S2 specifically includes: the edge computing unit on the UAV scales the same frame of river inspection image to a second pixel size, inputs it into a lightweight target detection model, and outputs N bounding boxes, each bounding box containing the target category, confidence value, and coordinate information of the bounding box in the image, wherein the target category includes sewage outlets, floating objects, and illegal buildings.

[0029] In another possible embodiment, the acquired river inspection images are scaled to 512×512 pixels and then input into a lightweight semantic segmentation model with MobileNetV3 as the backbone network and DeepLabv3+ as the segmentation head. This lightweight semantic segmentation model outputs pixel-level masks for four categories, where each category is encoded as 0 for background, 1 for normal water, 2 for shoreline, and 3 for abnormal water. The target detection task is specifically performed as follows: the same frame of river inspection images is scaled to 640×640 pixels and then input into a lightweight target detection model YOLOv5s. This lightweight target detection model YOLOv5s outputs N bounding boxes, where N is the number of detected targets. Each bounding box contains the target category, confidence value, and the pixel coordinates of the top left and bottom right corners of the bounding box in the image. Target categories include sewage outlets, floating objects, and illegal buildings.

[0030] By combining an optimized lightweight semantic segmentation model with an object detection model, we can ensure that the model can run in real time on the edge computing unit of the UAV and has sufficient detection accuracy. At the same time, we can unify the input and output format of the model, laying the foundation for subsequent multimodal fusion processing.

[0031] Preferably, in this embodiment, S4 specifically includes: performing the following steps at the edge node: S41, extracting anomaly type information, anomaly area information, and model confidence information for each connected region in the spatially aligned anomalous water body mask and each detection box in the discrete anomaly target detection box; S42, creating a three-channel floating-point matrix with the same resolution as the original image, denoted as... ,in Represents a multidimensional anomaly heatmap matrix. Represents the real number field. The height and width of the multidimensional anomaly heatmap are represented, and the values ​​are consistent with the height and width of the original image; S43, the anomaly type information is assigned to the first channel, which is a type channel and uses integer value encoding to identify the anomaly type of each pixel position; S44, the anomaly area information is normalized and assigned to the second channel, which is an area channel. The specific calculation method is as follows: for each connected anomaly region in the multidimensional anomaly heatmap, the number of pixels contained in the anomaly region is counted and recorded as... And set the area channel value of all pixels in the abnormal region to equal Divide by the number of pixels corresponding to the preset maximum area threshold. S45. The model confidence information is smoothed and then assigned to the third channel, which is the confidence channel.

[0032] In another possible embodiment, the multidimensional anomaly heatmap is a three-channel floating-point matrix with the same resolution as the original image, denoted as... ,in Represents a multidimensional anomaly heatmap matrix. Represents the real number field. This represents the height and width of the multidimensional anomaly heatmap, with values ​​consistent with the original image's height and width. The first channel is the type channel, encoded with integer values ​​to identify the anomaly type of each pixel location. Different integer values ​​correspond to different anomaly categories, allowing for intuitive differentiation of different types of river anomalies. The second channel is the area channel, storing normalized area values. Specifically, for each connected anomaly region in the multidimensional anomaly heatmap, the number of pixels contained within that region is counted, denoted as _____. Then, the area channel value of all pixels in the abnormal area is assigned a value. , The first channel represents the number of pixels corresponding to the preset maximum area threshold. Normalization can unify abnormal regions of different sizes into the same numerical range, which is convenient for subsequent comparison and analysis. The third channel is the confidence channel, whose value comes directly from the original confidence probability output by the semantic segmentation model or object detection model, and is smoothed by the Sigmoid function. The Sigmoid function can map any real number to between 0 and 1, making the confidence value more stable and easier to understand.

[0033] By defining a standardized three-channel floating-point matrix form of multidimensional anomaly heatmap data structure, the three core information of anomaly type, area, and confidence level are digitally encoded, realizing the structured and quantitative expression of anomaly information, which facilitates subsequent storage, transmission, analysis and application.

[0034] Preferably, in this embodiment, the basic early warning information in S5 is triggered by the cloud management platform according to the following rules: when the area of ​​the abnormal area is greater than the first area threshold and the confidence level is greater than the first confidence threshold, a red warning is triggered; when a suspected sewage outlet or the area of ​​the abnormal area is detected to be between the second area threshold and the first area threshold, a yellow warning is triggered, wherein the first area threshold is greater than the second area threshold.

[0035] In another possible embodiment, the tiered early warning rules specifically include the following judgment conditions: when the area of ​​the abnormal area is greater than 500 square meters and the confidence level is greater than 0.8, a red warning is triggered. A red warning represents the highest level of abnormality, requiring immediate action and dispatching personnel to the site for verification and handling. When a suspected sewage outlet or abnormal area is detected with an area between 100 and 500 square meters, a yellow warning is triggered. A yellow warning represents the second highest level of abnormality, requiring on-site verification and handling within a specified time. When the abnormal area is less than 100 square meters and the confidence level is greater than 0.5, an abnormality record is generated and marked as pending verification. The cloud management platform will generate corresponding basic early warning information based on the triggered warning level, including the basic location, type, and severity of the abnormal area, to facilitate managers in quickly grasping the situation of the abnormal area.

[0036] By establishing a hierarchical early warning mechanism based on anomaly area and confidence level, different early warning levels are set according to the severity of the anomaly, achieving differentiated alarms. This effectively filters out invalid early warning information, improves the relevance and effectiveness of early warning information, and facilitates the rational allocation of resources by management personnel.

[0037] Preferably, in this embodiment, the cloud platform in S6 performs the following steps: S61, extracting the current multidimensional anomaly heatmap. Historical multidimensional anomaly heatmap Binary masks for anomalous water body regions, denoted as follows: and ,in The time interval between historical data and current data; S62, if If so, the current abnormal region is directly determined to be a newly added abnormality, and enhanced trend information including the newly added abnormality marker is generated; S63, if The growth rate of the abnormal region area is calculated using the following expression: ,in This indicates the area of ​​the current abnormal water body. Indicates the area of ​​historically abnormal water bodies. This indicates the change in the area of ​​the abnormal region. Indicates the area of ​​historical anomalous regions; S64, if >When the change in the area of ​​the second area threshold or the area of ​​the abnormal region exceeds the preset growth rate threshold, it is determined that the abnormal region has significantly expanded, and enhanced trend information including a significant expansion marker is generated; S65, the enhanced trend information is attached to the basic warning information to form the final warning information and output to the terminal device.

[0038] In another possible embodiment, a multidimensional anomaly heatmap is first extracted. Historical multidimensional anomaly heatmap Binary masks for anomalous water body regions, denoted as follows: and A binary mask is an image that includes only two pixel values: 0 and 1. 1 represents an abnormal region, and 0 represents a normal region. The time interval between historical and current data can be set according to actual inspection needs. If the area of ​​the historical abnormal water body region is 0, the current abnormal area is directly determined to be a new anomaly, and enhanced trend information including a new anomaly marker is generated to alert management personnel that the abnormal area is a recently emerging problem that requires close attention. If the area of ​​the historical abnormal water body region is greater than 0, the growth rate of the abnormal area area is calculated, expressed as: It can intuitively reflect the change range of abnormal areas; if the difference between the current abnormal area and the historical abnormal area is greater than the preset second area threshold or the change in the abnormal area is greater than the preset growth rate threshold, it is determined that the abnormal area is significantly expanding, and enhanced trend information including a significant expansion marker is generated to prompt the management personnel that the abnormality is rapidly deteriorating and timely intervention measures are needed. Finally, the enhanced trend information is attached to the basic early warning information to form the final early warning information containing the basic situation of the abnormality and the evolution trend, which is output to the terminal device.

[0039] By using the historical heatmap spatiotemporal comparison mechanism, the dynamic evolution of anomalies can be automatically tracked, distinguishing between newly added anomalies and continuously expanding anomalies. It can also enhance trend information for significantly deteriorating anomalies, realizing the shift from post-event handling to pre-event prevention and significantly improving the foresight and accuracy of early warning.

[0040] Preferably, in this embodiment, the multimodal fusion and multidimensional anomaly heatmap generation operations in S3 are both completed at the edge node, and the early warning judgment in S5 and the historical comparison and enhanced trend information generation in S6 are both completed on the cloud management platform.

[0041] In another possible embodiment, multimodal fusion and multidimensional anomaly heatmap generation are both completed at the 5G-MEC edge node, while early warning judgment, historical comparison, and enhanced early warning generation are all completed on the cloud management platform. Data flow within the system is unidirectional. The UAV sends the generated abnormal water body mask and discrete anomaly target detection boxes as preliminary analysis results to the 5G-MEC edge node via the 5G uplink. The 5G-MEC edge node compresses and encrypts the constructed multidimensional anomaly heatmap before sending it to the cloud management platform. This unidirectional data flow effectively prevents data tampering and improves system security. Furthermore, communication between the UAV and the 5G-MEC edge node, and between the 5G-MEC edge node and the cloud management platform, uses the HTTP / 2 protocol carried over the user plane functions of the 5G network (HTTP / 2 over 5G UPF). The HTTP / 2 protocol has features such as multiplexing and header compression, which can significantly improve data transmission efficiency. UPF is the user plane functional entity of 5G networks, responsible for data routing and forwarding. Using this communication protocol can ensure that the end-to-end data transmission latency meets the real-time requirements of river inspection.

[0042] By adopting a three-tier collaborative architecture of drone terminals, 5G-MEC edge nodes, and cloud management platforms, computing tasks of varying complexity are distributed to nodes at different levels for execution, which reduces transmission bandwidth and latency while improving system reliability and security.

[0043] Example 2: Reference Figure 2 This embodiment also relates to an intelligent UAV inspection and early warning system based on the linkage of river semantic segmentation and anomaly heatmap, including a UAV terminal, edge nodes, and a cloud management platform. The UAV terminal is equipped with a visual recognition unit and an edge artificial intelligence computing unit. The edge artificial intelligence computing unit simultaneously deploys a lightweight semantic segmentation model and a lightweight target detection model for parallel operation of the lightweight semantic segmentation model and the lightweight target detection model, generating abnormal water body masks and discrete anomaly target detection boxes, respectively, and sending preliminary analysis results through the uplink. The MEC edge nodes are deployed along the river. On the base station side, a communication connection is established with the drone terminal via the network to receive the preliminary analysis results sent by the drone terminal, perform multimodal fusion calculations and generate a multidimensional anomaly heatmap, and process the generated multidimensional anomaly heatmap; the cloud management platform is deployed in a government cloud or private cloud environment, establishes a communication connection with the edge node via the network, receives and stores the multidimensional anomaly heatmap, performs hierarchical early warning to generate basic early warning information, retrieves historical heatmaps for comparison to generate enhanced trend information, attaches the enhanced trend information to the basic early warning information to form the final early warning information, and outputs it to the terminal device.

[0044] In another possible embodiment, the system consists of a drone, 5G-MEC edge nodes, and a cloud management platform. The drone is equipped with a visual recognition unit and an edge AI computing unit. The visual recognition unit is used to collect high-definition image data for river inspection. The edge AI computing unit simultaneously deploys a lightweight semantic segmentation model and a lightweight object detection model, which are run in parallel to generate abnormal water body masks and discrete abnormal object detection boxes, respectively. The preliminary analysis results are then sent out via a 5G uplink. The 5G-MEC edge nodes are deployed at communication base stations along the river and establish communication connections with the drone via the 5G network to receive the preliminary analysis data sent by the drone. As a result, multimodal fusion computation is performed locally to generate a multidimensional anomaly heatmap. The generated heatmap data is then compressed and encrypted before being uploaded. The cloud management platform is deployed in a government cloud or private cloud environment and establishes a communication connection with 5G-MEC edge nodes via a 5G network. This connection is used to receive and store multidimensional anomaly heatmap data from the edge nodes, perform judgments on hierarchical early warning rules, retrieve historical heatmaps, and calculate the rate of change of abnormal areas. Finally, alarm information is pushed to the terminal devices of the River Chief Office through a Web Application Programming Interface (WebAPI). A WebAPI is a standardized software interaction interface that enables data exchange and function calls between different systems.

[0045] Preferably, in this embodiment, the edge AI computing unit on the UAV side specifically adopts the NVIDIA Jetson AGX Orin chip; the lightweight semantic segmentation model is specifically a combination of MobileNetV3 and DeepLabv3+, the input image size of the lightweight semantic segmentation model is 512×512 pixels, and the output is a pixel-level mask for four categories; the lightweight object detection model is specifically YOLOv5s, the input image size of the lightweight object detection model is 640×640 pixels, and the output is a bounding box containing object category, confidence value, and coordinate information.

[0046] Preferably, in this embodiment, the edge nodes are configured to perform the following operations to generate a multidimensional anomaly heatmap: extracting anomaly type information, anomaly area information, and model confidence information for each connected region in the spatially aligned anomaly water mask and each detection box in the discrete anomaly target detection box; creating a three-channel floating-point matrix with the same resolution as the original image, denoted as... ,in Represents a multidimensional anomaly heatmap matrix. Represents the real number field. The height and width of the multidimensional anomaly heatmap are represented, and the values ​​are consistent with the height and width of the original image. The anomaly type information is assigned to the first channel, which is a type channel and uses integer value encoding to identify the anomaly type of each pixel location. The anomaly area information is normalized and assigned to the second channel, which is an area channel. The specific calculation method is as follows: for each connected anomaly region in the multidimensional anomaly heatmap, the number of pixels contained in the anomaly region is counted and recorded as... And set the area channel value of all pixels in the abnormal region to equal Divide by the number of pixels corresponding to the preset maximum area threshold. The model confidence information is smoothed and then assigned to the third channel, which is the confidence channel.

[0047] Preferably, in this embodiment, the cloud management platform is specifically configured to perform the following operations: store historical multidimensional anomaly heatmaps of the same river segment identified by GPS coordinate segments, with a time granularity of once per day; trigger basic early warning information according to preset rules, the basic early warning information including red and yellow warnings, wherein a red warning is triggered when the area of ​​the abnormal region is greater than a first area threshold and the confidence level is greater than the first confidence threshold, and a yellow warning is triggered when a suspected sewage outlet is detected or the area of ​​the abnormal region is between a second area threshold and a first area threshold; retrieve historical multidimensional anomaly heatmaps, if... If so, the current abnormal region is directly determined to be a newly added anomaly, and enhanced trend information including the newly added anomaly marker is generated; if The growth rate of the abnormal region area is calculated using the following expression: ,in This indicates the area of ​​the current abnormal water body. Indicates the area of ​​historically abnormal water bodies. This indicates the change in the area of ​​the abnormal region. Indicates the area of ​​the historical anomalous region; if When the change in the area of ​​the second area threshold or the area of ​​the abnormal region exceeds the preset growth rate threshold, it is determined that the abnormal region has significantly expanded, and enhanced trend information including a significant expansion marker is generated; the enhanced trend information is attached to the basic warning information to form the final warning information and pushed to the terminal device.

[0048] In some embodiments, the UAV intelligent inspection and early warning system based on river semantic segmentation and anomaly heat map linkage of the present invention can be implemented in a combination of hardware and software. As an example, the UAV intelligent inspection and early warning system based on river semantic segmentation and anomaly heat map linkage of the present invention can be a processor in the form of a hardware decoding processor, which is programmed to execute the UAV intelligent inspection and early warning method based on river semantic segmentation and anomaly heat map linkage of the present invention. For example, the processor in the form of a hardware decoding processor can adopt one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), or other electronic components.

[0049] The modules described in the embodiments of this invention can be implemented in software or hardware. The names of the modules are not, in some cases, limiting the scope of the module itself.

[0050] An electronic device according to an embodiment of the present invention includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements any of the above-mentioned intelligent UAV inspection and early warning methods based on the linkage of river semantic segmentation and anomaly heat map. That is, an electronic device according to an embodiment of the present invention may include, but is not limited to: a processor and a memory; the memory is used to store the computer program; the processor is used to execute the intelligent UAV inspection and early warning method based on the linkage of river semantic segmentation and anomaly heat map according to any embodiment of the present invention by calling the computer program.

[0051] In one alternative embodiment, an electronic device is provided, such as Figure 3 As shown, Figure 3 The illustrated electronic device 4000 includes a processor 4001 and a memory 4003. The processor 4001 and the memory 4003 are connected, for example, via a bus 4002. Optionally, the electronic device 4000 may further include a transceiver 4004, which can be used for data interaction between the electronic device and other electronic devices, such as sending and / or receiving data. It should be noted that in practical applications, the transceiver 4004 is not limited to one type, and the structure of the electronic device 4000 does not constitute a limitation on the embodiments of the present invention.

[0052] Processor 4001 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It can implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this invention. Processor 4001 may also be a combination that implements computational functions, such as including one or more microprocessor combinations, a combination of a DSP and a microprocessor, etc.

[0053] Bus 4002 may include a path for transmitting information between the aforementioned components. Bus 4002 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. Bus 4002 can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 3 The bus 4002 is represented by only one thick line, but this does not mean that there is only one bus or one type of bus.

[0054] The memory 4003 may be ROM (Read Only Memory) or other types of static storage devices capable of storing static information and instructions, RAM (Random Access Memory) or other types of dynamic storage devices capable of storing information and instructions, or EEPROM (Electrically Erasable Programmable Read Only Memory), CD-ROM (Compact Disc Read Only Memory) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited thereto.

[0055] The memory 4003 stores application code (computer program) for executing the present invention, and its execution is controlled by the processor 4001. The processor 4001 executes the application code stored in the memory 4003 to implement the content shown in the foregoing method embodiments.

[0056] Among them, electronic devices can also be terminal devices, which can be any device that can install applications, including at least one of smartphones, tablets, laptops, desktop computers, smart speakers, smartwatches, smart TVs, and smart in-vehicle devices.

[0057] It should be noted that, Figure 3 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of use of the embodiments of the present invention.

[0058] An embodiment of the present invention provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements any of the above-mentioned intelligent UAV inspection and early warning methods based on the linkage of river semantic segmentation and anomaly heatmap.

[0059] Alternatively, the computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a compact disc read-only memory (CD-ROM), magnetic tape, a floppy disk, and an optical data storage device, etc.

[0060] In an exemplary embodiment, a computer program product or computer program is also provided, which includes computer instructions stored in a computer-readable storage medium. The processor of an electronic device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the electronic device to perform the aforementioned UAV intelligent inspection and early warning method based on the linkage of river semantic segmentation and anomaly heatmap.

[0061] Computer program code for performing the operations of this invention can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0062] It should be understood that the flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of methods and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0063] The computer-readable storage medium provided in this invention can be, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EEPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this invention, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0064] The aforementioned computer-readable storage medium carries one or more programs, which, when executed by the electronic device, cause the electronic device to perform the method shown in the above embodiments.

[0065] The above description is merely a preferred embodiment of the present invention and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of disclosure in this invention is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-disclosed concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features with similar functions disclosed in this invention.

[0066] It should be noted that the terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and do not imply a specific order or sequence. Where appropriate, the order of use for similar objects can be interchanged so that the embodiments of this application described herein can be implemented in an order other than that shown or described.

[0067] Those skilled in the art will recognize that this invention can be implemented as a system, method, or computer program product. Therefore, this invention can be specifically implemented in the following forms: it can be entirely hardware, entirely software (including firmware, resident software, microcode, etc.), or a combination of hardware and software, generally referred to herein as a "circuit," "module," or "system." Furthermore, in some embodiments, this invention can also be implemented as a computer program product contained in one or more computer-readable media, which includes computer-readable program code.

[0068] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A method for intelligent inspection and early warning of unmanned aerial vehicles (UAVs) based on the linkage of river semantic segmentation and anomaly heatmap, characterized in that, Includes the following steps: S1. Collect river inspection images using drones to obtain raw image data; S2. The semantic segmentation task and the target detection task are executed in parallel on the edge computing unit of the UAV. The semantic segmentation task performs pixel-level classification on the preprocessed original image data, generates a pixel-level classification mask containing the abnormal water body category, and extracts the pixels of the abnormal water body category from the pixel-level classification mask to obtain the abnormal water body mask. The target detection task performs bounding box regression on the preprocessed original image data to obtain discrete abnormal target detection boxes and outputs the corresponding confidence scores. S3. Perform multimodal fusion at the edge nodes to align the abnormal water body mask with the discrete abnormal target detection box in the original image coordinate system. S4. The edge nodes construct a multidimensional anomaly heatmap that includes at least a type channel, an area channel, and a confidence channel, and determine the anomaly region based on the multidimensional anomaly heatmap; S5. The multidimensional anomaly heatmap is transmitted from the edge node to the cloud management platform, and the cloud management platform triggers basic early warning information of different levels according to the preset hierarchical early warning threshold. S6. The cloud management platform automatically retrieves historical multidimensional anomaly heat maps of the same river section, performs spatiotemporal comparison between the current multidimensional anomaly heat map and the historical multidimensional anomaly heat map, calculates the area change trend of the anomaly region, and generates enhanced trend information if the anomaly region shows a significant expansion compared to the same period in history. The enhanced trend information is attached to the basic early warning information to form the final early warning information, which is then output to the terminal device.

2. The method for intelligent UAV inspection and early warning based on the linkage of river semantic segmentation and anomaly heatmap as described in claim 1, characterized in that, The semantic segmentation task in S2 specifically includes: The edge computing unit at the drone end scales the collected river inspection images to the size of the first pixel, inputs them into a lightweight semantic segmentation model, and outputs four masks: 0=background, 1=normal water body, 2=shoreline, and 3=abnormal water body. The target detection task in S2 specifically includes: The edge computing unit at the drone end scales the same frame of river inspection image to the second pixel size, inputs it into a lightweight target detection model, and outputs N bounding boxes. Each bounding box contains the target category, confidence value, and the coordinate information of the bounding box in the image. The target categories include sewage outlets, floating objects, and illegal buildings.

3. The method for intelligent UAV inspection and early warning based on the linkage of river semantic segmentation and anomaly heatmap as described in claim 1, characterized in that, S4 specifically includes performing the following steps at the edge node: S41. Extract anomaly type information, anomaly area information, and model confidence information from each connected region in the spatially aligned anomaly water body mask and each detection box in the discrete anomaly target detection box. S42. Create a three-channel floating-point matrix with the same resolution as the original image, denoted as . ,in Represents a multidimensional anomaly heatmap matrix. Represents the real number field. This represents the height and width of the multidimensional anomaly heatmap, and the values ​​are consistent with the height and width of the original image; S43. Assign the anomaly type information to the first channel, where the first channel is a type channel and uses integer value encoding to identify the anomaly type of each pixel position; S44. The abnormal area information is normalized and assigned to the second channel, which is the area channel. The specific calculation method is as follows: for each connected abnormal region in the multidimensional abnormal heatmap, the number of pixels contained in the abnormal region is counted and recorded as... And set the area channel value of all pixels in the abnormal region to equal Divide by the number of pixels corresponding to the preset maximum area threshold. ; S45. The model confidence information is smoothed and then assigned to the third channel, which is the confidence channel.

4. The method for intelligent UAV inspection and early warning based on the linkage of river semantic segmentation and anomaly heatmap as described in claim 1, characterized in that, The basic early warning information in S5 is triggered by the cloud management platform according to the following rules: A red alert is triggered when the area of ​​the abnormal region is greater than the first area threshold and the confidence level is greater than the first confidence threshold. A yellow alert is triggered when the area of ​​a suspected sewage outlet or abnormal area is detected to be between the second area threshold and the first area threshold, wherein the first area threshold is greater than the second area threshold.

5. The intelligent UAV inspection and early warning method based on the linkage of river semantic segmentation and anomaly heatmap as described in claim 1, characterized in that, The cloud platform described in S6 performs the following steps: S61. Extract the current multidimensional anomaly heatmap. Historical multidimensional anomaly heatmap Binary masks for anomalous water body regions, denoted as follows: and ,in The time interval between historical data and current data; S62, if If so, the current abnormal area is directly determined to be a newly added abnormality, and enhanced trend information including the newly added abnormality marker is generated; S63, if The growth rate of the abnormal region area is calculated using the following expression: ,in This indicates the area of ​​the current abnormal water body. Indicates the area of ​​historically abnormal water bodies. This indicates the change in the area of ​​the abnormal region. Indicates the area of ​​historically abnormal regions; S64, if >When the change in the area of ​​the second area threshold or the area of ​​the abnormal region exceeds the preset growth rate threshold, the abnormal region is determined to have significantly expanded, and enhanced trend information including a significant expansion marker is generated; S65. The enhanced trend information is attached to the basic early warning information to form the final early warning information, which is then output to the terminal device.

6. The method for intelligent UAV inspection and early warning based on the linkage of river semantic segmentation and anomaly heatmap as described in claim 1, characterized in that, The multimodal fusion and multidimensional anomaly heatmap generation operations in S3 are both completed at the edge nodes, while the early warning judgment in S5 and the historical comparison and enhanced trend information generation in S6 are both completed on the cloud management platform.

7. A UAV intelligent inspection and early warning system based on the linkage of river channel semantic segmentation and anomaly heat map, used to execute the UAV intelligent inspection and early warning method based on the linkage of river channel semantic segmentation and anomaly heat map as described in any one of claims 1-6, characterized in that, This includes drone terminals, edge nodes, and cloud management platforms; The drone is equipped with a visual recognition unit and an edge AI computing unit. The edge AI computing unit is equipped with a lightweight semantic segmentation model and a lightweight target detection model, which are used to run the lightweight semantic segmentation model and the lightweight target detection model in parallel, generate abnormal water body masks and discrete abnormal target detection boxes, and send the preliminary analysis results through the uplink. The MEC edge node is deployed on the side of the communication base station along the river. It establishes a communication connection with the UAV terminal through the network to receive the preliminary analysis results sent by the UAV terminal, perform multimodal fusion calculations and generate a multidimensional anomaly heat map, and analyze the generated multidimensional anomaly heat map. The cloud management platform is deployed in a government cloud or private cloud environment. It establishes a communication connection with the edge node through the network to receive and store multi-dimensional abnormal heat maps, execute hierarchical early warning to generate basic early warning information, retrieve historical heat maps for comparison to generate enhanced trend information, attach the enhanced trend information to the basic early warning information to form the final early warning information, and output it to the terminal device.

8. The UAV intelligent inspection and early warning system based on the linkage of river semantic segmentation and anomaly heat map according to claim 7, characterized in that, The edge nodes are configured to perform the following operations to generate a multidimensional anomaly heatmap: For each connected region in the spatially aligned anomalous water body mask and each detection box in the discrete anomalous target detection box, extract the anomalous type information, anomalous area information, and model confidence information respectively. Create a three-channel floating-point matrix with the same resolution as the original image, denoted as . ,in Represents a multidimensional anomaly heatmap matrix. Represents the real number field. This represents the height and width of the multidimensional anomaly heatmap, and the values ​​are consistent with the height and width of the original image; The anomaly type information is assigned to the first channel, which is a type channel and uses integer value encoding to identify the anomaly type of each pixel position. The abnormal area information is normalized and assigned to the second channel, which is the area channel. Specifically, the calculation method is as follows: for each connected abnormal region in the multidimensional abnormal heatmap, the number of pixels contained in the abnormal region is counted and recorded as... And set the area channel value of all pixels in the abnormal region to equal Divide by the number of pixels corresponding to the preset maximum area threshold. ; The model confidence information is smoothed and then assigned to the third channel, which is the confidence channel.

9. The UAV intelligent inspection and early warning system based on the linkage of river semantic segmentation and anomaly heat map according to claim 7, characterized in that, The cloud management platform is specifically configured to perform the following operations: Store historical multidimensional anomaly heatmaps of the same river segment identified by GPS coordinate segments, with a time granularity of once per day; Basic early warning information is triggered according to preset rules. The basic early warning information includes red warnings and yellow warnings. The red warning is triggered when the area of ​​the abnormal area is greater than a first area threshold and the confidence level is greater than the first confidence threshold. The yellow warning is triggered when a suspected sewage outlet is detected or the area of ​​the abnormal area is between a second area threshold and a first area threshold. Retrieve historical multidimensional anomaly heatmaps, if If so, the current abnormal area is directly determined to be a newly added abnormality, and enhanced trend information including the newly added abnormality marker is generated; like The growth rate of the abnormal region area is calculated using the following expression: ,in This indicates the area of ​​the current abnormal water body. Indicates the area of ​​historically abnormal water bodies. This indicates the change in the area of ​​the abnormal region. Indicates the area of ​​historically abnormal regions; like >When the change in the area of ​​the second area threshold or the area of ​​the abnormal region exceeds the preset growth rate threshold, the abnormal region is determined to have significantly expanded, and enhanced trend information including a significant expansion marker is generated; The enhanced trend information is added to the basic early warning information to form the final early warning information, which is then pushed to the terminal device.

10. An electronic device, characterized in that, include: processor; Memory used to store the processor's executable instructions; The processor is configured to execute the instructions to implement the UAV intelligent inspection and early warning method based on the linkage of river semantic segmentation and anomaly heat map as described in any one of claims 1-6.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores at least one computer program, which is loaded and executed by a processor to enable the computer to implement the UAV intelligent inspection and early warning method based on the linkage of river semantic segmentation and anomaly heat map as described in any one of claims 1-6.

Images