Water area intelligent supervision and safety early warning system

Abstract

The application discloses a water area intelligent supervision and safety early warning system and relates to the technical field of water area monitoring.The system comprises a data acquisition and preprocessing unit, an image data extraction unit, a warning line data recognition unit and an audible and visual data recognition unit.The data acquisition and preprocessing unit is used for acquiring image data, warning line data and audible and visual data of a target water area and preprocessing the image data, the warning line data and the audible and visual data.The image data extraction unit is used for extracting building contours from the preprocessed image data by using an improved illegal building identification model, combining with geographic fence data to make illegal occupation judgment and obtaining a judgment result.The warning line data recognition unit is used for recognizing the state of the warning line from the preprocessed warning line data by using an improved warning line state detection model and generating a safety evaluation report.The audible and visual data recognition unit is used for recognizing the preprocessed audible and visual data by using audio analysis software and obtaining voiceprint data.The application builds a whole-chain management system from intelligent identification to emergency response to evidence solidification and improves the safety protection level.

Description

Technical Field

[0001] This invention relates to the field of water area monitoring technology, and more specifically, to a water area intelligent monitoring and safety early warning system. Background Technology

[0002] With the acceleration of urbanization and the increasing utilization of water resources, effectively monitoring the aquatic environment and ensuring the safety of water activities have become crucial issues. The intelligent water monitoring and safety early warning system aims to achieve real-time monitoring and safety management of rivers, lakes, and seas by integrating modern information technologies such as the Internet of Things, big data analytics, and artificial intelligence. It can not only monitor natural parameters such as water quality and flow velocity, but also identify emergencies such as illegal intrusions and drowning accidents, providing scientific basis and technical support for water management. The application of intelligent water monitoring and safety early warning systems is particularly important in densely populated areas or around important facilities such as tourist areas, ports, and reservoirs.

[0003] Current water area monitoring faces challenges in identifying buildings encroaching on floodplains and exceeding blue line boundaries. These challenges are easily disrupted by environmental factors such as changes in lighting, shooting angles, and object obstruction. Furthermore, the lack of precise spatial fusion methods combining GIS geofence data and real-time imagery results in insufficient spatial positioning accuracy, hindering accurate identification of illegal encroachment under refined spatial constraints. For slender targets like warning lines, issues such as breakage, offset, and obstruction frequently occur in complex backgrounds. Existing detection models, lacking targeted network structure design and loss function optimization, exhibit low recall rates and high latency, failing to provide high real-time support for safety warnings. The entire process of water area monitoring incidents, from discovery to handling, lacks a closed-loop collaborative mechanism. Each stage operates relatively isolatedly, resulting in inconsistent and incomplete monitoring records of enforcement actions and weak incident tracing capabilities, making it difficult to effectively guarantee enforcement compliance and handling efficiency. The assessment of construction activity lacks intelligent dynamic analysis, making it impossible to accurately identify the intensity of illegal activities through data such as voiceprint features. The fixed frequency and image acquisition rate of drone patrols make it difficult to adaptively adjust according to risk levels, leading to untimely coverage of high-risk areas and hindering the formation of a rapid response system.

[0004] Existing regulatory models lack generalization ability and are prone to accuracy fluctuations due to differences in terrain and environmental changes in various scenarios. Furthermore, model updates rely on complex cloud training processes, making it difficult to achieve efficient online fine-tuning and rapid iteration at the edge. This hinders their ability to flexibly adapt to the complex and ever-changing needs of water area regulatory scenarios and restricts the overall improvement of the level of intelligent regulation.

[0005] No effective solutions have yet been proposed to address the problems in the relevant technologies. Summary of the Invention

[0006] In response to the problems in related technologies, this invention proposes an intelligent water area monitoring and safety early warning system to overcome the aforementioned technical problems existing in the existing related technologies.

[0007] Therefore, the specific technical solution adopted by the present invention is as follows:

[0008] This invention provides an intelligent water area monitoring and safety early warning system, comprising:

[0009] The data acquisition and preprocessing unit is used to acquire image data, warning line data, and audio-visual data of the target water area, and to preprocess the image data, warning line data, and audio-visual data.

[0010] The image data extraction unit is used to extract building outlines from preprocessed image data using an improved illegal building recognition model, and combine it with geofence data to determine illegal encroachment and obtain the judgment result.

[0011] The warning line data identification unit is used to identify the status of the warning line from the preprocessed warning line data using an improved warning line status detection model, and generate a security assessment report.

[0012] The audio-visual data recognition unit is used to identify the pre-processed audio-visual data using audio analysis software to obtain voiceprint data.

[0013] The data fusion unit is used to fuse the judgment results, security assessment report and voiceprint data to build a backtracking correlation chain;

[0014] The monitoring and early warning unit is used to monitor voiceprint data in real time, analyze the activity of voiceprint data, dynamically adjust the acquisition frequency of image data and warning line data, generate illegal building evidence packages, detect illegal building evidence packages, and generate early warning information of different levels.

[0015] Furthermore, the data acquisition and preprocessing unit includes:

[0016] The image acquisition module is used to deploy the pan-tilt camera array to a high point along the target water area to acquire image data of the target water area in real time, and use a multispectral fusion algorithm to eliminate water surface reflection interference in the image data to obtain preprocessed image data.

[0017] The warning line acquisition module is used to inspect the target water area by using a drone swarm equipped with a three-axis stabilized gimbal and real-time dynamic carrier phase difference technology, generate warning line data, and use deep learning technology to enhance the clarity of the warning line data to obtain pre-processed warning line data.

[0018] The acoustic and optical acquisition module is used to deploy the acoustic and optical warning device to the ground of the target water area, identify the acoustic and optical data of the target water area, and use noise reduction technology to denoise the acoustic and optical data to obtain pre-processed acoustic and optical data.

[0019] Furthermore, the process involves using a drone swarm equipped with a three-axis stabilized gimbal and real-time dynamic carrier phase differential technology to inspect the target water area, generating warning line data. Deep learning image processing and analysis techniques are then used to enhance the clarity of the warning line data, resulting in pre-processed warning line data, including:

[0020] The Thiessen polygon algorithm is used to perform dynamic path planning and airspace allocation for drone swarms to ensure that drone swarms can work together, automatically avoid no-fly zones and conflicts, and maximize the patrol range and achieve blind-spot-free coverage.

[0021] Some drone swarms are responsible for performing oblique photography, acquiring images from different angles, and combining them with simultaneous localization and mapping (SLAM) technology to build 3D reality models;

[0022] Another group of drones focused on patrolling the river boundaries of the target waterway, collecting high-definition images of the physical warning line area and the electronic fence area to generate warning line data.

[0023] Furthermore, the image data extraction unit includes:

[0024] The standardization processing module is used to standardize the preprocessed image data to obtain standardized image data.

[0025] The model improvement module is used to increase the weight of buildings in the target water area by using the weighted cross-entropy loss function based on the illegal building identification model, and to use the cross-union ratio weighted loss to weight the loss of buildings in the target water area, so as to obtain an improved illegal building identification model and improve the accuracy of building identification by the improved illegal building identification model.

[0026] The building detection module is used to perform object detection on standardized image data and extract building outline data using an improved illegal building recognition model.

[0027] The boundary determination module is used to calculate the overlap between the building outline data and the specified boundary of the geofence data based on the building outline data and the geofence data, and to determine whether the building exceeds the specified boundary.

[0028] The "Not Exceeding Boundary" module is used to treat buildings as normal buildings if they do not exceed the specified boundaries, and stores them in a pre-built database.

[0029] The boundary exceedance module is used to identify buildings that exceed the prescribed boundaries as illegally encroaching on the building. It uses time-series image analysis technology to compare historically collected image datasets with the illegally encroached buildings, detect the appearance of new buildings or changes in the area of ​​existing buildings, and generate a judgment result.

[0030] Furthermore, the warning line data identification unit includes:

[0031] The model optimization module is used to annotate physical warning lines and electronic fences based on the target detection model framework, and introduce a long strip anchor frame design. Using a data-driven method, the size of the anchor frame is dynamically adjusted by cluster analysis of the warning line data to obtain an improved warning line status detection model to adapt to warning lines and fences of different sizes.

[0032] The error correction module is used to detect the preprocessed warning line data using an improved warning line status detection model, identify abnormal states in the warning line data, and correct the improved warning line status detection model through an error correction mechanism, ultimately outputting a security assessment report.

[0033] Furthermore, the improved warning line status detection model is used to detect the preprocessed warning line data, identify abnormal states in the warning line data, and correct the improved warning line status detection model through an error correction mechanism. The final output security assessment report includes:

[0034] An improved warning line status detection model is used to detect the preprocessed warning line data, identify the abnormal status of each warning line data, and compare the abnormal status of the warning line data with the labeled status tags.

[0035] If the abnormal status of the warning line data does not exceed the labeled status tags, a security assessment report will be generated directly.

[0036] If the abnormal state of the warning line data exceeds the labeled state tags, the error correction mechanism is triggered. The positioning error, classification error, and confidence error in the improved warning line state detection model are calculated using the loss function and corrected and optimized to generate a safety assessment report.

[0037] Furthermore, the formula for calculating the loss function is as follows:

[0038] ;

[0039] In the formula, This represents the value of the loss function; The weighting coefficients representing the positioning error; Indicates the number of grid cells in the feature map; Indicates the indicator function, if the first... iThe grid, the first j Each box is responsible for a specific objective; This represents the weighting coefficients with target confidence error; The weighting coefficients represent the error in the confidence level without a target. The weighting coefficients representing the classification error; This indicates that the box is not responsible for detecting any targets; Indicates the first i The grid, the first j The target confidence level of each prediction box; Both indicate an abnormal state in the warning line data; All of these represent status labels that have already been marked; The conditional category probability representing the abnormal state of the warning line data; This represents the probability of the conditional category for the labeled status tag. c This indicates a category index.

[0040] Furthermore, the data fusion unit includes:

[0041] The data storage module is used to standardize the judgment results, security assessment reports and voiceprint data, and store them in a pre-built database;

[0042] The data traceability module is used to construct a backtracking chain by matching the standardized judgment results, security assessment reports and voiceprint data in the database with timestamps and spatial coordinates.

[0043] Furthermore, the monitoring and early warning unit includes:

[0044] The voiceprint monitoring module is used to collect sound and light data of the target water area in real time through the sound and light warning device, and to identify the sound and light data using audio analysis software to generate voiceprint data.

[0045] The activity level determination module is used to determine the activity level of the acoustic data of the target water area by analyzing the frequency of the real-time acquired acoustic data. When it exceeds the preset activity level, a dynamic adjustment mechanism is triggered to dynamically adjust the acquisition frequency of image data and warning line data to generate an illegal building evidence package.

[0046] The safety early warning module is used to detect illegal building evidence packages using a lightweight model, confirm whether the illegal encroachment on buildings meets the standards for illegal construction, generate detection results, and generate early warning information of different levels based on the detection results.

[0047] Furthermore, by analyzing the frequency of the real-time acquired voiceprint data to determine the activity level of the target water area, and when the activity level exceeds a preset level, a dynamic adjustment mechanism is triggered to dynamically adjust the acquisition frequency of image data and warning line data, generating an illegal building evidence package including:

[0048] Analyze the frequency of real-time acquired acoustic data to determine the activity level of acoustic data in the target water area;

[0049] If the activity level does not exceed the preset level, the activity level of the target water area will continue to be monitored.

[0050] When the preset activity level is exceeded, a dynamic adjustment mechanism is triggered to calculate and output the optimal acquisition frequency of the PTZ camera array and the drone cluster based on the activity change rate of the voiceprint data.

[0051] Based on the adjusted frequency control, the PTZ camera array and the drone cluster synchronously acquire image data and warning line data. The acquired image data and warning line data are then input into the improved illegal building recognition model and the improved warning line status detection model for processing. At the same time, the judgment results and safety assessment reports are extracted.

[0052] The judgment results are aligned with the safety assessment report using timestamps, and spatiotemporal fusion processing is performed to generate an evidence package of illegal buildings.

[0053] The beneficial effects of this invention are as follows:

[0054] 1) This invention achieves a leapfrog improvement in spatial positioning accuracy and scene adaptability through the improved illegal building identification model, while the improved warning line status detection model achieves breakthroughs in small target detection and real-time response capabilities. The collaborative operation increases the illegal building detection rate from 65% to 98%, shortens the average handling cycle from 30 days to 72 hours, and reduces the warning line abnormal response time to less than 2 minutes. It constructs a full-chain governance system from intelligent identification to emergency response to evidence solidification, thereby improving law enforcement efficiency and safety protection level.

[0055] 2) This invention utilizes an improved illegal building identification model that spatially fuses and compares GIS geofence data with real-time images acquired by drones or fixed cameras to automatically determine whether a building encroaches on flood control areas or exceeds the blue line boundary. By introducing a spatial alignment algorithm and sub-pixel-level coordinate registration, the model's identification error in structurally similar areas is controlled within 0.5m, achieving intelligent identification under refined spatial constraints and solving the problem that traditional image recognition is easily affected by lighting, angle, and occlusion.

[0056] 3) This invention utilizes an improved warning line state detection model, introducing orientation-sensitive convolutional kernels and adaptive anchor box design into the network structure, and adding a proportional penalty term for slender targets to the loss function. This significantly improves the model's ability to identify warning line breaks, offsets, and occlusions in complex backgrounds. The recall rate is increased from 80% to 95%, and the detection latency is less than 1.5 seconds, providing a high real-time foundation for subsequent dynamic responses.

[0057] 4) This invention achieves closed-loop management throughout the entire process through a backtracking chain, ensuring that each link is interconnected and works collaboratively. Every step from the discovery of an incident to its handling is precisely monitored and recorded, ensuring the legality and efficiency of law enforcement actions. The backtracking chain not only provides in-depth traceability of incidents but also offers transparent and reliable evidence in case of disputes or reviews, further enhancing the compliance and effectiveness of the law enforcement process.

[0058] 5) This invention utilizes a dynamic adjustment mechanism to automatically determine construction activity levels based on the analysis results of mechanical acoustic signature intensity and activity. When high-intensity violations are detected, the inspection frequency and image acquisition rate of the drone are dynamically adjusted to ensure rapid coverage of high-risk areas, forming an intelligent real-time response system. Relying on a lightweight model incremental evolution mechanism, online fine-tuning can be performed at the edge, and rapid iterative updates can be achieved through cloud-based comparative learning. Attached Figure Description

[0059] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0060] Figure 1 This is a schematic diagram of a water area intelligent monitoring and safety early warning system according to an embodiment of the present invention.

[0061] In the picture:

[0062] 1. Data acquisition and preprocessing unit; 2. Image data extraction unit; 3. Warning line data recognition unit; 4. Audio-visual data recognition unit; 5. Data fusion unit; 6. Monitoring and early warning unit. Detailed Implementation

[0063] To further illustrate the various embodiments, the present invention provides accompanying drawings, which are part of the disclosure of the present invention. These drawings are mainly used to illustrate the embodiments and can be used in conjunction with the relevant descriptions in the specification to explain the operating principles of the embodiments. With reference to these drawings, those skilled in the art should be able to understand other possible implementation methods and the advantages of the present invention.

[0064] According to an embodiment of the present invention, a water area intelligent monitoring and safety early warning system is provided.

[0065] The present invention will now be further described in conjunction with the accompanying drawings and specific embodiments, such as... Figure 1 As shown, the intelligent waterway monitoring and safety early warning system according to an embodiment of the present invention includes:

[0066] The data acquisition and preprocessing unit 1 is used to acquire image data, warning line data and audio-visual data of the target water area, and to preprocess the image data, warning line data and audio-visual data.

[0067] In this optional embodiment, the data acquisition and preprocessing unit includes:

[0068] The image acquisition module is used to deploy the pan-tilt camera array to a high point along the target water area to acquire image data of the target water area in real time, and use a multispectral fusion algorithm to eliminate water surface reflection interference in the image data to obtain preprocessed image data.

[0069] The warning line acquisition module is used to inspect the target water area by using a drone swarm equipped with a three-axis stabilized gimbal and real-time dynamic carrier phase difference technology (RTK), generate warning line data, and use deep learning technology to enhance the clarity of the warning line data to obtain pre-processed warning line data.

[0070] The acoustic and optical acquisition module is used to deploy the acoustic and optical warning device to the ground of the target water area, identify the acoustic and optical data of the target water area, and use noise reduction technology (such as frequency domain filtering and bandpass filtering) to denoise the acoustic and optical data to obtain pre-processed acoustic and optical data.

[0071] In this optional embodiment, the process of inspecting the target water area using a drone swarm equipped with a three-axis stabilized gimbal and real-time dynamic carrier phase differential technology to generate warning line data, and then using deep learning image processing and analysis techniques to enhance the clarity of the warning line data, resulting in preprocessed warning line data including:

[0072] The Voronoi diagram algorithm is used to perform dynamic path planning and airspace allocation for the drone swarm, so as to ensure that the drone swarm can work together, automatically avoid no-fly zones and conflicts, and maximize the patrol range and achieve blind-spot-free coverage.

[0073] Some drone swarms are responsible for performing oblique photography, acquiring images from different angles, and combining them with simultaneous localization and mapping (SLAM) technologies to build 3D reality models;

[0074] Another group of drones focused on patrolling the river boundaries of the target waterway, collecting high-definition images of the physical warning line area and the electronic fence area to generate warning line data.

[0075] Specifically, a 40-megapixel PTZ camera array is deployed at high points along the target waterway. Each PTZ camera has a horizontal rotation range of 355° and a vertical rotation range of -45° to 90°, supports 30x optical zoom and AI tracking, and has a coverage radius of 3 kilometers of water. The cameras capture image data of the waterway from a high-altitude perspective and process the images using a multispectral fusion algorithm (visible light and near-infrared) to eliminate water surface reflection interference and enhance the ability to identify the outlines of buildings behind the embankment bushes.

[0076] Inspection missions are carried out using a drone swarm system. Each drone is equipped with a three-axis stabilization gimbal and a high-precision RTK module to ensure high-precision positioning and stable image acquisition during flight. The drones fly at an altitude of 100 meters, achieving an image resolution of 2cm. Using oblique photogrammetry combined with Simultaneous Localization and Mapping (SLAM) technology, a 3D reality model of the river's perimeter is generated, providing accurate spatial data support. Another group of drones focuses on inspecting the waterway boundaries, specifically monitoring physical warning lines (such as damaged guardrails or displaced buoys) and electronic fences (such as broken infrared fences). The drone swarm system uses an improved Voronoi diagram algorithm for dynamic path planning and airspace allocation, ensuring that 50 drones can work collaboratively, automatically avoiding no-fly zones and optimizing inspection routes to achieve blind-spot-free coverage, significantly improving inspection efficiency.

[0077] The system utilizes audible and visual warning devices to monitor sound and light data within waterways. An integrated AI audio analysis module can identify noise signals during construction, particularly the characteristic sound signatures of excavators. By processing the sound and light data and employing denoising techniques (such as frequency domain filtering and bandpass filtering), unnecessary noise is removed, and useful sound and light data is extracted, further enhancing the monitoring capabilities of waterway activities. The combination of sound and light data with image and warning line data enables the monitoring system to provide real-time warnings from multiple dimensions, ensuring accurate identification of violations within the waterways.

[0078] Image data extraction unit 2 is used to extract building outlines from preprocessed image data using an improved illegal building recognition model, and combine it with geofencing data (GIS) to determine illegal encroachment and obtain the determination result;

[0079] In this optional embodiment, the image data extraction unit includes:

[0080] The standardization processing module is used to perform standardization processing (denoising, enhancement, and size standardization) on the preprocessed image data to obtain standardized image data.

[0081] The model improvement module is used to increase the weight of buildings in the target water area by using the weighted cross-entropy loss function based on the illegal building identification model (Mask R-CNN architecture), and to use the intersection-over-union weighted loss (IoU weighted loss) to weight the loss of buildings in the target water area, so as to obtain an improved illegal building identification model and improve the accuracy of building identification by the improved illegal building identification model.

[0082] The building detection module is used to perform object detection on standardized image data and extract building outline data using an improved illegal building recognition model.

[0083] The boundary determination module is used to calculate the overlap between the building outline data and the specified boundary of the geofence data based on the building outline data and the geofence data, and to determine whether the building exceeds the specified boundary.

[0084] The "Not Exceeding Boundary" module is used to treat buildings as normal buildings if they do not exceed the specified boundaries, and stores them in a pre-built database.

[0085] The boundary exceedance module is used to identify buildings that exceed the prescribed boundaries as illegally encroaching on the building. It uses time-series image analysis technology to compare historically collected image datasets with the illegally encroached buildings, detect the appearance of new buildings or changes in the area of ​​existing buildings, and generate a judgment result.

[0086] Specifically, the preprocessed image data undergoes standardization processing through methods such as denoising, enhancement, and size standardization to ensure that the image data meets a unified processing standard. The denoising step removes noise from the image (such as water reflections and shadows), the enhancement module improves the recognizability of buildings through contrast enhancement and edge enhancement, and size standardization ensures that the image meets consistent size requirements when input into subsequent processing models.

[0087] Building detection is performed using an illegal building recognition model (such as the Mask R-CNN architecture). Mask R-CNN is a deep learning framework that generates high-quality segmentation masks for each target (building) in an image. Improvements were made to Mask R-CNN, particularly in the loss function. By introducing a weighted cross-entropy loss function, buildings within the target water area are given higher weights, making the model more focused on identifying building regions. Intersection over Union (IoU) weighted loss is used to optimize building boundary extraction, improving the accuracy of building contour recognition. The model effectively improves building detection accuracy, especially in complex environments (such as water reflections and vegetation occlusion). After model improvements, the improved illegal building recognition model is used to detect objects in standardized image data, extracting building contour data from the water area. The building contours can be accurately separated from the image, providing crucial data for subsequent encroachment determination.

[0088] The extracted building outline data is compared with geofence data (GIS data). The system calculates the overlap of each building outline to check if the building crosses the prescribed boundaries. If the building outline overlaps with the boundaries of water areas such as river blue lines or embankment lines, the system determines that the building exceeds the prescribed boundaries. Ensuring accurate matching between buildings and geofences is a crucial step in determining whether a building is illegally encroaching on the territory.

[0089] The system can automatically detect whether buildings encroach on water areas or are illegally constructed, ensuring the accuracy and real-time nature of monitoring. If a building does not exceed the prescribed boundaries, the "not exceeding boundaries" module will treat it as a normal building and store it in a pre-built database. Conversely, if a building exceeds the prescribed boundaries, the "exceeding boundaries" module will treat it as an illegally encroaching building and conduct further analysis. Using time-series image analysis technology, historically acquired image datasets are compared with images of current illegal buildings to detect new buildings or changes in the area of ​​existing buildings.

[0090] In practical application, a group of 40-megapixel PTZ cameras were deployed in a certain river area to collect image data in real time, assuming an image resolution of 2 centimeters. During image processing, a standardization module was used to remove interference from water reflections and ripples, enhancing the image's ability to recognize building outlines. The image size was adjusted to 512x512 pixels to meet the input requirements of subsequent models. The model improvement module used an improved model based on Mask R-CNN, employing a weighted cross-entropy loss function for focused identification of building areas. The system calculated the overlap between these outline data and the geofence data for the area, checking whether buildings crossed the designated blue line of the river channel. If a building exceeded the designated boundary, the system marked it as "illegal encroachment" and, through an boundary exceedance module, compared it with historical image data to detect newly added buildings or changes to existing buildings.

[0091] Warning line data identification unit 3 is used to identify the status of the warning line from the preprocessed warning line data using an improved warning line status detection model, and generate a security assessment report;

[0092] In this optional embodiment, the warning line data recognition unit includes:

[0093] The model optimization module is used to annotate physical warning lines and electronic fences based on the target detection model (YOLOv8) framework, and introduces a long strip anchor frame design. Using a data-driven method, the anchor frame size is dynamically adjusted by cluster analysis (k-means clustering algorithm) of the warning line data to obtain an improved warning line status detection model to adapt to warning lines and fences of different sizes.

[0094] The error correction module is used to detect the preprocessed warning line data using an improved warning line status detection model, identify abnormal states in the warning line data, and correct the improved warning line status detection model through an error correction mechanism, ultimately outputting a security assessment report.

[0095] In this optional embodiment, the improved warning line status detection model is used to detect the preprocessed warning line data, identify abnormal states of the warning line data, and correct the improved warning line status detection model through an error correction mechanism. The final output security assessment report includes:

[0096] An improved warning line status detection model is used to detect the preprocessed warning line data, identify the abnormal status of each warning line data, and compare the abnormal status of the warning line data with the labeled status tags.

[0097] If the abnormal status of the warning line data does not exceed the labeled status tags, a security assessment report will be generated directly.

[0098] If the abnormal state of the warning line data exceeds the labeled state tags, the error correction mechanism is triggered. The positioning error, classification error, and confidence error in the improved warning line state detection model are calculated using the loss function and corrected and optimized to generate a safety assessment report.

[0099] In this optional embodiment, the formula for calculating the loss function is:

[0100] ;

[0101] In the formula, This represents the value of the loss function; The weighting coefficients representing the positioning error; Indicates the number of grid cells in the feature map; Indicates the indicator function, if the first... i The grid, the first j If a box is responsible for a certain target, it is 1; otherwise, it is 0. This represents the weighting coefficients with target confidence error; The weighting coefficients represent the error in the confidence level without a target. The weighting coefficients representing the classification error; The value is 1 if the box is not responsible for detecting any target, and 0 otherwise. Indicates the first i The grid, the first j The target confidence level of each prediction box; Both indicate an abnormal state in the warning line data; All of these represent status labels that have already been marked; The conditional category probability representing the abnormal state of the warning line data; This represents the probability of the conditional category for the labeled status tag. c Represents the category index, traversing all categories ( classes (Indicates the total number of categories).

[0102] Specifically, the YOLOv8 framework, based on object detection, is employed, offering high accuracy and real-time performance in image data processing, enabling efficient annotation of physical warning lines and electronic fences. The YOLOv8 framework itself excels at detecting small objects (such as warning lines); therefore, a long, narrow anchor frame design is specifically introduced for warning line state detection, which helps improve the recognition of slender objects (such as warning lines and fences). To accommodate warning lines or fences of different sizes, the system uses k-means clustering for scale analysis. Warning line data of different sizes can be automatically grouped, and the anchor frame size is dynamically adjusted based on each group to improve the model's adaptability to warning lines of varying sizes. The error correction module plays a crucial role. The system utilizes an improved warning line state detection model to detect pre-processed warning line data, identifying abnormal states. For example, broken or offset warning lines, or the presence of physical obstacles, are all considered abnormal states. Upon detecting these abnormal states, the system compares them with the labeled status tags to ensure the correctness and consistency of the identification results. If the detected abnormal state does not exceed the labeled status tags (i.e., falls within the expected normal abnormal range), the system directly generates a safety assessment report, recording the status of the warning line and indicating whether any potential problems exist. If the abnormal state of the warning line data exceeds the labeled status tags, i.e., the detection result is inconsistent with expectations, the error correction mechanism is triggered. The error correction mechanism evaluates the errors in the improved warning line status detection model by calculating a loss function. These errors include positioning error (i.e., the positioning deviation of the warning line in the image), classification error (i.e., the probability of misclassifying the warning line), and confidence error (i.e., the model's confidence error in the presence or absence of the warning line). Through the calculation of these errors, the model can be corrected and optimized according to the actual situation, improving the accuracy of warning line status detection. After correction, the system generates a final safety assessment report, which details the status of the warning line and whether any abnormalities or potential safety hazards exist, ensuring the real-time and accurate management of water safety.

[0103] In practical applications, assuming 30 monitoring areas are deployed within the target waterway, each with a warning line approximately 200 meters long, comprising both physical and electronic fences, the system uses a model optimization module to process the warning line image data using the YOLOv8 framework and employs a k-means clustering algorithm to analyze the scale of the warning line data. If a warning line in a certain area is broken, misaligned, or incompletely covered, the improved model can accurately detect these anomalies, using elongated anchor frames to adapt to different warning line lengths and shapes. For example, if a warning line in a certain area has a break exceeding 5 meters, the system will compare this data with the already labeled status tags. If this anomaly is within the labeled allowable range (e.g., a break less than 5 meters is considered acceptable error), the system will directly generate a "safety" assessment report, indicating that the warning line condition is normal. Conversely, if an abnormal condition (such as a warning line break exceeding 5 meters) fails to meet the expected tolerance range, the system will trigger an error correction mechanism. This mechanism will analyze the model for errors and adjust it based on positioning errors, classification errors, and confidence errors to improve detection accuracy. After correction, the generated safety assessment report will detail the warning line breakage and provide repair recommendations.

[0104] The audio-visual data recognition unit 4 is used to identify the pre-processed audio-visual data using audio analysis software (such as an integrated AI audio analysis module) to obtain voiceprint data.

[0105] Specifically, audio analysis software (such as an AI audio analysis module) is used to identify the preprocessed audio-visual data. By analyzing the audio information in the audio-visual data, voiceprint data is extracted to identify specific sound events or behaviors. For example, the system can identify construction noise, the sound of machinery operation, and even the sounds of illegal intruders. The AI ​​audio analysis module, based on deep learning models such as convolutional neural networks (CNNs) or recurrent neural networks (RNNs), can identify and classify different types of audio signals. In this way, the system can monitor illegal noise behavior around waterways in real time, providing early warnings for safety management. Data stored in the database is traced and correlated using timestamps and spatial coordinate information.

[0106] In practical applications, suppose 20 audio-visual warning devices are deployed around a river area, each capable of monitoring audio-visual data within a 500-meter radius. The system uses an AI audio analysis module to analyze the audio signals from these devices in real time, identifying noise from construction (such as excavator operation) and other violations. If a device detects abnormal noise (such as high-intensity excavator operation), the system will use sound recognition technology to mark it as "illegal construction noise" and generate corresponding voiceprint data. This voiceprint data is then stored in a database.

[0107] Data fusion unit 5 is used to fuse the judgment results, security assessment report and voiceprint data to build a retrospective association chain;

[0108] In this optional embodiment, the data fusion unit includes:

[0109] The data storage module is used to standardize the judgment results, security assessment reports and voiceprint data, and store them in a pre-built database;

[0110] The data traceability module is used to construct a backtracking chain by matching the standardized judgment results, security assessment reports and voiceprint data in the database with timestamps and spatial coordinates.

[0111] Specifically, the judgment results, security assessment reports, and voiceprint data are standardized and stored in a pre-built database. The purpose of standardization is to ensure that different types of data can be stored compatiblely and maintain consistency in subsequent analysis and queries. Judgment results and security assessment reports are typically static data generated at specific points in time, and their storage can be ordered chronologically. Voiceprint data, on the other hand, is usually dynamically collected audio features, requiring feature extraction and formatting before storage. Standardization steps include unifying timestamp formats, coordinate formats, and data identifiers, enabling seamless integration of different data types into the database.

[0112] The standardized data stored in the database is linked to build a backtracking chain. The stored data is sorted based on timestamps to ensure consistency of different data types over time.

[0113] Voiceprint data at a certain moment (such as audio recordings of construction sounds) will be matched with safety assessment reports generated at the same time (such as abnormal status of warning lines).

[0114] Further matching is performed based on spatial coordinates (such as GPS coordinates or sensor installation locations), associating different data types by geographical location. If a device (such as an audible and visual alarm device) captures construction noise at a specific location, the system will link this event with the corresponding area's safety assessment report and judgment results through spatial coordinates, constructing a complete traceability chain. Through this data fusion and association, data from different sources will form a layered, fully aligned traceability chain in time and space.

[0115] The monitoring and early warning unit 6 is used to monitor voiceprint data in real time, analyze the activity of voiceprint data, dynamically adjust the acquisition frequency of image data and warning line data, generate illegal building evidence packages, detect illegal building evidence packages, and generate early warning information of different levels.

[0116] In this optional embodiment, the monitoring and early warning unit includes:

[0117] The voiceprint monitoring module is used to collect sound and light data of the target water area in real time through the sound and light warning device, and to identify the sound and light data using audio analysis software to generate voiceprint data.

[0118] The activity level determination module is used to determine the activity level of the voiceprint data of the target water area by analyzing the frequency of the voiceprint data acquired in real time. When it exceeds the preset activity level (such as illegal construction site, or repeated confirmation of illegal behavior), a dynamic adjustment mechanism is triggered to dynamically adjust the acquisition frequency of image data and warning line data and generate illegal building evidence package.

[0119] The safety early warning module is used to detect illegal building evidence packages using a lightweight model (YOLO-NAS), confirm whether the illegal encroachment on buildings meets the standards for illegal construction, generate detection results, and generate early warning information of different levels based on the detection results.

[0120] In this optional embodiment, the step of analyzing the frequency of real-time acquired voiceprint data to determine the activity level of the target water area, and triggering a dynamic adjustment mechanism when the activity level exceeds a preset level, dynamically adjusting the acquisition frequency of image data and warning line data, and generating an illegal building evidence package includes:

[0121] Analyze the frequency of real-time acquired acoustic data to determine the activity level of acoustic data in the target water area;

[0122] If the activity level does not exceed the preset level, the activity level of the target water area will continue to be monitored.

[0123] When the preset activity level is exceeded, a dynamic adjustment mechanism is triggered to calculate and output the optimal acquisition frequency of the PTZ camera array and the drone cluster based on the activity change rate of the voiceprint data.

[0124] Based on the adjusted frequency control, the PTZ camera array and the drone cluster synchronously acquire image data and warning line data. The acquired image data and warning line data are then input into the improved illegal building recognition model and the improved warning line status detection model for processing. At the same time, the judgment results and safety assessment reports are extracted.

[0125] The judgment results are aligned with the safety assessment report using timestamps, and spatiotemporal fusion processing is performed to generate an evidence package of illegal buildings.

[0126] Specifically, ambient audio is collected in real time using sound and light warning devices deployed around the target water area. The collected data is fed into AI audio analysis software for feature extraction and recognition, outputting high-dimensional voiceprint data to identify potential abnormal sound sources, such as excavator noise, metal cutting sounds, and explosions. By continuously analyzing the voiceprint data characteristics, spectral structure, and confidence level, it is possible to quickly determine whether the sound has obvious construction or interference characteristics, laying the foundation for subsequent activity analysis. Frequency statistics and time series modeling are performed on the voiceprint data, analyzing the number of abnormal voiceprint events occurring per unit time according to a set time window (e.g., every 5 minutes).

[0127] When the voiceprint activity level does not exceed the set threshold (e.g., 10 times / 5 minutes), the system maintains the current monitoring frequency. If the activity level significantly increases and continues to exceed the preset threshold (e.g., 15 times / 5 minutes, or a growth rate exceeding 30%), the system determines that the target water area may be in a high-risk state of violation and immediately triggers a dynamic adjustment mechanism. This mechanism will dynamically calculate the optimal acquisition frequency of the PTZ camera and drone cluster based on the activity change rate using a calculation model (e.g., exponential smoothing + adaptive step size algorithm), for example, adjusting from once every 15 minutes to once every 3 minutes.

[0128] The PTZ camera array will automatically adjust its angle and increase the image capture frequency, while the drone will take off and plan its flight path to conduct low-altitude patrols of key areas, collecting high-resolution image data and warning line data. After collection, this data will be input into two optimized models: image data will be fed into an improved illegal building recognition model (improved Mask R-CNN) to extract building outlines and determine whether there is encroachment; while warning line data will be processed by a warning line status detection model (improved YOLOv8) to identify whether there is damage, movement, or failure. The model output includes the judgment result and a security assessment report, which are fused through a unified timestamp alignment mechanism. Combining the spatial coordinates of the image and voiceprint sources, a package of illegal building evidence is formed, which includes image evidence, voiceprint records, warning line status, equipment location, time information, and AI model analysis conclusions.

[0129] The lightweight model YOLO-NAS is invoked to quickly re-confirm the image content in the illegal building evidence package and determine whether the building meets the "illegal encroachment" standard, such as construction beyond the boundary, illegal building area exceeding the minimum identification threshold (e.g., 5 square meters), or whether it is located in a prohibited building area.

[0130] The YOLO-NAS model balances speed and accuracy, making it suitable for the rapid inference needs of edge devices. Based on the matching degree and confidence level of the model output, the system classifies early warning information into different levels. For example, Level 1 events (such as encroachment on flood diversion areas) automatically trigger drones to approach and collect evidence, and synchronize with law enforcement departments; Level 2 events (such as damage to warning lines) dispatch maintenance robots to handle the situation on-site and send voice alerts. Finally, the early warning information is pushed to management personnel through a network platform or SMS, and hotspots are visually marked on the map system to assist in decision-making.

[0131] In practical applications, assuming 15 sets of audio-visual equipment, 4 sets of pan-tilt cameras (360° horizontal rotation, 30x zoom), and a 4-drone formation (cruising speed 15m / s, flight time 40 minutes) are deployed in a 10 square kilometer river and lake area, a drone swarm of 4 drones is used. The system's preset soundprint activity threshold is 10 times / 5 minutes. During the time window of 14:55-15:00 on November 20, 2025, device number 9 continuously detected 16 construction sounds (including 3 heavy impacts, 6 cutting sounds, and 7 track friction sounds), exceeding the activity threshold by 60%. The system triggers a dynamic acquisition mechanism, adjusting the image acquisition frequency from 15 minutes / time to 3 minutes / time. Drones simultaneously take off to patrol a 300-meter radius area around device number 9. The images captured by the pan-tilt equipment, using an improved Mask R-CNN model, identify the outline of a suspected illegal building, with a boundary exceeding the geofence by 3.2 meters and an area of ​​56 square meters. The warning line model identifies voltage breakpoints in the electronic fence, suspected of being intentionally damaged. Voiceprint, image, and warning line data are fused together using the timestamp "2025-11-20 15:03:00" to generate a complete illegal building evidence package. The YOLO-NAS model further confirms that it is a "medium-sized illegal building" with a confidence level of 88.6%. The system generates a "Level 2 event warning" and pushes it to the management platform. At the same time, the area is highlighted on the map as a "high-risk construction hotspot".

[0132] To verify the beneficial effects of the improved illegal building identification model and the improved warning line status detection model described in this invention, a typical river and lake water area was selected as the experimental scenario, with a monitored area of ​​approximately 10 square kilometers. Within the area, 120 illegal building samples and 68 warning line deployment points were pre-marked and confirmed. Simultaneously, 15 sets of audio-visual warning devices, 4 sets of PTZ camera arrays, and 1 drone swarm were deployed. The experiment lasted for 90 consecutive days. Under the premise of consistent hardware conditions, acquisition resolution, and data scale, traditional fixed-frequency patrols and general target detection models were used as control schemes. The overall scheme of this invention, which employs voiceprint-driven dynamic scheduling, the improved illegal building identification model, and the improved warning line status detection model, was used as the experimental scheme for comparative testing. The results show that, in terms of illegal building identification, the control scheme identified only 78 illegal buildings during the test period, with a detection rate of approximately 65%. In contrast, the present invention, through voiceprint activity-triggered high-frequency image acquisition, multi-view fusion, and spatial fence constraints, identified a total of 118 illegal buildings, increasing the detection rate to approximately 98%. Furthermore, since the system automatically generates an evidence package for illegal buildings after identification, including image evidence, voiceprint records, and warning line status, law enforcement agencies no longer need to conduct repeated field investigations. The average time from the occurrence of a violation to the formation of complete and enforceable evidence is shortened from approximately 30 days in the traditional method to 72 hours.

[0133] In terms of warning line detection, a comparison was made based on 3,000 test images from the same batch containing complex scenes such as breakage, offset, and occlusion. The recall rate of traditional detection models in complex backgrounds was about 80%, while the present invention, by introducing orientation-sensitive convolutional kernels (CNN) into the network structure, adaptive anchor box design, and adding a proportional penalty term for slender targets to the loss function, enhances the recognition ability of warning line breakage and offset, and improves the recall rate to about 95%. At the same time, the single-frame inference time on the edge computing device is controlled within 300 milliseconds, the overall detection latency from data acquisition to anomaly judgment and warning output is less than 1.5 seconds, and the warning line anomaly response time is consistently less than 2 minutes. The above experimental results were obtained under a unified test environment and evaluation indicators, which fully demonstrate that the present invention has verifiable technical advantages in terms of illegal building detection efficiency, real-time response capability to warning line anomalies, and the whole-chain governance effect from intelligent identification, emergency response to evidence solidification, as shown in Table 1.

[0134] Table 1. Comparison of key performance characteristics between the present invention and existing technologies.

[0135]

[0136] In summary, by utilizing the technical solution of this invention, sound and light alarm devices are used to collect sound and light data of the target water area in real time. Audio analysis software is then used to identify the data and generate voiceprint data to determine the presence of violations. Combining image recognition and GIS geofence data, an improved model is used to extract building outlines, and boundary judgment is used to determine whether buildings encroach on water areas or are illegally constructed, generating relevant safety assessment reports. The judgment results, voiceprint data, and safety assessment reports are standardized, and a retrospective association chain is generated through spatiotemporal fusion to ensure that different data sources can match each other and form a complete historical record, supporting efficient data tracing and event analysis. Through voiceprint data activity analysis, the activity level of the water area is determined in real time. When abnormal activity is detected, a dynamic adjustment mechanism is triggered to optimize the collection frequency of image and warning line data, so as to promptly generate illegal building evidence packages and provide rapid early warnings. Finally, the system uses a lightweight model (such as YOLO-NAS) to detect the illegal building evidence packages, confirm the existence of illegal buildings, and generate different levels of early warning information based on confidence levels.

[0137] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A smart waterway monitoring and safety early warning system, characterized in that, include: The data acquisition and preprocessing unit is used to acquire image data, warning line data, and audio-visual data of the target water area, and to preprocess the image data, warning line data, and audio-visual data. The image data extraction unit is used to extract building outlines from preprocessed image data using an improved illegal building recognition model, and combine it with geofence data to determine illegal encroachment and obtain the judgment result. A warning line data identification unit is used to identify the status of warning lines from preprocessed warning line data using an improved warning line status detection model, and generate a security assessment report; including: The model optimization module is used to annotate physical warning lines and electronic fences based on the target detection model framework, and introduce a long strip anchor frame design. Using a data-driven method, the size of the anchor frame is dynamically adjusted by cluster analysis of the warning line data to obtain an improved warning line status detection model to adapt to warning lines and fences of different sizes. The error correction module is used to detect the preprocessed warning line data using an improved warning line status detection model, identify abnormal states of the warning line data, and correct the improved warning line status detection model through an error correction mechanism, ultimately outputting a security assessment report. The audio-visual data recognition unit is used to identify the pre-processed audio-visual data using audio analysis software to obtain voiceprint data. The data fusion unit is used to fuse the judgment results, security assessment report and voiceprint data to build a backtracking correlation chain; The monitoring and early warning unit is used to monitor voiceprint data in real time, analyze the activity level of the voiceprint data, dynamically adjust the acquisition frequency of image data and warning line data, generate evidence packages of illegal buildings, and detect the evidence packages of illegal buildings to generate early warning information of different levels; including: The voiceprint monitoring module is used to collect sound and light data of the target water area in real time through sound and light warning devices, and to identify the sound and light data using audio analysis software to generate voiceprint data. The activity level determination module is used to determine the activity level of the acoustic data of the target water area by analyzing the frequency of the real-time acquired acoustic data. When it exceeds the preset activity level, a dynamic adjustment mechanism is triggered to dynamically adjust the acquisition frequency of image data and warning line data to generate an illegal building evidence package. The safety early warning module is used to detect illegal building evidence packages using a lightweight model, confirm whether the illegal encroachment on buildings meets the standards for illegal construction, generate detection results, and generate early warning information of different levels based on the detection results.

2. The intelligent waterway monitoring and safety early warning system according to claim 1, characterized in that, The data acquisition and preprocessing unit includes: The image acquisition module is used to deploy the pan-tilt camera array to a high point along the target water area to acquire image data of the target water area in real time, and use a multispectral fusion algorithm to eliminate water surface reflection interference in the image data to obtain preprocessed image data. The warning line acquisition module is used to inspect the target water area by using a drone swarm equipped with a three-axis stabilized gimbal and real-time dynamic carrier phase difference technology, generate warning line data, and use deep learning technology to enhance the clarity of the warning line data to obtain pre-processed warning line data. The acoustic and optical acquisition module is used to deploy the acoustic and optical warning device to the ground of the target water area, identify the acoustic and optical data of the target water area, and use noise reduction technology to denoise the acoustic and optical data to obtain pre-processed acoustic and optical data.

3. The intelligent waterway monitoring and safety early warning system according to claim 2, characterized in that, The process involves using a drone swarm equipped with a three-axis stabilized gimbal and real-time dynamic carrier phase differential technology to inspect the target water area, generating warning line data. Deep learning image processing and analysis techniques are then used to enhance the clarity of the warning line data, resulting in pre-processed warning line data, including: The Thiessen polygon algorithm is used to perform dynamic path planning and airspace allocation for drone swarms to ensure that drone swarms can work together, automatically avoid no-fly zones and conflicts, and maximize the patrol range and achieve blind-spot-free coverage. Some drone swarms are responsible for performing oblique photography, acquiring images from different angles, and combining them with simultaneous localization and mapping (SLAM) technology to build 3D reality models; Another group of drones focused on patrolling the river boundaries of the target waterway, collecting high-definition images of the physical warning line area and the electronic fence area to generate warning line data.

4. The intelligent waterway monitoring and safety early warning system according to claim 1, characterized in that, The image data extraction unit includes: The standardization processing module is used to standardize the preprocessed image data to obtain standardized image data. The model improvement module is used to increase the weight of buildings in the target water area by using the weighted cross-entropy loss function based on the illegal building identification model, and to use the cross-union ratio weighted loss to weight the loss of buildings in the target water area, so as to obtain an improved illegal building identification model and improve the accuracy of building identification by the improved illegal building identification model. The building detection module is used to perform object detection on standardized image data and extract building outline data using an improved illegal building recognition model. The boundary determination module is used to calculate the overlap between the building outline data and the specified boundary of the geofence data based on the building outline data and the geofence data, and to determine whether the building exceeds the specified boundary. The "Not Exceeding Boundary" module is used to treat buildings as normal buildings if they do not exceed the specified boundaries, and stores them in a pre-built database. The boundary exceedance module is used to identify buildings that exceed the prescribed boundaries as illegally encroaching on the building. It uses time-series image analysis technology to compare historically collected image datasets with the illegally encroached buildings, detect the appearance of new buildings or changes in the area of ​​existing buildings, and generate a judgment result.

5. The intelligent waterway monitoring and safety early warning system according to claim 1, characterized in that, The improved warning line status detection model is used to detect preprocessed warning line data, identify abnormal states in the warning line data, and correct the improved warning line status detection model through an error correction mechanism. The final output security assessment report includes: An improved warning line status detection model is used to detect the preprocessed warning line data, identify the abnormal status of each warning line data, and compare the abnormal status of the warning line data with the labeled status tags. If the abnormal status of the warning line data does not exceed the labeled status tags, a security assessment report will be generated directly. If the abnormal state of the warning line data exceeds the labeled state tags, the error correction mechanism is triggered. The positioning error, classification error, and confidence error in the improved warning line state detection model are calculated using the loss function and corrected and optimized to generate a safety assessment report.

6. The intelligent waterway monitoring and safety early warning system according to claim 5, characterized in that, The formula for calculating the loss function is as follows: ； In the formula, This represents the value of the loss function; The weighting coefficients representing the positioning error; Indicates the number of grid cells in the feature map; Indicates the indicator function, if the first... i The grid, the first j Each box is responsible for a specific objective; This represents the weighting coefficients with target confidence error; The weighting coefficients represent the error in the confidence level without a target. The weighting coefficients representing the classification error; This indicates that the box is not responsible for detecting any targets; Indicates the first i The grid, the first j The target confidence level of each prediction box; Both indicate an abnormal state in the warning line data; All of these represent status labels that have already been marked; The conditional category probability representing the abnormal state of the warning line data; This represents the probability of the conditional category for the labeled status tag. c This indicates a category index.

7. The intelligent waterway monitoring and safety early warning system according to claim 1, characterized in that, The data fusion unit includes: The data storage module is used to standardize the judgment results, security assessment reports and voiceprint data, and store them in a pre-built database; The data traceability module is used to construct a backtracking chain by matching the standardized judgment results, security assessment reports and voiceprint data in the database with timestamps and spatial coordinates.

8. The intelligent waterway monitoring and safety early warning system according to claim 7, characterized in that, The process involves analyzing the frequency of real-time acquired voiceprint data to determine the activity level of the target water area. When the activity level exceeds a preset threshold, a dynamic adjustment mechanism is triggered to dynamically adjust the acquisition frequency of image data and warning line data, generating an illegal building evidence package that includes: Analyze the frequency of real-time acquired acoustic data to determine the activity level of acoustic data in the target water area; If the activity level does not exceed the preset level, the activity level of the target water area will continue to be monitored. When the preset activity level is exceeded, a dynamic adjustment mechanism is triggered to calculate and output the optimal acquisition frequency of the PTZ camera array and the drone cluster based on the activity change rate of the voiceprint data. Based on the adjusted frequency control, the PTZ camera array and the drone cluster synchronously acquire image data and warning line data. The acquired image data and warning line data are then input into the improved illegal building recognition model and the improved warning line status detection model for processing. At the same time, the judgment results and safety assessment reports are extracted. The judgment results are aligned with the safety assessment report using timestamps, and spatiotemporal fusion processing is performed to generate an evidence package of illegal buildings.

Images