An underwater scene change detection and abnormality early warning method and system based on periodic acousto-optic scanning, and a storage medium

Abstract

The application discloses a kind of based on periodic acoustooptic scanning underwater scene change detection and abnormal early warning method and system, storage medium, the application constructs five-link closed loop self-evolution architecture: the reference scene model of containing sonar three-dimensional geometry, optical texture, target semantic annotation three layer alignment information definition normality underwater;Acoustic priority is connected to space registration to subsequent periodic acoustooptic data, and three-dimensional change detection fusion is generated to candidate set in parallel target level, feature level, pixel level three scales, and pixel level only uses acoustic echo to realize turbidity optical failure bottom line.This application solves four major pain points that temporary disturbance and normal evolution cannot be distinguished in long-term underwater periodic monitoring scene, high turbidity environment detection failure, false alarm flooding, reference model long-term distortion, greatly reduces false alarm rate and report rate, realizes long-term adaptive underwater security monitoring in complex water environment.

Description

Technical Field

[0001] This invention belongs to the fields of underwater monitoring, underwater security, and multimodal sensor fusion technology. Specifically, it relates to a method and supporting system and storage medium for underwater scene change detection, false alarm suppression, anomaly classification early warning and baseline model adaptive evolution based on acoustic and optical dual-modal periodic scanning for long-cycle underwater inspection scenarios such as ports, submarine pipelines, and marine ranches. Background Technology

[0002] Long-term, periodic underwater monitoring faces inherent environmental constraints: water significantly attenuates electromagnetic waves, and optical imaging is limited by turbidity and illumination, resulting in extremely short visibility. Sonar, while not limited by optical visibility, lacks texture and semantic information; both types of single sensors have inherent limitations. Existing underwater change detection technologies fall into three categories, all of which suffer from the core flaw of failing to resolve the conflict between "normal evolution and anomaly detection" in long-term monitoring.

[0003] Pure acoustic single-modal change detection: It only compares two-dimensional acoustic images or three-dimensional acoustic point cloud geometry, and cannot obtain target texture and semantic information. It can only identify shape changes and cannot distinguish target types such as fishing nets, shipwrecks, and marine life. Moreover, it lacks an optical cross-validation mechanism, and fish and suspended objects are prone to generating a large number of false alarms. The baseline model only contains geometric occupancy information and lacks texture and semantic support.

[0004] Acoustic 3D model real-time update detection: By comparing geometric changes through the occupied grid, but using a real-time update strategy that updates the baseline upon single detection, temporarily moored ships, short-term construction equipment, and seasonal biomes will be permanently written into the baseline model. After long-term operation, the baseline is contaminated by temporary objects, loses the ability to define a normal baseline, and subsequently generates a large number of false alarms due to disappearance.

[0005] Real-time target detection using audio-visual fusion: It only solves the problem of target recognition in a single frame image, does not support long-term change comparison across cycles, and lacks a complete closed-loop mechanism for benchmark scene modeling, temporal filtering, dual-modal cross-confirmation, and persistent model updates. It cannot be adapted to long-term underwater inspection operations with cycles of hours / days / weeks.

[0006] Existing patents US8942062, CN118363089A, and US7990804 do not propose a closed-loop architecture consisting of five stages: hierarchical modeling, multi-scale detection, dual-modal decoupling verification, temporal transient filtering, and N persistent updates for long-term periodic monitoring. They cannot quantitatively distinguish between legitimate temporary underwater disturbances and persistent anomalies that require early warning. Long-term monitoring scenarios have high rates of missed reports and false alarms. The benchmark model cannot adaptively follow the natural evolution of the scenario, making it difficult to balance anomaly sensitivity and benchmark stability. Summary of the Invention

[0007] To address the following issues in existing technologies for long-term underwater periodic monitoring scenarios: poor robustness of single-modal detection; lack of fallback detection links when turbidity increases due to optical failure; lack of a cross-validation mechanism for heterogeneous acoustic and optical modes, leading to rampant false alarms; lack of hierarchical benchmark scene definition, making it impossible to distinguish target semantics or quantify the level of anomaly risk based solely on geometric information; lack of temporal filtering mechanism, resulting in the continuous generation of invalid change candidates by instantaneous interferences such as fish, water flow, and suspended matter, significantly increasing computational load and the probability of false alarms; and real-time updates to the benchmark model leading to temporary object contamination of the benchmark, making it impossible to distinguish between short-term disturbances and long-term scene evolution, resulting in a continuous decline in long-term monitoring accuracy.

[0008] This invention fully constructs a closed-loop technical system of "benchmark modeling - multi-scale change detection - spatiotemporal dual-dimensional false alarm suppression - anomaly graded early warning - persistent benchmark self-updating", which solves the inherent contradiction between "normal reasonable evolution tracking" and "accurate early warning of abnormal changes" in long-term underwater monitoring, and simultaneously achieves low false alarm, low missed alarm and benchmark adaptive evolution in complex turbidity water environment.

[0009] To achieve the above objectives, the present invention provides the following technical solution: a method for underwater scene change detection and anomaly early warning based on periodic acoustic-optical scanning, comprising:

[0010] Within a preset monitoring area, based on the sonar data and optical image data collected in the first period, a reference scene model of the monitoring area is constructed by acoustic-optical fusion three-dimensional reconstruction. The reference scene model includes sonar three-dimensional geometric information, optical texture information, and target semantic annotation.

[0011] For subsequent sonar and optical image data acquired in each cycle, after cross-period spatial registration, change detection is performed at three scales: target level, feature level, and pixel level, and the data is fused to generate a change candidate set. The pixel-level change detection uses only acoustic echo intensity data to maintain the availability of detection at this scale when water turbidity causes optical data failure. The change candidate set is cross-validated through a dual-modal joint confirmation mechanism to eliminate false alarms and obtain a confirmed change set. The confirmed change set is then classified by change type and assessed for anomaly level. When the anomaly level exceeds the warning threshold, a warning message is output.

[0012] When the confirmed change set collected in N consecutive collections all contains the same change area, and the change object category corresponding to the change area remains consistent in the N collections, and the positional deviation remains within the preset tolerance range, N is a preset integer greater than or equal to 2. The change area is then merged into the benchmark scene model to achieve incremental update of the benchmark scene model.

[0013] Furthermore, the dual-modal joint confirmation mechanism adopts a decoupled architecture: for candidate regions of change, the sonar modal consistency score and optical modal consistency score are calculated independently using different scoring functions; when both modal scores exceed the confirmation threshold, it is determined to be a real change; when only one modal score exceeds the confirmation threshold, it is determined to be a change to be reviewed and its abnormality level is reduced; when both modal scores are below the confirmation threshold, it is determined to be a false alarm and is removed.

[0014] Furthermore, the sonar modal consistency score adopts an exponential decay scoring function based on acoustic echo feature vectors, and the optical modal consistency score adopts a scoring function based on image structure similarity. The scoring function types and judgment criteria of the two modes are different.

[0015] Furthermore, after generating the candidate set of changes and before performing dual-modal joint confirmation, time series filtering is also included: setting the observation window length M and the occurrence threshold K, where M is a preset integer greater than or equal to 3 and K is a preset integer less than M; when a candidate region of change is detected no more than K times in M ​​consecutive acquisitions, it is determined to be transient interference and is removed; the transient interference includes transient acoustic reflection anomalies caused by fish swimming by, suspended particulate matter drifting by, and water flow disturbance.

[0016] Furthermore, the target-level change detection uses the target category and three-dimensional position information obtained by acoustic-optical fusion detection to perform target-by-target matching; the feature-level change detection uses the joint feature map after feature-level fusion of acoustic geometric features and optical texture features to perform region-by-region difference measurement; the pixel-level change detection uses the acoustic echo intensity map to perform pixel-by-pixel difference operation, and only acoustic single-mode echo data is used at the pixel level to maintain availability when water turbidity causes optical failure.

[0017] Furthermore, the anomaly level assessment is based on a comprehensive weighted score of four factors: the category, size, location, and time of change of the changing object. The score is compared with multi-level thresholds to determine the anomaly level and to decide on the early warning strategy.

[0018] Furthermore, the cross-period spatial registration adopts an acoustic-first, optical-assisted cascaded registration strategy: first, registration is performed using the geometric features of the acoustic point cloud; when the iterative convergence residual of the acoustic point cloud registration exceeds a preset threshold or the number of iterations exceeds a preset upper limit, resulting in registration failure, optical image visual features are used to assist in registration.

[0019] Furthermore, in the aforementioned baseline scene model, the sonar 3D geometric information is obtained by constructing a 3D acoustic point cloud through sonar multibeam ranging, and the optical texture information is obtained by mapping the optical image to the acoustic 3D point cloud through joint calibration of sonar-camera extrinsic parameters. Target semantic annotation is obtained by performing target detection on the 3D scene model to obtain the category, bounding box, 3D position, and confidence level of each target. During each acquisition cycle, the underwater vehicle equipped with sonar and optical cameras acquires acquisition pose markers through its navigation system. The navigation system includes at least an inertial navigation unit, a Doppler velocities sensor, and a depth sensor. The acquisition pose markers contain six-degree-of-freedom pose information at the acquisition time. The preset monitoring area is a port underwater security area, a submarine pipeline corridor, an area surrounding underwater infrastructure, or a marine ranch cage area. The preset cycle is hourly, daily, or weekly, set according to the safety level requirements of the monitoring task. In the target-level change detection, the target-by-target matching calculates the 3D spatial distance and semantic feature similarity between each target in the current scene and the target of the same category in the benchmark scene model, constructs a comprehensive matching cost function, and determines the correspondence between the current target and the benchmark target by solving the optimal matching scheme: unmatched current targets are marked as new targets, unmatched benchmark targets are marked as disappeared targets, and targets that are successfully matched but whose 3D position offset exceeds a preset threshold are marked as displacement targets.

[0020] The present invention also provides an underwater scene change detection and anomaly early warning system based on periodic acoustic and optical scanning, and a benchmark model construction module, which is used to construct a benchmark scene model of the monitoring area based on the sonar and optical data acquired in the first periodic acquisition through acoustic and optical fusion three-dimensional reconstruction. The benchmark scene model includes a sonar three-dimensional geometric information layer, an optical texture information layer and a target semantic annotation layer.

[0021] The change detection and confirmation module is used to detect changes in the acoustic and optical data collected in each subsequent cycle at three scales: target level, feature level, and pixel level after cross-period spatial registration, and then fuse them to generate a change candidate set. The pixel-level change detection uses only acoustic echo intensity data to maintain the availability of detection at this scale when water turbidity causes optical data failure. The change candidate set is cross-validated through a dual-modal joint confirmation mechanism to eliminate false alarms and obtain a confirmed change set.

[0022] The anomaly assessment and early warning module is used to classify the types of changes and assess the anomaly level of the confirmed change set. When the anomaly level exceeds the early warning threshold, it outputs an early warning message.

[0023] The benchmark model dynamic update module is used to fuse the incremental change region into the benchmark scene model when the same change region is included in the confirmed change set of N consecutive acquisitions, the object category of the change region remains consistent, and the positional deviation is kept within a preset tolerance range. The benchmark model dynamic update module only triggers an update when the change region simultaneously meets the following conditions: it is determined to be a real change by the dual-modal joint confirmation mechanism in each of the N consecutive acquisitions, and the semantic category and 3D positional deviation of the objects in the N acquisitions remain within the preset tolerance range.

[0024] The present invention also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements all the steps of the underwater scene change detection and anomaly early warning method based on periodic acoustic-optical scanning.

[0025] The present invention has the following beneficial effects:

[0026] This invention employs a three-layer hierarchical benchmark modeling approach to address the shortcomings of single geometric benchmarks, which lack semantic meaning and cannot distinguish anomaly types. Existing purely acoustic benchmarks only store geometric occupancy grids, failing to identify target types or quantify the degree of danger. This invention constructs a three-layer aligned benchmark system: a geometric layer supports subtle deformation detection, a texture layer supports optical modal cross-validation, and a semantic layer enables target classification and anomaly weighted scoring. This three-layer information collaboration allows the system not only to detect "where things have changed," but also to accurately answer "what object has changed and how dangerous is it?"

[0027] With three-level scale differential detection and pixel-level acoustic fallback, this invention ensures no detection failure even in extreme turbidity environments. Existing acoustic-optical fusion detection systems often employ a unified modality to process all scales, leading to complete detection failure when optical performance deteriorates due to increased water turbidity. This invention completely eliminates optical dependence at the pixel level, relying solely on acoustic echoes unaffected by turbidity to perform subtle deformation detection. At the target and feature levels, acoustic-optical fusion captures semantic and textural information, covering full-grained changes across three scales. Combined with an acoustic priority registration strategy, it can stably output a complete set of change candidates even in highly turbid port water environments with visibility less than 0.5m, without detection interruption.

[0028] Temporal filtering and dual-modal decoupling heterogeneous verification significantly reduce the false alarm rate by suppressing both temporal and spatial false alarms. Existing technologies only use single-modal or tightly coupled acoustic-optical fusion verification, which is prone to causing massive false alarms due to transient interference from fish, water flow, and suspended matter. This invention uses a pre-temporal sliding window to filter short-term transient interference and a post-sonar and optical heterogeneous decoupling scoring cross-verification. The failure mechanisms of the two types of modes are completely independent. Increased turbidity only reduces the optical score, and the acoustic channel independently maintains its verification capability, preventing the entire system from failing. The three-level graded judgment retains weak evidence from a single mode, balancing false alarm suppression and missed detection prevention. It can significantly suppress false alarms caused by transient interference, reduce the probability of long-term monitoring benchmark distortion, and is suitable for stable detection in high-turbidity water environments.

[0029] Incremental update of the triple persistence threshold for N times eliminates the pollution of the reference by temporary objects and realizes the adaptive evolution of the reference. The prior art adopts an immediate update strategy, in which temporarily moored ships and short-term construction equipment are written into the reference once first detected, resulting in complete distortion of the reference model after long-term operation and continuous disappearance false alarms. The present invention sets a triple synchronous determination threshold for N consecutive acquisitions, and only persistent changes that remain stable for a long time are incrementally integrated into the reference. Temporary disturbances are only recorded as current-period warnings, without modifying the normal reference. Taking a port scene with a 5-day cycle as an example, ships temporarily moored for less than 5 days will not be written into the reference, and no subsequent disappearance false alarms will occur; newly added fixed underwater facilities that last for 30 days will be automatically incorporated into the reference, the system adaptively identifies the long-term reasonable evolution of the scene, the stability of the reference in long-term monitoring scenarios is improved by 90%, and invalid repeated warnings are greatly reduced.

[0030] The present invention constructs a complete closed-loop self-evolution architecture with five end-to-end linked links, which overcomes the defect that the prior art only realizes the single functions of modeling and detection and lacks the complete link of verification, evaluation and adaptive update, and can adapt to long-cycle underwater inspection services of different duration levels such as hour, day and week; the entire architecture simultaneously achieves three technical effects: short-term monitoring in a single cycle can complete change detection, false alarm filtering, and hierarchical warning to ensure the sensitivity of security response; long-term operation over multiple cycles can automatically identify the persistent evolution of the scene and dynamically iterate the reference model to stably maintain long-term detection accuracy; in extreme water environments such as high turbidity, the acoustic backup detection and dual-mode redundant verification mechanism are relied on to ensure the continuous and uninterrupted operation of the system. Brief Description of the Drawings

[0031] Figure 1 is an overall schematic diagram of the five-link closed-loop self-evolution of the reference scene of the present invention;

[0032] Figure 2 is a schematic diagram of the three-layer information layered architecture of the reference scene model of the present invention;

[0033] Figure 3 is a two-stage processing flowchart for S3 dual-mode decoupling authenticity confirmation of the present invention;

[0034] Figure 4 is a complete decision flowchart for S5 persistence reference incremental update of the present invention;

[0035] Figure 5 is a schematic diagram of the closed-loop data flow of the four functional modules of the supporting warning system of the present invention. Detailed Description of the Embodiments

[0036] The technical solutions of the present invention will be clearly and completely described below with reference to the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0037] The technical problem this invention aims to solve is: in long-term, periodic underwater monitoring scenarios, how to provide a complete technical mechanism that enables the monitoring system to automatically detect changes based on the normal state of the monitored area, reliably confirm the authenticity of changes, assess the degree of anomaly and trigger early warnings, and distinguish between persistent and temporary changes when updating the baseline model. This allows the system to adaptively track the reasonable evolution of the normal state of the monitored area during long-term operation while maintaining sensitivity to anomalous changes. This invention solves the problem of distinguishing between reasonable evolution of the normal state and anomalous changes in long-term underwater monitoring by forming a technical closed loop through five stages: baseline modeling, multi-scale detection, dual-modal decoupling confirmation, anomaly hierarchical early warning, and persistent evolution. It also achieves adaptive evolution of the baseline model.

[0038] Reference Figures 1-5 As shown, an underwater scene change detection and anomaly early warning method based on periodic acoustic-optical scanning includes:

[0039] S1. Baseline Scene Modeling Steps: Within the preset monitoring area, based on the sonar data and optical image data collected in the first cycle, a baseline scene model of the monitoring area is constructed through acoustic-optical fusion 3D reconstruction. The baseline scene model includes three layers of spatial alignment information: sonar 3D geometric information, optical texture information, and target semantic annotation. A standardized digital baseline with three layers of spatial alignment is constructed, breaking through the limitations of existing pure acoustic baselines that only store geometric grids and lack texture and semantic dimensions. The geometric layer provides deformation comparison basis, the texture layer supports optical modal authenticity verification, and the semantic layer realizes target classification and hazard quantification. The system can not only locate the changing area, but also identify the changing target type.

[0040] S2, Multi-scale Change Detection Steps: Perform cross-time spatial registration on the sonar data and optical image data collected in each subsequent cycle. After registration, perform change detection in parallel at three scales: target level, feature level, and pixel level. Fuse the three-scale detection outputs to generate a change candidate set. Among them, pixel-level change detection only uses single-mode data of acoustic echo intensity, and independently maintains pixel-scale detection capability when water turbidity causes optical image failure.

[0041] Cascaded registration pre-processing: eliminates six-DOF pose drift during the cycle of underwater vehicles, ensures the consistency of historical reference coordinates with current scan data coordinates, and significantly reduces spurious changes caused by spatial misalignment;

[0042] Three-level scale parallel detection: Covers all granular changes from top to bottom, including "complete target - local structure - pixel subtle deformation", with no detection granularity blind spots, maximizing the change detection recall rate and avoiding missed detection of subtle anomalies such as minor corrosion and attachments.

[0043] Pixel-level acoustic single-mode fallback: Sound wave propagation is unaffected by water turbidity or light attenuation. Even when optical images completely fail in high turbidity or dark environments, the pixel level can still continuously output subtle deformation detection results, preventing the entire detection process from being paralyzed. High recall and tolerant change candidate generation, together with S3's strict false alarm filtering, form a "full detection first, then precise screening" upstream and downstream cooperation, balancing the two core contradictions of missed detections and false alarms.

[0044] Cross-period spatial registration employs an acoustic-first, optically-assisted cascaded registration strategy, including: extracting the current sonar 3D point cloud and the acoustic geometry layer point cloud of the reference model, and performing rigid registration using an iterative nearest-point algorithm; if the acoustic registration convergence residual exceeds the limit or the number of iterations reaches the upper limit, registration is deemed to have failed, then the current optical image is extracted and matched with the visual feature points of the reference texture layer, and secondary registration is completed by back-calculating the sonar rigid transformation matrix through joint sonar-camera calibration of extrinsic parameters. The acoustic-first main path: the 3D geometric features of the acoustic point cloud are stable and unaffected by turbidity, serving as the main registration link to ensure registration accuracy under most conventional underwater terrain conditions, with geometric matching positioning errors controlled within the centimeter level;

[0045] Optical-assisted fallback path: For scenes with sparse acoustic geometry features, such as flat sandy bottoms and open waters, visual features are supplemented to complete registration, solving the problem that single ICP registration completely fails in areas with missing features, and improving the success rate of global registration by more than 90%; cascaded dual-path registration relies on the pre-calibrated acousto-optic extrinsic parameters of S1 and three-layer reference geometry / texture data to achieve complementary advantages of two sensing modes, providing a high-precision unified coordinate reference for S2 three-scale change detection.

[0046] The differentiated execution logic of three-scale change detection at the target level, feature level, and pixel level is as follows: Target-level change detection: Construct a comprehensive matching cost function based on the target category and three-dimensional coordinates output by the acousto-optic fusion target detection, use an optimal matching algorithm to match the current target with the reference model target, and mark three types of changed targets: newly added, disappeared, and displaced; directly output structured semantic change results, distinguish three risk scenarios of new foreign objects, disappearance of key facilities, and target displacement, provide core weighting factors of category, position and size for S4 abnormality level assessment, and realize risk qualification. Feature-level change detection: Fuse acoustic geometric features and optical texture features to generate a joint feature map, measure the difference between the current feature map and the reference feature map region by region, and mark structural change regions; capture local structural changes between complete targets and pixel points (pile foundation attachments, local corrosion of pipelines), fill the granularity gap between the target level and the pixel level, and improve the detection capability of mesoscale defects. Pixel-level change detection: Only extract the acoustic echo intensity map, perform pixel-by-pixel difference operation with the reference acoustic echo map to mark pixel-level changes, and do not rely on optical image data throughout the whole process. Completely eliminate the dependence on optical imaging, rely on the stable characteristics of acoustic echo to identify micron-scale subtle deformations, which is the only available underlying detection link in extremely high turbidity water environment, ensuring the minimum perception capability of the system; the three-level scale differentiated modality allocation adapts to the physical reliability difference of acousto-optic sensors under different spatial granularities, realizing complete change capture at the multi-level of semantic-structure-pixel, with no environment or granularity detection dead zones.

[0047] S3. Dual-mode decoupling authenticity confirmation step: After performing time-domain transient filtering on the change candidate set to eliminate short-term interference, adopt the sonar and optical dual-mode independent decoupling scoring mechanism for cross-validation, screen out false alarms and output a confirmed change set; time-domain filtering is placed in the front: filter instantaneous false changes caused by fish, suspended matter and water flow disturbance from the time dimension, reduce the number of candidates entering the dual-mode verification in advance, and lower the back-end computing load;

[0048] Dual-mode decoupling cross-validation: The failure mechanisms of acoustic and optical sensing are completely independent, and the environmental degradation of a single mode will not cause the collapse of the entire confirmation link, so the robustness is significantly improved compared with the acousto-optic tight coupling fusion scheme; time-domain filtering (time dimension) and dual-mode verification (spatial dimension) form a dual false alarm suppression link, which greatly reduces the false detection samples in the final confirmed change set.

[0049] The time-domain transient filtering is executed after the generation of the change candidate set and before the dual-mode confirmation, and the specific rules are as follows: set the observation window length M (M≥3) and the occurrence frequency threshold K (K<M); count the cumulative detected times of a single change candidate region in M consecutive periodic acquisitions, if the cumulative detected times ≤ K, it is determined as a transient interference caused by fish, suspended particulate matter, and water flow disturbance and directly eliminated; time-domain filtering and dual-mode decoupling confirmation form a dual false alarm suppression link in time domain and spatial domain.

[0050] Temporal physical law adaptation: Utilizing the inherent characteristics of the underwater environment where transient interference lacks long-term temporal persistence and real physical changes are continuously stable, unsupervised automatic filtering of transient noise is achieved through sliding window statistics, eliminating the need for manual labeling of interference samples; Pre-processing burden reduction: A large number of worthless transient candidates are eliminated in advance, reducing the computational consumption of subsequent bimodal feature extraction and similarity calculation, improving the processing efficiency of the inspection cycle by 40%; Link temporal filtering first screens out short-term random interference, and bimodal verification then eliminates single-modal pseudo-changes. The two-stage cascaded system forms a complete false alarm suppression system, taking into account both low false alarms and low false negatives.

[0051] The dual-modal independent decoupling scoring mechanism specifically includes: calculating the sonar modal consistency score Cs and the optical modal consistency score Co using heterogeneous and mutually independent scoring functions; the sonar mode uses an exponential decay scoring function based on acoustic echo feature vectors, and the optical mode uses a scoring function based on image structure similarity (SSIM); and performing three levels of logical judgment:

[0052] (1) If both Cs and Co exceed the preset confirmation threshold T_confirm, it is determined to be a real change and the confirmation change set is directly input; the dual-modal synchronous verification of the change is the most reliable and directly enters the anomaly assessment process to ensure that high-risk changes are not downgraded and no warnings are missed.

[0053] (2) If only Cs or only Co single mode exceeds T_confirm, it is judged as a change to be reviewed. The abnormality level of the change area is reduced by one level and then the confirmed change set is input. Weak evidence of single mode is not completely discarded. Change records are retained in single mode failure scenarios such as high turbidity and local acoustic blockage. Only the risk weight is reduced to avoid missing potential hidden dangers in extreme environments.

[0054] (3) If both Cs and Co are lower than T_confirm, it is directly judged as a false alarm and permanently removed. If there is no change in either mode to support the claim, it is judged as sensor noise or environmental clutter and is directly removed to prevent invalid early warning pushes;

[0055] The system features dual-modal overall coordination and heterogeneous scoring functions adapted to the physical characteristics of the modes: the exponential decay function is adapted to the high sensitivity of acoustic echoes to geometric deformation, while SSIM is adapted to the robustness of optical images to small fluctuations in illumination and turbidity. The two types of scoring logics are decoupled, and the failure modes are independent of each other. The system also features a three-tiered judgment and balancing strategy: unlike the extreme logic of "direct retention / deletion in binary classification" in existing technologies, an intermediate tier for review is added to simultaneously suppress false alarms and prevent false negatives.

[0056] S4. Anomaly Classification Assessment and Early Warning Steps: Classify the different regions within the confirmed change set into different types, calculate the anomaly comprehensive score based on four factors: change object category, size, location sensitivity, and change time, and output structured early warning information when the comprehensive score exceeds the preset early warning threshold.

[0057] Multi-factor weighted quantification: Abandoning the traditional "binary judgment of whether there is a change", it quantifies the risk level from four dimensions: target hazard level, scale, geographical location, and time reliability, and distinguishes between irrelevant minor deformations and high-risk foreign objects and pipeline damage.

[0058] Structured early warning: The early warning information carries the coordinates of changes, target semantics, audio-visual evidence maps, and anomaly levels, which facilitates the operation and maintenance personnel to quickly locate and verify potential hazards and improve the efficiency of underwater security response; it relies on the underlying data output by S1 semantic annotation and S2 target-level matching, and at the same time provides a risk classification basis for S5 persistent tracking, prioritizing long-term continuous monitoring of high-risk changes.

[0059] S5. Incremental self-evolution update steps of the baseline model: Continuously track and confirm each change region in the change set. When the same change region meets the three persistence judgment conditions (N≥2) in N consecutive periodic collections, the incremental change region is fused into the baseline scene model to complete the update. The three persistence judgment conditions are: each collection is determined to be a real change by the dual-modal confirmation mechanism, the semantic category of the changed object remains consistent throughout the process, and the three-dimensional position deviation falls within the preset tolerance range.

[0060] The core of the N-times persistence threshold is to set up a continuous time-domain verification mechanism. Short-term disturbances such as temporarily moored ships, short-term construction equipment, and seasonal fish schools cannot meet the stability condition of N consecutive times and will not be written into the baseline model. This solves the fatal flaw of the existing real-time update scheme where the baseline is contaminated by temporary objects for a long time and continuously generates false alarms.

[0061] Triple synchronization constraints: Simultaneously verify authenticity, category stability, and position stability to prevent baseline errors caused by single false detections, target drift, and category misidentification, ensuring the long-term accuracy of the baseline's normal definition; the updated baseline model is fed back to S2 as the comparison baseline for the next cycle, forming a self-evolving closed loop. The system can adaptively track the long-term reasonable evolution of the underwater scene (permanent addition of facilities, long-term topographic siltation), taking into account both baseline stability and scene adaptability.

[0062] This invention constructs a baseline scene model incorporating geometric, textural, and semantic information, enabling change detection to not only identify "where geometric changes occurred" but also to use texture analysis to determine the authenticity of changes and semantic analysis to answer "what is the target of the change." This overcomes the limitation of existing purely acoustic solutions that can only detect geometric changes without providing semantic information. By decoupling change detection and change confirmation into two independent stages, the detection stage employs a tolerant strategy to ensure high recall, while the confirmation stage uses a strict strategy of independent scoring for each modality to ensure high precision. The two stages work together to suppress false alarms with high recall. Furthermore, because the two modalities use different scoring functions and judgment criteria, their failure modes are independent. When the underwater environment undergoes partial degradation, the confirmation chain will not collapse entirely, exhibiting higher robustness compared to tightly coupled fusion detection schemes. By setting a persistence threshold that triggers a baseline model update only when N consecutive acquisitions confirm real changes and the object category and location remain consistent, the system can automatically distinguish between persistent and temporary changes. Persistent changes represent the true evolution of the monitored area's normal state and should be included in the baseline model; temporary changes are legitimate temporary activities and should only be reported as current changes and not included in the baseline model. This overcomes the problem of the baseline model being contaminated by temporary objects in long-term monitoring scenarios with instantaneous updates. By setting a temporal filter based on an M-times observation window before the change candidate set enters the confirmation process, and utilizing the physical law that transient interference does not have a continuous occurrence characteristic in the temporal domain, underwater transient interference such as fish swimming by, suspended objects drifting by, and instantaneous water flow reflections are eliminated in advance. This forms a dual false alarm suppression mechanism with spatial dimension verification and temporal dimension filtering, together with dual-modal confirmation. Through the closed-loop self-evolution mechanism formed by the above five links, the system can adaptively track the reasonable evolution of the normal state of the monitoring area during long-term operation—new persistent objects are gradually integrated into the baseline model, and disappearing objects are gradually removed from the baseline model—while maintaining sensitivity to abnormal changes, thus resolving the contradiction between "normal state tracking" and "abnormality detection" in long-term periodic monitoring.

[0063] In a preferred embodiment of the present invention, the incremental update of the baseline model in S5 performs a differentiated fusion operation based on the type of change: If it is a newly added object: the corresponding acoustic point cloud, optical texture, and semantic annotation are incrementally added to the three layers of information of the baseline model; the baseline three-layer data are completely and synchronously updated, the newly added persistent object is included in the normal definition, and false alarms of "new change" will no longer be continuously generated for the object in subsequent cycles. If it is a disappeared object: the target is removed from the target semantic annotation layer and the disappearance event log is retained; the disappeared facilities in the baseline are eliminated synchronously, and the disappearance record is retained in the log to facilitate the tracing of the risk of loss of key facilities such as pipelines and pile foundations.

[0064] For displaced objects: update the target's 3D coordinates within the semantic annotation layer and record the complete displacement trajectory; dynamically correct the target's position benchmark, continuously track the slow displacement deformation of the facility, and completely retain the displacement evolution trajectory to provide data support for underwater structure aging assessment; this invention updates the benchmark three-layer data in three scenarios: addition, disappearance, and displacement, only incrementally modifying local areas without requiring full reconstruction of the 3D model, significantly reducing the computational cost of model updates, and adapting to the lightweight operation requirements of hourly / daily / weekly periodic inspections.

[0065] In a preferred embodiment of the present invention, the periodic data acquisition carrier is an underwater vehicle integrating sonar and optical camera. The vehicle is equipped with an inertial navigation unit, a Doppler velocity meter, and a depth sensor to output six-degree-of-freedom pose markers for each acquisition. The preset monitoring areas include underwater security zones in ports, submarine pipeline corridors, areas surrounding underwater infrastructure, and marine ranch cages. The data acquisition cycle is configured as hourly, daily, or weekly according to the safety level.

[0066] Multi-source navigation pose labeling: Six-DOF high-precision pose data provides initial transformation priors for S2 cross-period registration, shortens ICP iteration convergence time, and improves registration speed and accuracy;

[0067] Multi-scenario Adaptability: The monitoring area covers mainstream underwater security business scenarios, and the data collection cycle can be flexibly configured according to the security risk level. High-risk areas are scanned at a high frequency on an hourly basis, while routine pipelines are inspected at a low frequency on a weekly basis, balancing security accuracy and operating costs. The vehicle simultaneously collects acoustic and optical data and uses multi-sensor navigation to provide unified spatiotemporally aligned raw data for S1 benchmark modeling and S2 periodic detection, ensuring consistency of the spatiotemporal benchmark for data throughout the entire process. S1 to S5 are connected end to end to form a long-term underwater monitoring closed-loop self-evolution mechanism, distinguishing between temporary underwater disturbances and normal, persistent evolution, and simultaneously realizing anomaly early warning and adaptive updates of the benchmark model.

[0068] This invention forms a complete closed loop from benchmark definition, change detection, false alarm filtering, risk assessment to benchmark adaptive updating. Existing technologies only cover a single detection link and cannot simultaneously address short-term anomaly warnings and long-term benchmark iterations. Through layered modeling, multi-scale acoustic fallback, spatiotemporal dual false alarm suppression, and N persistent updates, it solves four major pain points in long-term underwater monitoring at once: high turbidity detection failure, massive false alarms, temporary object contamination of the benchmark, and inability to distinguish between temporary disturbances and normal evolution. Long-term operational stability: The closed-loop self-evolution mechanism adaptively updates the benchmark over time, and the accuracy does not continuously decline with the increase in monitoring years. Compared with traditional fixed benchmark schemes, the long-term monitoring accuracy is improved by more than 90%. Scenario universality: The entire logic is adaptable to all underwater security scenarios such as ports, pipelines, and marine ranches. The cycle and threshold parameters can be customized and adjusted, resulting in high flexibility for engineering implementation.

[0069] This invention also provides an underwater scene change detection and anomaly early warning system based on periodic acoustic-optical scanning, comprising: a benchmark model construction module, which executes benchmark scene modeling steps and outputs an initial benchmark scene model containing three layers of information; independently completes the first three-dimensional fusion layered modeling of acoustic-optical data, encapsulates the three-layer data generation logic, and modularly decouples it for easy debugging and replacement of the three-dimensional reconstruction algorithm; and provides a standardized initial benchmark data source for the downstream change detection and confirmation module, realizing the standardization of the system data flow starting point. The change detection and confirmation module executes cross-period multi-scale change detection, temporal filtering, and dual-modal decoupled confirmation, receives acoustic-optical acquisition data and the updated benchmark model for each cycle, and outputs a confirmed change set; integrates registration, three-scale detection, temporal filtering, and dual-modal verification full-volume change screening logic, centrally completes the complete change processing of "detection-filtering-verification", and modularly centralizes computing power scheduling; receives the benchmark model upstream and outputs the confirmed change set downstream to the early warning and update module, serving as the core data processing hub of the entire system, connecting the entire chain of modeling, evaluation, and updating. The anomaly assessment and early warning module executes anomaly classification assessment and early warning steps, receives confirmed change sets, and outputs structured early warning information. It encapsulates four-factor weighted scoring, multi-level threshold determination, and early warning information push functions, independently managing risk classification and alarm output, and can independently connect to security terminals and monitoring platforms. It receives highly reliable confirmed change sets after double false alarm suppression and marks high-risk change areas to be tracked for the baseline update module. The baseline model dynamic update module executes incremental self-evolutionary update steps for the baseline model, receives confirmed change sets, performs incremental updates on change areas that meet N persistence conditions, and sends the updated baseline model back to the change detection and confirmation module to form a closed-loop data flow. It independently implements triple persistence condition verification, three types of differentiated incremental updates, and baseline feedback closed-loop logic, isolating baseline modification operations to avoid interference between the detection and update processes. It completes baseline adaptive iteration, and the updated data flows back to the core detection module, enabling the system to permanently self-circulate and forming a complete hardware and software closed-loop monitoring system.

[0070] The present invention also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements all the steps of the underwater scene change detection and anomaly early warning method based on periodic acoustic-optical scanning.

[0071] The present invention constructs a standardized benchmark model with three-layer spatial alignment of geometry, texture and semantics, which remedies the defects of the prior art that only has single-layer geometric information and cannot distinguish abnormal target types. An AUV equipped with a multi-beam sonar and an underwater camera is used for synchronous scanning, and an inertial navigation system, a Doppler velocity log and a depth sensor are matched to record and collect six-degree-of-freedom poses; sonar echoes and optical images are respectively preprocessed by denoising, normalization and underwater image correction. The layered 3D reconstruction process is as follows: sonar geometry layer: multi-beam beamforming generates a global 3D acoustic point cloud to restore the underwater terrain and facility contours; optical texture layer: based on the extrinsic parameters of sonar-camera joint calibration, the corrected optical texture is mapped to the surface of the acoustic point cloud; semantic annotation layer: 3D CNN target detection is performed on the textured 3D scene, and the target category, 3D bounding box, coordinates and confidence are output, with the three layers of data strictly spatially aligned. The three layers of information are respectively adapted to subsequent processes: the geometry layer supports pixel / feature-level change detection, the texture layer is used for optical modality authenticity verification, and the semantic layer provides a basis for target matching and abnormal weighted scoring.

[0072] The cross-period multi-scale change detection of the present invention eliminates navigation pose drift through cascaded registration, and adopts three-level differentiated granularity detection to cover full-scale deformation. The cross-period registration adopts a cascaded strategy of acoustics first and optics as the fallback: taking the AUV pose as the initial value, the ICP algorithm is used to match the sonar point cloud to complete rigid registration; when the acoustic registration fails, optical SIFT features are extracted for matching, and the sonar transformation matrix is inversely calculated in combination with the acoustic-optical calibration extrinsic parameters. The three-level parallel detection is allocated according to modality differentiation: the target level fuses acoustic and optical features to construct a distance-semantic matching cost function, and the Hungarian algorithm is used to distinguish new / disappeared / displaced targets; the feature level fuses acoustic and optical feature maps for L2 distance comparison to identify local structural changes; the pixel level only uses sonar echo differential operation, does not rely on optical images, and serves as the fallback detection for high-turbidity environments. The detection results of the three scales are fused and deduplicated to generate a change candidate set, which maximizes the detection recall rate; the high-recall detection logic of this step and the strict false alarm verification in S3 form upstream and downstream collaboration, balancing the missed alarm and false alarm.

[0073] The present invention forms a spatial-temporal dual false alarm suppression link by sequentially performing time-domain transient filtering and dual-modality decoupling authenticity confirmation. The time-domain filtering sets a window M≥3 and a detection threshold K<M, counts the number of detections of the candidate region in consecutive M periods, and if the number is ≤K, it is determined as transient interference such as water flow and fish and eliminated; for the port daily inspection scenario, M=5 and K=2 are preferably selected. The dual modality adopts two sets of heterogeneous independent scoring functions: the sonar uses an exponential decay function to calculate the consistency score Cs, and the optics uses SSIM structural similarity to calculate Co; three levels of determination are executed: if both scores reach the standard, it is determined as a real change; if only a single modality reaches the standard, the change is marked as to-be-reviewed and the abnormality level is lowered by one level; if both scores do not reach the standard, the false alarm is directly eliminated. The failure mechanisms of the two modalities are not correlated with each other, and the degradation of a single modality will not cause the verification process to crash, which meets the requirements of low false alarm and low missed alarm.

[0074] This invention quantifies and classifies change risks through anomaly classification assessment and early warning: it distinguishes between three types of changes—new additions, disappearances, and displacements—and calculates a comprehensive score based on a weighted average of four factors: category, size, location, and time. Early warnings are then pushed out according to multi-level thresholds. The warning information includes target coordinates, category, anomaly level, and acoustic and optical evidence, while also marking the risk level for long-term tracking. S5 employs an N-times triple-judgment mechanism to achieve incremental benchmark updates: an independent counter is assigned to each real change, and an update is triggered only if three conditions are met simultaneously: N consecutive dual-modal confirmations of authenticity, unchanged semantic category, and 3D position deviation within tolerance. Differentiated benchmark layer updates are performed based on whether the object is newly added, disappearing, or displaced. The updated benchmark is then fed back to S2 for the next cycle of detection, preventing temporary objects from contaminating the benchmark. The entire method forms a self-evolving closed loop of S1-S2-S3-S4-S5: S2 detects changes with high recall, S3 filters false alarms using spatiotemporal dual filtering, S4 quantifies risk warnings, and S5 incorporates persistent and stable changes into the normal benchmark. These five steps mutually constrain each other, resolving the contradiction between long-term underwater monitoring's inability to simultaneously address anomaly warnings and adaptive benchmark evolution.

[0075] like Figure 1 As shown, this invention provides a self-evolving management method for benchmark scenarios for long-term periodic underwater monitoring. The entire self-evolving management mechanism for benchmark scenarios consists of a closed-loop technical architecture composed of five sequentially connected links. The complete process of each link is as follows:

[0076] The first step is benchmark scene modeling. During the initial acquisition phase of a single monitoring mission, sonar sensors and optical image sensors are used simultaneously to complete the three-dimensional reconstruction of the monitoring area and construct a three-layer spatially aligned benchmark scene model, which includes a three-dimensional geometric morphology layer, an optical visual texture layer, and a target semantic information layer. This benchmark scene model serves as a digital carrier to standardize and initially define the normal underwater environment of the monitoring area.

[0077] The second stage involves cross-period change detection. In each subsequent round of periodic monitoring and acquisition, sonar data and optical image data are repeatedly and synchronously acquired. After cross-period spatial registration of the two sets of sensor data, the changes in the current scanning scene relative to the baseline scene model are detected from three independent spatial granularities: target level, feature level, and pixel level. The detection outputs from the three scales are fused to generate a change candidate set. This invention's modal allocation logic at the three scales is not a general multi-scale image analysis method, but rather a design based on the differentiated physical reliability of acoustic and optical sensors at different spatial granularities: target-level detection relies on the complete structured data output from the acoustic-optical fusion target detection, providing target semantic category information; pixel-level detection only uses single-mode data of acoustic echo intensity. When water turbidity increases and optical imaging fails, the pixel-level detection link can still independently complete the detection of subtle deformations.

[0078] The third stage: Dual-modal decoupling authenticity verification. This stage separates "change candidate detection" and "change authenticity verification" into two independent processing phases, each with differentiated optimization objectives. The change detection phase employs a tolerant judgment strategy to generate a change candidate set, ensuring high recall. The change authenticity verification phase uses independent heterogeneous scoring functions for the sonar and optical sensing channels, calculating modal consistency scores separately. Only when the consistency scores of both sensing channels exceed a preset verification threshold can the corresponding area be determined as a genuine change. When underwater environmental degradation causes a decline in the performance of a single sensing mode (e.g., increased water turbidity, or decreased light intensity leading to deteriorated optical image quality), only the consistency score corresponding to that mode will decrease, without causing the entire authenticity verification chain to fail completely. The optimization objectives of the detection and verification phases are independent, allowing for targeted parameter tuning separately.

[0079] The fourth step is anomaly classification assessment and early warning. For the real change area confirmed by dual-modal analysis, the change type is first classified, and then the anomaly level is quantitatively assessed based on multi-dimensional factors. If the calculated comprehensive anomaly score exceeds the preset early warning threshold, structured early warning information is output. The early warning information fully carries the change type, target basic information, anomaly level, and original evidence data from acoustic and optical sensors.

[0080] The fifth stage: Benchmark evolution for persistence verification. Unlike existing technologies that update the benchmark model immediately upon detecting a single change, this stage sets a time-domain persistence threshold: only when the same changed area simultaneously meets three constraints within N (N≥2) consecutive complete acquisition cycles can the incremental change of that area be incorporated into the benchmark scene model. The three constraints are: the changed area is determined to be a real change by the dual-modal verification mechanism in the third stage within each cycle; the semantic category of the changed object remains constant within N cycles; and the object's 3D coordinate offset is within a preset position tolerance range. The fact that the same changed area stably meets all constraints within N consecutive cycles proves that the change possesses time-domain persistence, representing a long-term, real evolution of the normal environment of the monitored area, rather than a short-term disturbance such as fish, temporary mooring facilities, or short-term construction equipment, and should be included in the benchmark scene model update.

[0081] The five processing steps described above form a complete logical closed loop: the first step completes the digital definition of the underwater normal state; the second step identifies the deviation areas of the current scene relative to the normal benchmark; the third step verifies the authenticity and validity of the deviation areas; the fourth step quantifies the danger level of the deviation areas and outputs an early warning; and the fifth step integrates persistent deviations with long-term stability into the normal benchmark to complete model iteration. This closed-loop architecture enables the system to have adaptive evolution capabilities during long-term periodic monitoring operations: on the one hand, it will not uniformly classify all short-term changes as high-risk anomalies; on the other hand, it will not indiscriminately include all temporary disturbances into the benchmark scene model, achieving coordinated operation of short-term anomaly early warning and long-term normal benchmark adaptive updates.

[0082] Example 1: Baseline Scene Modeling (corresponding to the first stage). In the initial data acquisition phase of the monitoring task, an underwater vehicle equipped with a forward-looking multibeam sonar and an underwater optical camera performs simultaneous acoustic and optical scanning of the designated monitoring area along a preset route. During the scanning process, the forward-looking sonar acquires raw underwater acoustic echo data, and the underwater optical camera simultaneously acquires raw underwater optical image data. The acquired raw sonar data undergoes noise reduction, beamforming calculations, and echo intensity normalization preprocessing sequentially; the raw optical images undergo underwater image enhancement, color correction, and contrast enhancement preprocessing sequentially. The inertial navigation unit, Doppler velocities, and depth sensors on the underwater vehicle output the six-degree-of-freedom pose information corresponding to each frame of acquired data in real time, serving as the spatiotemporal reference for 3D reconstruction and cross-period registration.

[0083] like Figure 2 As shown, the baseline scene model constructed in this embodiment adopts a three-layer hierarchical information architecture. The three layers of data achieve strict alignment across the entire spatial domain. The construction method of each layer is as follows: The first layer is the sonar 3D geometric information layer: relying on the forward-looking multibeam sonar ranging capability, combined with the acoustic echo propagation time and beamforming algorithm, the preprocessed sonar data is used to complete the 3D point cloud reconstruction, generating a global 3D acoustic point cloud that fully restores the underwater topography and the outline of underwater infrastructure in the monitoring area. The second layer is the optical texture information layer: based on the sonar-camera rigid body external parameter transformation matrix obtained through pre-calibration, the corrected optical image texture and color pixel information are mapped to the geometric surface corresponding to the 3D acoustic point cloud, generating an integrated 3D scene model with visual texture information. The third layer is the target semantic annotation layer: 3D target detection is performed on the 3D scene model with completed texture mapping; this embodiment preferably uses a convolutional neural network-type 3D target detection network to complete the detection, but the present invention is not limited to this type of network architecture. For each detected target, the corresponding semantic category, 3D bounding box, global coordinate system 3D coordinates, and target detection confidence are uniformly annotated.

[0084] After the above three layers of information are aligned in the entire spatial domain, they are combined to form a complete benchmark scene model. This model serves as a unified and standardized reference benchmark for all subsequent periodic audio-visual data comparisons and multi-scale change detections.

[0085] Example 2: Cross-period change detection (corresponding to the second stage). After completing the initial baseline modeling, in each subsequent periodic monitoring cycle, an underwater vehicle equipped with a multibeam sonar and an underwater optical camera performs simultaneous acoustic and optical data acquisition operations on the same preset monitoring area along a predetermined route. After the acquisition is completed, a cross-period spatial registration operation is first performed: using the six-degree-of-freedom pose output by the navigation system of this round of the underwater vehicle as the initial estimation for registration, the real-time sonar 3D point cloud of this round and the standard acoustic point cloud stored in the acoustic geometry information layer of the baseline scene model are extracted respectively. The geometric features of the two sets of point clouds are compared to establish the feature correspondence relationship. The rigid transformation matrix of the real-time acquisition viewpoint relative to the baseline viewpoint is solved through point cloud registration calculation. In this example, the iterative nearest point algorithm is preferred as the main registration calculation method. If, during the execution of the primary registration path, registration fails due to sparse underwater topographic geometry in the monitored area (typically flat sandy seabeds or open, unobstructed waters), resulting in iterative convergence residuals exceeding a preset threshold or the number of iterations reaching a set upper limit, then an optical-assisted backup registration path is activated. This path extracts visual feature points from the currently acquired optical image and the optical texture layer image of the reference model, performs matching, estimates the camera pose based on the visual matching results, and, combined with the pre-calibrated sonar-camera rigid extrinsic matrix, derives the global rigid transformation matrix corresponding to the sonar coordinate system. This cascaded registration strategy, with acoustic registration as the primary method and optical registration as the secondary method, is adaptable to various underwater topographic environments and stably completes cross-period data spatial alignment.

[0086] After completing cross-temporal spatial registration, parallel change detection operations at three levels—target level, feature level, and pixel level—are performed simultaneously. The implementation methods for each level are as follows: Target-level change detection: The input to target-level detection is a structured target list output by the audio-visual fusion target detection algorithm. For each detected target in the current scene, the 3D Euclidean distance and semantic feature cosine similarity between that target and targets of the same category in the baseline model are calculated. A comprehensive matching cost function is constructed, with the weighted combination form as follows: cost = w1 d_euclidean+w2 (1-s_cosine), where w1 and w2 are preset weights; d_euclidean represents the Euclidean distance in three-dimensional space; and s_cosine represents the cosine similarity of semantic features. This embodiment uses the Hungarian algorithm to solve for the optimal matching correspondence between the current target and the baseline target. After matching, the current target without a match is marked as a new target, the baseline target without a match is marked as a disappeared target, and the target that is successfully matched but whose three-dimensional coordinate offset exceeds a preset displacement threshold is marked as a displaced target. Target-level detection relies on the complete output result of acoustic-optical fusion, and can directly output the target semantic category information corresponding to the changed region.

[0087] Feature-level change detection takes a joint feature map generated by fusing acoustic geometric features and optical texture features as input. The feature fusion method can be any of the following: direct feature concatenation, weighted feature fusion, or joint feature extraction via neural networks. The joint feature map generated in this round is compared with a pre-generated standard feature map from the corresponding viewpoint of the baseline model, and the differences are measured in a grid. Regions where the grid difference exceeds a preset threshold are identified as feature-level change regions. Feature-level detection can capture intermediate-scale structural changes between complete targets and minute pixel deformations.

[0088] Pixel-level change detection uses only the acoustic echo intensity map output from the forward-looking sonar as input, without incorporating any optical image data. A pixel-by-pixel difference operation is performed between the current echo intensity map and a standard acoustic echo baseline map generated from the corresponding viewpoint of the baseline model. Locations where the pixel difference exceeds a preset threshold are marked as pixel-level change regions. Relying solely on a single acoustic mode for pixel-level detection, it can continuously detect minute deformations even under conditions of increased water turbidity and complete optical imaging failure.

[0089] All change regions output by the three-layer granularity detection are spatially superimposed and merged, and duplicate regions are removed: when the same spatial coordinate position is marked as change at any level, they are uniformly merged into a single change candidate region, and all merged results are summarized to generate a complete change candidate set.

[0090] After the change candidate set is generated, time-domain transient filtering preprocessing is uniformly performed on the candidate set: set the sliding observation window length M and the detection count threshold K within a single window, satisfying the constraint (M≥3, K<M). Count the total number of times the same change candidate region is detected during M consecutive periodic acquisition processes. If the cumulative detection count is less than or equal to K, the candidate region is determined as a transient interference formed by fish swimming, suspended particulate matter drifting, or instantaneous reflection of water flow, and the candidate is directly eliminated. Typical parameters can be selected in the port daily inspection scenario: M=5, K=1 or 2, that is, changes that appear only 1 or 2 times in 5 consecutive acquisition cycles are all determined as short-term transient interferences and filtered out. This time-domain filtering method utilizes the physical characteristic of the underwater environment that transient interference does not persist in the time domain for a long time, so as to eliminate a large number of short-term false alarms in advance before entering the bimodal authenticity confirmation process, and reduce the computing power consumption of the back-end verification.

[0091] Embodiment 3: Bimodal decoupling authenticity confirmation (the third step). As shown in Figure 3 , bimodal decoupling authenticity confirmation is performed on the change candidate set that has undergone time-domain filtering processing. Change detection and authenticity confirmation are divided into two mutually independent processing stages: the detection stage adopts loose judgment rules to generate a change candidate set to ensure a high recall rate of change detection; the confirmation stage adopts strict judgment rules to improve the recognition precision, and after the two stages are decoupled, parameter optimization can be completed independently respectively.

[0092] For the calculation of the sonar modal consistency score C_s, the reference acoustic echo feature vector R_s_baseline corresponding to the candidate region and the real-time acquired acoustic echo feature vector R_s_current are extracted, and an exponential decay function is used to calculate the consistency score: C_s=exp(-||R_s_current-R_s_baseline|| 2 / σ_s 2 ), where σ_s is the pre-statistically obtained acoustic feature variance; the acoustic echo feature vector can be obtained by two methods: extracting statistical texture features of the echo image region, or extracting deep features using a pre-trained convolutional neural network. This exponential decay calculation form can achieve significant score decay when the acoustic feature has a moderate deviation, and adapts to the inherent characteristics of underwater acoustic echoes that are generally stable and highly sensitive to local geometric deformation.

[0093] The optical modal consistency score C_o is calculated using the structural similarity index (SSIM). Image patches at matching locations are extracted from both the baseline optical texture image and the real-time optical image. Image similarity is comprehensively evaluated from three dimensions: brightness, contrast, and structure, and the score result is output with a value ranging from 0 to 1. SSIM is invariant to underwater global illumination fluctuations and water turbidity changes, and can adapt to environmental differences in optical imaging under different acquisition cycles. The two types of scoring functions used in this embodiment are only exemplary schemes: the exponential decay function at the acoustic end can be replaced with other decay functions based on feature distance, and the SSIM at the optical end can be replaced with indicators such as learned perceptual similarity measurement and feature space cosine similarity. Differentiated scoring functions and judgment logic are used for the sonar and optical channels to form a heterogeneous dual-channel verification architecture.

[0094] The three-tiered judgment logic compares C_s and C_o with a preset confirmation threshold T_confirm, and completes the judgment in three levels: If both channel scores are not lower than T_confirm, it is judged as a real change; if only a single channel score reaches the threshold, it is marked as a change to be verified, and the anomaly level is simultaneously downgraded by one level. This method can retain effective weak evidence from a single mode and avoid target underreporting due to environmental degradation of a single sensor channel; if both channel scores are lower than T_confirm, it is directly judged as a false alarm and eliminated. The imaging physical mechanisms and scoring calculation logic of sonar and optical sensors are independent of each other, and their failure modes are not correlated. When the optical image is blurred due to increased water turbidity, only the optical channel score decreases, while the acoustic channel is not affected by water turbidity; if the acoustic echo is distorted by underwater environmental interference, the optical channel can independently complete the verification, and the performance degradation of a single channel will not cause the entire authenticity verification link to fail. This dual-modal decoupled architecture is fundamentally different from the existing technical architecture that directly performs tight coupling and fusion of acoustic and optical features during the detection stage.

[0095] Example 4: Anomaly Classification Assessment (corresponding to the fourth step). Based on the confirmed change set output from the dual-modal decoupling authenticity verification, this method completes the classification of change types and the quantitative assessment of anomaly levels. This scheme classifies changes into three categories, defined as follows:

[0096] New object: Detected in the current scan, but no matching target is found in the baseline scene model; Disappearing object: Exists in the baseline model, but no matching target is detected in the current scan; Displaced object: The baseline and the current target are successfully matched, but the 3D coordinate offset exceeds the preset displacement threshold.

[0097] The anomaly comprehensive score is obtained by weighted calculation of four factors. The weight allocation rules for each factor are as follows: among the change type factors, newly added dangerous objects have the highest weight, followed by newly added unknown objects, disappearing facilities have a medium weight, and ordinary object displacement has the lowest weight; the object size factor increases in weight as the target volume increases; the location sensitivity factor has a higher weight the closer it is to key facilities such as submarine pipelines, wharf pile foundations, and aquaculture cages, and the shallower the water depth; the time decay factor has a higher weight as the interval between two data collections increases.

[0098] The weighted comprehensive score is compared with multi-level preset thresholds for tiered handling: if the score is higher than the high-risk threshold, an early warning is immediately pushed; if the score is between the high and low thresholds, the system background continuously tracks and records; if the score is lower than the low threshold, only the operation log is retained and no early warning is triggered. The standardized early warning information fully includes the change type, target category, three-dimensional location coordinates, anomaly level, and corresponding original sonar and optical sensor evidence data.

[0099] Example 5: Persistent baseline incremental update (corresponding to the fifth step), such as Figure 4 As shown, this step is the core processing flow of the entire baseline self-evolution closed loop. A persistence determination period N is set, with a constraint N≥2. N is flexibly selected based on the monitoring cycle and security level: for daily-cycle port security, N can be set to 3 to 5, corresponding to 3 to 5 days of continuous confirmation; for weekly-cycle pipeline inspection, N can be set to 2 to 3, corresponding to 2 to 3 weeks of continuous confirmation.

[0100] During long-term underwater monitoring at hourly, daily, and weekly levels, legitimate temporary changes may occur. Typical scenarios include vessels temporarily moored, underwater construction equipment operating at temporary intervals, and seasonal biological communities. These changes are recognized as real disturbances in the current period, but they are only temporary states and do not belong to the long-term evolution of the underwater normal. If the baseline is updated immediately upon a single detection, the temporary object will be permanently written into the model, continuously generating false alarms such as "target disappearance," thus undermining the baseline's ability to define the normal state.

[0101] This invention sets an N-times continuous judgment threshold, and only incrementally updates the benchmark when stable and persistent changes satisfy all constraints. Temporary disturbances are only recorded in the current period and do not modify the benchmark. The N parameters can be adapted to various underwater inspection scenarios.

[0102] For areas confirmed as having undergone genuine changes in the third stage, the system allocates an independent persistent tracking counter. The second and third stages of detection and verification are continuously executed in subsequent N-1 cycles. A baseline incremental update is triggered only when all three of the following constraints are met simultaneously; if any condition is not met, the counter is reset and re-accumulated:

[0103] Within N consecutive cycles, each round of this region is determined to be a real change by bimodal analysis;

[0104] Throughout N rounds of monitoring, the target semantic category remains constant without switching; throughout N rounds of monitoring, the target's 3D coordinate offset does not exceed the preset position tolerance. Differentiated baseline update operations are performed based on the type of change: For newly added objects: the corresponding acoustic point cloud, optical texture, and semantic annotation increments are written into the baseline's three-layer information layer; for disappeared objects: the target is removed from the semantic annotation layer, and the disappearance event log is simultaneously retained; for displaced objects: the semantic layer's 3D target coordinates are updated, and the displacement evolution trajectory is fully stored.

[0105] like Figure 5 As shown, this invention also provides an underwater scene change early warning system adapted to the aforementioned method, which consists of four functional modules forming a closed-loop data flow:

[0106] The benchmark model building module receives the first cycle of sonar and optical acquisition data, performs acoustic-optical fusion 3D reconstruction, and outputs a standardized benchmark scene model that includes a sonar 3D geometry layer, an optical texture layer, and a target semantic annotation layer.

[0107] Change detection and confirmation module: Receives acoustic and optical acquisition data for each cycle and dynamically updated baseline model, and sequentially performs cross-period cascaded registration, three-scale parallel change detection, candidate set fusion, and dual-modal authenticity verification, and outputs a confirmed change set;

[0108] Anomaly assessment and early warning module: Reads the confirmed change set, quantifies the anomaly level based on four factors: category, size, location, and time, and outputs structured early warning data when the score exceeds the threshold;

[0109] The benchmark model dynamic update module receives the confirmed change set, performs differentiated incremental updates on change regions that satisfy N times the triple persistence constraint, and sends the updated benchmark model back to the change detection and confirmation module as the benchmark for the next round of comparison. The data flow is: Benchmark Model Construction Module → Change Detection and Confirmation Module → Anomaly Assessment and Early Warning Module / Benchmark Model Dynamic Update Module → Change Detection and Confirmation Module.

[0110] It should be noted that the acousto-optic sensor model, 3D reconstruction algorithm, point cloud registration method, target detection network, modal consistency scoring function, and anomaly weighting formula are all alternative subordinate implementation methods and do not limit the overall protection scope of this invention. Exemplary alternative solutions include: point cloud registration can be performed using the iterative nearest point algorithm or feature matching registration; optical similarity can be performed using SSIM or learned perceptual metric; and target detection can be performed using CNN or Transformer network.

[0111] In this invention, the dual-modal decoupled verification design principle ensures that the failure mechanisms of underwater acoustic and optical sensors are independent: acoustic failure stems from topographical sparseness, multipath reflection, and sound-absorbing media; optical failure stems from high turbidity, light attenuation, and water scattering. If a tightly coupled acoustic-optical fusion judgment is used, any single-channel performance degradation will cause overall verification failure. This invention employs two heterogeneous scoring functions—acoustic exponential attenuation and optical SSIM—for decoupled dual-channel verification, with no correlation between their failure modes; increased water turbidity only reduces the optical score, leaving the acoustic channel unaffected, allowing independent verification of authenticity and overcoming the limitation of traditional tightly coupled schemes where a single-channel failure results in overall system failure.

[0112] The design principle of N-round persistent updates differs from existing instant update schemes, which modify the baseline upon a single detected change. This results in short-term moored vessels and temporary construction equipment being permanently stored in the model, leading to continuous false alarms and long-term baseline distortion. In long-term monitoring, temporary disturbances exist only briefly, while normal evolution exhibits temporal continuity. This invention uses an N-round triple-synchronous judgment threshold to include only continuously stable changes in the baseline, effectively isolating temporary objects from contaminating the baseline. The N parameter adapts to different inspection cycles. For example, with a 5-day port inspection and N=7, vessels moored for 5 days do not meet the condition of 7 consecutive rounds of stability and will not be included in the baseline; fixed underwater facilities continuously for 30 days are automatically included in the model, and the baseline can adaptively track long-term terrain and facility evolution.

[0113] The design principle of time-domain transient filtering is based on the fact that fish, suspended particulate matter, and water flow disturbances only occur once or in a few cycles, and lack continuity in the time domain; while real changes such as new facilities and pipeline deformation will be detected stably over multiple cycles. A sliding window of length M and a detection frequency threshold K are set, and the cumulative detection frequency of candidate regions is statistically analyzed to filter short-term random interference. Time-domain filtering and dual-modal spatial verification constitute a dual false alarm suppression link, eliminating two types of false changes from the perspectives of time and dimension, respectively.

[0114] The principle of the three-level differentiated detection modal allocation addresses the varying reliability of acoustic and optical methods at different granularities: the target level requires semantic information and employs acoustic-optical fusion; the feature level requires joint texture and geometric features; and pixel-level optics is susceptible to turbidity interference and relies solely on stable acoustic echo differential. This three-level modal allocation is not a general multi-scale image processing method but a targeted design based on the physical characteristics of underwater sensing. In highly turbid port environments with visibility less than 0.5m, the pixel-level acoustic link can continuously detect minute deformations without interrupting the entire detection process.

[0115] Comparison of existing technologies and the working conditions of this invention: Benchmark update strategy comparison with traditional real-time update schemes: In a 30-day daily inspection scenario, temporary vessels staying for 5 days are written to the benchmark in rounds 1-5, and from round 6 onwards, false alarms of target disappearance are continuously output; This invention's N=7 persistence threshold: Short-term vessels cannot meet the stable condition for 7 consecutive rounds, so they are not written to the benchmark, and there are no subsequent invalid warnings for disappearance. Acoustic-optical verification architecture comparison with traditional tightly coupled fusion schemes: After a sudden increase in water turbidity or failure of optical imaging, the quality of fused features declines significantly, and the entire verification process fails; This invention's decoupled dual-channel architecture: When the optical score decreases, the acoustic channel takes effect independently, the change is marked as pending review, the anomaly level is downgraded, and the system maintains basic detection and early warning capabilities.

[0116] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for detecting underwater scene changes and providing early warning of anomalies based on periodic acoustic-optical scanning, characterized in that, comprising: S1. Within a preset monitoring area, construct a reference scene model of the monitoring area through acoustic-optical fusion 3D reconstruction based on sonar data and optical image data collected in the first cycle; The reference scene model contains three layers of spatially aligned information: sonar 3D geometric information, optical texture information, and target semantic annotation; S2. Perform cross-temporal spatial registration on the sonar data and optical image data collected in each subsequent cycle. After the registration is completed, change detection is executed in parallel at three scales: target level, feature level, and pixel level, and the detection outputs of the three scales are fused to generate a change candidate set; Wherein the pixel-level change detection uses single-modal data of acoustic echo intensity to independently maintain the pixel-scale detection capability when the optical image fails due to water turbidity; S3. After performing temporal transient filtering on the change candidate set to eliminate short-term interference, a sonar and optical dual-modal independent decoupling scoring mechanism is adopted for cross-validation to screen out false alarms and output a confirmed change set; S4. Classify each change region in the confirmed change set, calculate a comprehensive abnormality score by weighting four factors: the category of the changed object, size, position sensitivity, and change time, and output structured early warning information when the comprehensive score exceeds a preset early warning threshold; S5. Continuously track each change region in the confirmed change set. When the same change region satisfies three persistence determination conditions for N consecutive cycle acquisitions, where N≥2, the change region is incrementally fused into the reference scene model to complete the update; the three persistence determination conditions are: each acquisition is determined as a real change by the dual-modal confirmation mechanism, the semantic category of the changed object remains consistent throughout the process, and the 3D position deviation falls within the preset tolerance range; The steps S1 to S5 are connected end to end to form a long-term underwater monitoring closed-loop self-evolution mechanism, which distinguishes temporary underwater disturbances from normal persistent evolution, and realizes abnormal early warning and adaptive update of the reference model synchronously; 2. The underwater scene change detection and anomaly early warning method based on periodic acoustic-optical scanning according to claim 1, characterized in that, The dual-modal independent decoupling scoring mechanism in S3 specifically includes: using heterogeneous and mutually independent scoring functions to calculate the sonar modal consistency score Cs and the optical modal consistency score Co respectively; the sonar modal adopts an exponential decay scoring function based on the acoustic echo feature vector, and the optical modal adopts a scoring function based on image structural similarity SSIM; three levels of logical determination are executed: (1) Both Cs and Co exceed the preset confirmation threshold T_confirm, which is determined as a real change and directly input into the confirmed change set; (2) Only Cs or only Co, a single modal, exceeds T_confirm, which is determined as a change to be reviewed, and the abnormality level of this change region is synchronously reduced by one level before being input into the confirmed change set; (3) Both Cs and Co are lower than T_confirm, which is directly determined as a false alarm and permanently eliminated; 3. The underwater scene change detection and anomaly early warning method based on periodic acoustic-optical scanning according to claim 1, characterized in that, The temporal transient filtering in S3 is executed after the change candidate set is generated and before dual-modal confirmation. The specific rules are: set the observation window length M, where M≥3, and the occurrence count threshold K, where K<M; count the cumulative detected times of a single change candidate region in M consecutive cycle acquisitions, if the cumulative detected times ≤ K, it is determined as transient interference caused by fish, suspended particles, and water flow disturbance and directly eliminated.

4. The underwater scene change detection and anomaly early warning method based on periodic acoustic-optical scanning according to claim 1, characterized in that, The differentiated execution logic for target-level, feature-level, and pixel-level change detection in S2 is as follows: Target-level change detection: Based on the target category and three-dimensional coordinates output by acoustic-optical fusion target detection, a comprehensive matching cost function is constructed. The optimal matching algorithm is used to match the current target with the benchmark model target, and three types of changed targets are marked: added, disappeared, and displaced. Feature-level change detection: Acoustic geometric features and optical texture features are fused to generate a joint feature map. The difference between the current feature map and the benchmark feature map is measured region by region, and structurally changed regions are marked. Pixel-level change detection: The acoustic echo intensity map is extracted, and pixel-by-pixel difference operation is performed with the benchmark acoustic echo map to mark pixel-level changes.

5. The underwater scene change detection and anomaly early warning method based on periodic acoustic-optical scanning according to claim 1, characterized in that, The S2 cross-period spatial registration adopts an acoustic-first, optical-assisted cascade registration strategy, including: extracting the current sonar 3D point cloud and the reference model acoustic geometry layer point cloud, and performing rigid registration using the iterative nearest point algorithm; if the acoustic registration convergence residual exceeds the limit or the number of iterations reaches the upper limit, the registration is judged to have failed, and the current optical image is extracted and matched with the reference texture layer visual feature points, and the sonar rigid transformation matrix is ​​calculated by inverse calculation of the sonar extrinsic parameters through sonar-camera joint calibration to complete the secondary registration.

6. The underwater scene change detection and anomaly early warning method based on periodic acoustic-optical scanning according to claim 1, characterized in that, The construction process of the three-layer information of the reference scene model in S1 is as follows: a three-dimensional acoustic point cloud of the monitoring area is generated by multi-beam sonar ranging and beamforming; based on the pre-calibrated rigid external parameters of the sonar-camera, the corrected underwater optical image texture is mapped onto the surface of the acoustic three-dimensional point cloud; three-dimensional target detection is performed on the textured three-dimensional scene model, and semantic category, three-dimensional bounding box, global coordinates, and detection confidence are labeled for each target.

7. The underwater scene change detection and anomaly early warning method based on periodic acoustic-optical scanning according to claim 1, characterized in that, In S5, the incremental update of the baseline model performs a differentiated fusion operation based on the type of change: if it is a new object, the corresponding acoustic point cloud, optical texture, and semantic annotation increments are added to the three layers of information of the baseline model; If the object disappears: remove the object from the target semantic annotation layer and retain the disappearance event log; if the object is displaced: update the target's 3D coordinates within the semantic annotation layer and record the complete displacement trajectory; The updated baseline model is fed back to S2 as the baseline for registration and change detection in the next cycle.

8. The underwater scene change detection and anomaly early warning method based on periodic acoustic-optical scanning according to claim 1, characterized in that, The periodic data acquisition carrier is an underwater vehicle integrating sonar and optical camera. The vehicle is equipped with an inertial navigation unit, Doppler velocity meter, and depth sensor to output six-degree-of-freedom pose markers for each acquisition. The preset monitoring area includes underwater security zones of ports, submarine pipeline corridors, underwater infrastructure surroundings, and marine ranch cages. The acquisition period is configured as hourly, daily, or weekly according to the safety level.

9. An underwater scene change detection and anomaly early warning system based on periodic acoustic-optical scanning, characterized in that, The method for implementing any one of claims 1 to 8 comprises: a baseline model construction module, which executes the baseline scene modeling step S1 and outputs an initial baseline scene model containing three layers of information; a change detection and confirmation module, which executes the cross-period multi-scale change detection, the temporal filtering and dual-modal decoupling confirmation step S3, receives the acoustic and optical acquisition data and the updated baseline model for each cycle, and outputs a confirmed change set; an anomaly assessment and early warning module, which executes the anomaly classification assessment and early warning step S4, receives the confirmed change set and outputs structured early warning information; and a baseline model dynamic update module, which executes the baseline model incremental self-evolution update step S5, receives the confirmed change set, performs incremental updates on the change regions that meet the N-times persistence condition, and sends the updated baseline model back to the change detection and confirmation module to form a closed-loop data stream.

10. A computer-readable storage medium storing a computer program thereon, characterized in that, When the computer program is executed by the processor, it implements all the steps of the underwater scene change detection and anomaly early warning method based on periodic acoustic-optical scanning as described in any one of claims 1 to 8.

Images