Waterway beacon state monitoring method and system based on deep learning

Abstract

The application discloses a waterway navigation mark state monitoring method and system based on deep learning, relates to the ship monitoring technical field, and fuses shore-based video, shipborne video and attitude, radar echo, AIS track and hydrodynamic environment data, constructs a navigation mark alignment observation data set and generates a drift constraint parameter set, combines multi-modal deep perception and prior constraint state estimation to output navigation mark state parameters and reliability, realizes reliable discrimination and trend early warning of drift, inclination, lamp quality and structure anomaly under complex conditions such as fog, rain and night, and outputs review collection and collection frequency regulation strategies, reduces false positives and false negatives, and improves operation and maintenance efficiency.

Description

Technical Field

[0001] This invention relates to the field of ship monitoring technology, and in particular to a method and system for monitoring the status of navigation marks in waterways based on deep learning. Background Technology

[0002] Waterways rely on buoys, light buoys, and buoys to guide vessels safely. Due to low visibility at night in fog and rain, strong currents and winds, swells, and ship wake disturbances, buoys are prone to problems such as slow drifting, tilting, abnormal draft, light quality degradation, or abnormal flashing. Current maintenance largely depends on manual inspections or single shore-based video surveillance, which is prone to misjudgment when encountering obstructions, strong reflections, or rain, fog, and noise. Furthermore, a single perspective makes it difficult to distinguish between actual buoy deviations and observation platform vibrations, hindering the timely detection of gradual anomalies and prioritization of appropriate actions. This results in potential hazards accumulating over a long period before being discovered.

[0003] Currently, Chinese invention patent application number 202411833595.9 discloses a method and system for monitoring the operational status of water transport vessels based on waterway information fusion. The method includes: acquiring multi-source data of a target waterway, including real-time vessel navigation data and waterway environmental data, and conducting data value assessment; fusing the multi-source data based on the assessment results to generate comprehensive status monitoring data of the target waterway, and constructing a water transport vessel monitoring platform; using the platform to conduct real-time monitoring of vessels in the target waterway to obtain operational status data; and assessing the environmental adaptability of vessels based on this data, thereby optimizing the allocation strategy of monitoring resources. This invention improves the accuracy and resource utilization efficiency of water transport vessel monitoring and is applicable to intelligent waterway management.

[0004] The aforementioned technologies are insufficient to reliably identify and provide early warnings of navigational beacon drift, tilt, light quality, and structural anomalies based on observations in complex environments such as foggy and rainy nights, strong currents, and strong winds, while also outputting executable verification and data acquisition control strategies. Summary of the Invention

[0005] The technical problem solved by this invention is that existing technologies are unable to reliably identify and provide early warnings of navigation mark drift, tilt, light quality and structural anomalies based on observation in complex environments such as foggy and rainy nights and strong currents and winds, while outputting executable verification and data acquisition control strategies.

[0006] To solve the above-mentioned technical problems, the present invention provides the following technical solution: The deep learning-based method for monitoring the status of navigation aids in waterways includes the following steps: Step S1: Collect shore-based video data, shipborne video data, shipborne attitude data, navigation beacon radar echo data, ship AIS track data, and hydrodynamic environment data, and perform time synchronization and spatial alignment to form a navigation beacon aligned observation dataset; Step S2: Based on the beacon alignment observation dataset, construct beacon reference data and drift prior constraint data, and form a drift constraint parameter set; Step S3: Input the navigation mark alignment observation dataset into the multimodal depth perception model, and output navigation mark target data and navigation mark observation feature data; Step S4: Input the navigation mark observation feature data and drift constraint parameter set into the prior constraint state network, and output the navigation mark state parameter data; Step S5: Based on the navigation mark status parameter data, perform anomaly identification and output abnormal navigation mark status data and monitoring and handling strategy data; Step S6: Transmit the monitoring and handling strategy data to the control terminal.

[0007] Preferably, step S1 includes the following sub-steps: Step S101: Collect shore-based video data and shipborne video data, and extract shooting timestamps and camera pose information; Step S102: Collect shipboard attitude data and align it with shipboard video data based on timestamps; Step S103: Collect radar echo data of navigation beacons and extract target range and azimuth sequence; Step S104: Collect ship AIS track data and generate waterway traffic density data; Step S105: Collect hydrodynamic environmental data, including water level data, flow velocity and direction data, and wind speed and direction data; Step S106: Synchronize and spatially align shore-based video data, shipborne video data, shipborne attitude data, navigation beacon radar echo data, ship AIS track data, and hydrodynamic environment data to output a navigation beacon aligned observation dataset.

[0008] Preferably, the spatial alignment is as follows: A mapping relationship between shore-based pixel coordinates and channel plane coordinates is established based on camera pose information from shore-based video data and channel electronic reference map data. Attitude compensation is performed on shipborne video data based on shipborne attitude data, and a mapping relationship between shipborne pixel coordinates and channel plane coordinates is established. Coordinate transformation is performed between the target range and azimuth sequence based on navigation radar echo data and the channel plane coordinates. The mapped shore-based video data, shipborne video data, and converted navigation beacon radar echo data are resampled and fused according to a unified waterway plane coordinate system to form a navigation beacon aligned observation dataset with a unified coordinate index.

[0009] Preferably, step S2 includes the following sub-steps: Step S201: Obtain navigation mark reference coordinate data from the electronic reference chart data of the waterway and generate waterway reference grid data; Step S202: Calculate disturbance intensity data based on hydrodynamic environment data and ship AIS track data; Step S203: Establish prior drift constraint data based on disturbance intensity data and hydrodynamic environment data. The prior drift constraint data includes drift direction distribution data and upper bound drift velocity data. Step S204: Combine the drift direction distribution data with the upper bound data of drift velocity to generate a drift constraint parameter set.

[0010] Preferably, step S3 includes the following sub-steps: Step S301: Construct visual feature tensor data based on shore-based video data and shipborne video data, and perform temporal encoding on the visual feature tensor data; Step S302: Construct radar feature sequence data based on navigation beacon radar echo data and perform time-series encoding on the radar feature sequence data; Step S303: Construct environmental traffic feature data based on ship AIS track data and hydrodynamic environment data, and embed the features. Step S304: Input the visual feature tensor data, radar feature sequence data and environmental traffic feature data into the multimodal depth perception model for cross-modal attention fusion, and output navigation target data and navigation observation feature data.

[0011] Preferably, the beacon observation feature data includes light quality timing feature data, shape and structure feature data, and radar stability feature data; The light quality temporal characteristic data is generated from the brightness time series of the luminous area of ​​the navigation mark in the visual feature tensor data and includes flash period data and on / off duty data. The external structural feature data is generated from the target contour sequence corresponding to the navigation target data and includes contour integrity data and reflectivity data; The radar stability characteristic data is generated from radar characteristic sequence data and includes scattering intensity fluctuation data and target track continuity data.

[0012] Preferably, step S4 includes the following sub-steps: Step S401: Calculate the navigation mark offset data based on the navigation mark reference coordinate data and the navigation mark target data. The navigation mark offset data includes lateral offset data and lateral offset data. Step S402: Input the navigation beacon observation feature data and navigation beacon deviation data into the prior constraint state network for time-series state estimation, and output the navigation beacon attitude state data and navigation beacon drift trend data. Step S403: Perform prior consistency verification on the navigation beacon drift trend data based on the drift constraint parameter set. If the drift direction distribution data is inconsistent with the navigation beacon drift trend data or the upper limit of the drift speed data is exceeded, reduce the credibility of the navigation beacon state parameter data and mark it as a state that needs to be reviewed. Step S404: Combine the beacon offset data, beacon attitude status data, beacon drift trend data, and reliability data to output beacon status parameter data.

[0013] Preferably, step S5 includes the following sub-steps: Step S501: Generate anomaly discrimination feature data based on beacon status parameter data. The anomaly discrimination feature data includes the over-limit amount of lateral deviation data, the tilt amount of beacon attitude status data, the period deviation amount of light quality timing feature data, and the decrease in continuity of radar stability feature data. Step S502: Based on the anomaly discrimination feature data and the confidence data, the abnormal navigation mark status data is discriminated. The abnormal navigation mark status data includes drift anomaly data, tilt anomaly data, light quality anomaly data and structural anomaly data. Step S503: Generate monitoring and handling strategy data based on abnormal navigation mark status data and credibility data. The monitoring and handling strategy data includes review collection strategy data and collection frequency adjustment strategy data.

[0014] Preferably, the logic for generating the monitoring and handling strategy data is as follows: If abnormal navigation mark status data exists and the credibility data is greater than or equal to the preset credibility threshold, then the verification collection strategy data will be output and the collection frequency will be adjusted to a high-frequency collection frequency. If abnormal navigation mark status data exists and the credibility data is less than the preset credibility threshold, then the verification collection strategy data will be output and the collection frequency will be adjusted to the medium frequency collection frequency. If abnormal navigational beacon status data is not available, the acquisition frequency adjustment strategy data is output and the baseline acquisition frequency is maintained.

[0015] The deep learning-based waterway navigation mark status monitoring system includes a data acquisition and alignment module, a benchmark construction and constraint module, a deep perception and reasoning module, a prior constraint estimation module, an anomaly detection and generation module, and a strategy transmission and control module. The data acquisition and alignment module is used to acquire shore-based video data, shipborne video data, shipborne attitude data, navigation beacon radar echo data, ship AIS track data, and hydrodynamic environment data, and to perform time synchronization and spatial alignment to form a navigation beacon alignment observation dataset. The benchmark construction constraint module is used to construct beacon benchmark data and drift prior constraint data based on the beacon alignment observation dataset and form a drift constraint parameter set. The deep perception inference module is used to input the navigation mark alignment observation dataset into the multimodal deep perception model and output navigation mark target data and navigation mark observation feature data. The prior constraint estimation module is used to input the navigation mark observation feature data and the drift constraint parameter set into the prior constraint state network and output the navigation mark state parameter data. The anomaly detection generation module is used to detect anomalies based on navigation mark status parameter data and output abnormal navigation mark status data and monitoring and handling strategy data. The strategy transmission and control module is used to transmit monitoring and handling strategy data to the control terminal.

[0016] The beneficial effects of this invention are as follows: This invention integrates shore-based video, shipborne video, attitude, radar echo, AIS track, and hydrodynamic environment data to construct a navigation beacon alignment observation dataset and generate a drift constraint parameter set. By combining multimodal depth perception and prior constraint state estimation, it outputs navigation beacon state parameters and reliability, enabling reliable discrimination and trend warning of drift, tilt, light quality, and structural anomalies under complex conditions such as fog, rain, and night. It also outputs verification acquisition and acquisition frequency adjustment strategies to reduce false alarms and missed alarms and improve operation and maintenance efficiency. Attached Figure Description

[0017] Figure 1 A flowchart illustrating the steps of a deep learning-based method for monitoring the status of navigation aids in waterways, as provided in one embodiment of the present invention. Figure 2 This is a basic flowchart of a deep learning-based waterway navigation mark status monitoring system provided in one embodiment of the present invention. Detailed Implementation

[0018] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0019] Example 1, referring to Figure 1 This paper presents a deep learning-based method for monitoring the status of navigation marks in waterways, including the following steps: Step S1: Collect shore-based video data, shipborne video data, shipborne attitude data, navigation beacon radar echo data, ship AIS track data, and hydrodynamic environment data, and perform time synchronization and spatial alignment to form a navigation beacon aligned observation dataset.

[0020] Step S2: Based on the navigation mark alignment observation dataset, construct navigation mark reference data and drift prior constraint data, and form a drift constraint parameter set.

[0021] Step S3: Input the navigation mark alignment observation dataset into the multimodal depth perception model and output navigation mark target data and navigation mark observation feature data.

[0022] Step S4: Input the navigation mark observation feature data and drift constraint parameter set into the prior constraint state network, and output the navigation mark state parameter data.

[0023] Step S5: Based on the navigation mark status parameter data, perform anomaly identification and output abnormal navigation mark status data and monitoring and handling strategy data.

[0024] Step S6: Transmit the monitoring and handling strategy data to the control terminal.

[0025] This embodiment focuses on a verification scenario for inland waterways. Shore-based cameras are deployed along both banks of the waterway to acquire shore-based video data. Inspection vessels or work vessels are equipped with shipborne cameras and inertial measurement units to acquire shipborne video and attitude data. Shore-based or near-navigation buoy radar devices output navigation buoy radar echo data. The management platform accesses ship AIS track data and hydrodynamic environmental data output from water level stations and meteorological stations. The platform uses the waterway's planar coordinates as a unified spatial reference. The database contains pre-set electronic reference map data for the waterway, including navigation buoy reference coordinate data and a geometric description of the waterway centerline, for subsequent spatial alignment and offset calculations. Steps S1 to S6 are executed at the reference acquisition frequency. After outputting monitoring and handling strategy data, the control terminal adjusts the acquisition frequency of the shore-based and shipborne cameras based on the acquisition frequency adjustment strategy data, and sends the verification acquisition strategy data to the inspection task module to trigger close-range verification acquisition.

[0026] This invention integrates shore-based video, shipborne video, attitude, radar echo, AIS track, and hydrodynamic environment data to construct a navigation beacon alignment observation dataset and generate a drift constraint parameter set. Combining multimodal depth perception and prior constraint state estimation, it outputs navigation beacon state parameters and reliability, enabling reliable discrimination and trend warning of drift, tilt, light quality, and structural anomalies under complex conditions such as fog, rain, and night. It also outputs verification acquisition and acquisition frequency adjustment strategies to reduce false alarms and missed alarms and improve operation and maintenance efficiency.

[0027] Step S1 includes the following sub-steps: Step S101: Collect shore-based video data and shipboard video data, and extract the shooting timestamp and camera pose information.

[0028] In this embodiment, the shore-based video data includes image frame data arranged in frame order and corresponding timestamps, while the shipborne video data includes image frame data and corresponding timestamps. Camera pose information includes calibration results for camera installation height, pitch angle, and azimuth angle, used to establish the mapping relationship between pixel coordinates and channel plane coordinates. To ensure the computability of subsequent light quality temporal feature data, both the shore-based and shipborne video data retain uncompressed luminance channels or equivalent grayscale channels, and the exposure time and gain parameters are retained as quality control fields and written into the metadata of the beacon alignment observation dataset.

[0029] Step S102: Collect shipboard attitude data and align it with shipboard video data based on timestamps.

[0030] Shipborne attitude data includes time series of roll angle, pitch angle, heading angle and angular velocity. Each frame of shipborne video data is associated with the shipborne attitude data at the same time by nearest neighbor time matching or linear interpolation, and attitude compensation parameter data required for attitude compensation is generated. This attitude compensation parameter data is only used as an intermediate quantity in spatial alignment calculation and does not appear as a new independent data term in subsequent transmission. It is still written into the beacon alignment observation dataset in the end.

[0031] Step S103: Collect radar echo data of navigation marks and extract target range and azimuth sequence.

[0032] The beacon radar echo data includes the echo intensity array for each scanning cycle. Constant false alarm rate (CFAR) detection is performed on the echo intensity array to obtain the target point set. The target point set is then correlated with the flight path, and the target range-azimuth sequence is output as the structured part of the beacon radar echo data, which is used for subsequent radar stability feature data calculation and spatial coordinate transformation.

[0033] Step S104: Collect ship AIS track data and generate waterway traffic density data.

[0034] Ship AIS track data includes ship identifier, latitude and longitude, speed, heading, and timestamp, with a radius set centered on the navigational beacon reference coordinates. The scope of influence and the time window The internal statistics include the number of ships passing by and their average speed, set at... The ship set within the time window is , No. The ship in The position is The reference coordinates of the navigation mark are The distance is Generate waterway traffic density data : ; ; in, Distance weights; The waterway traffic density data is written into the navigation mark alignment observation dataset and used in step S2 for disturbance intensity data calculation.

[0035] Step S105: Collect hydrodynamic environmental data, including water level data, flow velocity and direction data, and wind speed and direction data.

[0036] Hydrodynamic environmental data were collected and a unified time-granularity hydrodynamic environmental data sequence was formed. The hydrodynamic environmental data included water level data, flow velocity and direction data, and wind speed and direction data. Data from each station was mapped to the channel reference grid according to the grid where the navigation mark is located. Data from different sampling periods were resampled to obtain a consistent time series that is either consistent with or interpolable to the timestamps of the shore-based video data. This time series was written into the navigation mark-aligned observation dataset for subsequent calculations of drift prior constraints and upper bounds of drift velocity.

[0037] Step S106: Synchronize and spatially align shore-based video data, shipborne video data, shipborne attitude data, navigation beacon radar echo data, ship AIS track data, and hydrodynamic environment data to output a navigation beacon aligned observation dataset.

[0038] Time synchronization employs a unified time base, mapping timestamps from all data sources to the same clock and removing samples exceeding the synchronization threshold. Spatial alignment utilizes two types of mapping: shore-based video data establishes a homography mapping matrix from pixel coordinates to channel plane coordinates based on camera pose information and channel electronic reference map data. Pixels of shore-based video data Represented in homogeneous coordinates as ,for pixel The horizontal coordinates in the image, For pixels Given the vertical coordinates of the point in the image and the horizontal coordinates of the channel, then the homography matrix is... satisfy ,in, Obtained by calibration using camera pose information and electronic reference map data of the waterway. This indicates proportionality in homogeneous coordinates. Shipborne video data is first used for attitude compensation based on shipborne attitude data, rotating the line-of-sight direction of the image frames to a stable reference. Then, a mapping matrix is ​​established using camera pose information, thereby reducing the impact of ship sway on beacon position estimation.

[0039] Target range and azimuth sequence obtained from radar echo data analysis The origin of the radar coordinate system is The conversion between the radar coordinate system and the channel plane coordinate system into a translation vector. With rotation angle ,but: ; in, These are the transformed planar coordinate points, used for unified fusion with the coordinates aligned to the video. The aligned multi-source samples form a beacon-aligned observation dataset with a unified coordinate index and a unified time index, ensuring that subsequent model inputs can directly retrieve multimodal observations of the same beacon at the same time.

[0040] Spatial alignment is as follows: A mapping relationship between shore-based pixel coordinates and channel plane coordinates is established based on camera pose information from shore-based video data and channel electronic reference map data.

[0041] Attitude compensation is performed on shipborne video data based on shipborne attitude data, and a mapping relationship between shipborne pixel coordinates and channel plane coordinates is established.

[0042] Coordinate transformation is performed between the target range and azimuth sequence based on navigation radar echo data and the channel plane coordinates.

[0043] The mapped shore-based video data, shipborne video data, and converted navigation beacon radar echo data are resampled and fused according to a unified waterway plane coordinate system to form a navigation beacon aligned observation dataset with a unified coordinate index.

[0044] Step S2 includes the following sub-steps: Step S201: Obtain navigation mark reference coordinate data from the electronic reference map data of the waterway and generate waterway reference grid data.

[0045] The channel reference grid data is used to align the results of shore-based video data, shipborne video data, and navigation beacon radar echo data into a unified grid. It is also used to calculate the lateral and longitudinal offset data in the navigation beacon offset data. The lateral direction is taken as the normal direction of the channel centerline, and the longitudinal direction is taken as the tangential direction of the channel centerline. For each navigation beacon reference coordinate data, the nearest centerline point is calculated, and the tangential and normal vectors at that point are obtained as the reference for offset decomposition.

[0046] Step S202: Calculate disturbance intensity data based on hydrodynamic environment data and ship AIS track data.

[0047] Disturbance intensity data is used to characterize the disturbance level of the flow field near the navigation mark due to the combined effects of the environment and traffic, and is used for the subsequent generation of drift prior constraint data. This embodiment uses disturbance intensity data... Defined as the weighted sum of normalized flow velocity amplitude, normalized wind speed amplitude, and waterway traffic density data, i.e.: ; in, For the velocity amplitude, For wind speed amplitude, This is a normalization constant for the upper bound of the flow velocity amplitude. This is the normalization constant for the upper bound of the wind speed amplitude. This is a normalization constant for the upper bound of waterway traffic density data. This definition makes... The increase occurs during periods of strong currents, strong winds, or dense shipping traffic, providing an interpretable basis for upward adjustments to the upper limit of drift speed data. , and These are the weights for the influence of flow velocity, wind speed, and traffic, respectively, and they satisfy the following relationship: ; ; , , Calculated using historically confirmed drift samples. These samples include hydrodynamic environmental data, channel traffic density data, and the actual drift speed of the navigation mark, confirmed through inspection or manual verification. Let the first... The normalized values ​​of flow velocity, wind speed, and traffic density in the historical drift samples are as follows: ; ; ; Let the first The actual drift speed of the beacons in the historical confirmed drift samples was: The normalized upper bound of the drift velocity is The actual drift response value is: ; Among them, the actual drift speed of the navigation mark Calculated from the center point of the same navigation mark at adjacent confirmation times: ; in, For the first The center point of the navigational target at the confirmed moment. The center point of the navigation target at the previous confirmation time. The time interval between the two confirmation moments.

[0048] Based on multiple sets of historically confirmed drift samples, the constrained least squares method is used to determine... , , : ; The constraints are: ; ; When the number of historically confirmed drift samples is less than the preset sample number threshold, , , Using initial equilibrium weights: ; Once the number of historically confirmed drift samples reaches a preset sample size threshold, the constrained least squares method described above is used to recalculate. , , For example, in 180 historically confirmed drift samples of a certain inland waterway, calculations yielded... =0.52, =0.28, =0.20 indicates that the drift of navigation marks in this section is mainly affected by the current speed, followed by the wind speed, and the impact of ship traffic disturbance is relatively small.

[0049] Step S203: Establish prior drift constraint data based on disturbance intensity data and hydrodynamic environment data. The prior drift constraint data includes drift direction distribution data and upper bound drift velocity data.

[0050] Drift direction distribution data Orientation statistics derived from historical drift samples, which can be obtained through manual surveys or long-term estimations, show the orientation angles of the historical drift vectors. Discretized to The sector, recorded as falling into the first sector. The number of samples in each sector is The total sample size is ,but: ; And segmented priors are obtained by segmenting the data according to the disturbance intensity, so that in different The distribution has different directions within the range.

[0051] Upper bound data of drift speed Generated by hydrodynamic environment data and disturbance intensity data, this embodiment uses: ; in, , , and The drift constraint parameter set is determined by fitting historical samples and ensures that it covers the normal drift velocity at a given confidence level. The upper bound data of the drift velocity and the distribution data of the drift direction together constitute the drift constraint parameter set.

[0052] The drift direction distribution data and the upper bound data of drift velocity jointly constrain the drift trend output in step S4, enabling the present invention to distinguish between actual drift and observed jitter.

[0053] Step S204: Combine the drift direction distribution data with the upper bound data of drift velocity to generate a drift constraint parameter set.

[0054] The drift constraint parameter set is updated over time and bound to the observation dataset aligned with the beacon by time index. Subsequent prior consistency checks directly read the direction prior and velocity upper bound at the corresponding time from the drift constraint parameter set.

[0055] Step S3 includes the following sub-steps: Step S301: Construct visual feature tensor data based on shore-based video data and shipborne video data, and perform temporal encoding on the visual feature tensor data.

[0056] The visual encoder uses a convolutional backbone network to extract multi-scale features from each frame, and then uses temporal attention to aggregate the features within a time window, outputting visual feature tensor data. To ensure the computability of subsequent lamp quality temporal feature data, the brightness time series of the beacon's luminous area is extracted simultaneously during the visual encoding stage. The brightness time series is the median of the pixel brightness within the beacon target area and is formed as a sequence over time.

[0057] Step S302: Construct radar feature sequence data based on navigation beacon radar echo data and perform time-series coding on the radar feature sequence data.

[0058] The scattering intensity statistics at each moment are calculated for the target range-azimuth sequence to form radar feature sequence data. The scattering intensity can be taken as the mean or quantile of the concentrated echo intensity of the target point. Then, a gated recurrent network is used for time-series encoding to obtain a robust characterization of intermittent point loss. The output is radar feature sequence data that is time-aligned with the visual feature tensor data.

[0059] Step S303: Construct environmental traffic feature data based on ship AIS track data and hydrodynamic environment data, and embed the features.

[0060] The environmental traffic feature data includes waterway traffic density data, flow velocity and direction data, wind speed and direction data, and water level data. The system embeds the flow direction and wind direction in a sine and cosine form to eliminate angular discontinuities, and performs differential analysis on the water level data to obtain the water level change rate to enhance the sensitivity to short-term fluctuations, forming environmental traffic feature data that can be input into the fusion network.

[0061] Step S304: Input the visual feature tensor data, radar feature sequence data and environmental traffic feature data into the multimodal depth perception model for cross-modal attention fusion, and output navigation target data and navigation observation feature data.

[0062] The multimodal depth perception model includes a cross-modal attention layer that uses visual features as queries and performs key-value attention on radar and environmental features. This allows for compensation of localization and stability assessments by radar and environmental features when visual quality deteriorates in fog, rain, or at night. The navigational aid target data includes the target center point sequence and target outline sequence in the channel plane coordinates. The navigational aid observation feature data includes light quality time-series feature data, shape and structure feature data, and radar stability feature data. The light quality time-series feature data is calculated from the brightness time series, the shape and structure feature data is calculated from the target outline sequence, and the radar stability feature data is calculated from the radar feature sequence data.

[0063] Let the time series of brightness of the luminous area of ​​the navigation mark be as follows: The peak time sequence is , This represents the total number of peak moments detected within a statistical time window. For the first The time of occurrence of the second flash peak, and the flash period data in the lamp quality timing characteristic data. for: ; Let the threshold for determining the bright state be . , No. The duration of the bright state within each cycle is: On / off duty data for: ; ; in, This is an indicator function.

[0064] The contour integrity data in the shape and structure feature data can be defined as the ratio of the current contour area to the contour area during the historical stable period, and the reflectivity data can be defined as the ratio of the average brightness of the reflective area to the average brightness of the background. The scattering intensity fluctuation data in radar stability characteristic data is taken as the standard deviation of the scattering intensity sequence, that is: Let the scattering intensity sequence be Sampling Then the scattering intensity fluctuation data for: ; The target track continuity data is taken as the ratio of the number of valid track points to the theoretical number of track points within the time window, i.e.: Let the theoretical number of track points within the window be... The actual number of valid trackpoints is Then the target track continuity data for: ; All of the above features are written into the navigation mark observation feature data and then proceed to the state estimation in step S4.

[0065] The navigational aid observation characteristic data includes light quality timing characteristic data, shape and structure characteristic data, and radar stability characteristic data.

[0066] The light quality temporal characteristic data is generated from the brightness time series of the luminous area of ​​the navigation mark in the visual feature tensor data and includes flash period data and on / off duty data.

[0067] The shape and structure feature data are generated from the target outline sequence corresponding to the navigation target data and include outline integrity data and reflectivity data.

[0068] Radar stability characteristic data is generated from radar characteristic sequence data and includes scattering intensity fluctuation data and target track continuity data.

[0069] Step S4 includes the following sub-steps: Step S401: Calculate the navigation mark offset data based on the navigation mark reference coordinate data and the navigation mark target data. The navigation mark offset data includes lateral offset data and lateral offset data.

[0070] The center point of the navigation target at each moment Calculate the difference vector with the corresponding navigational reference coordinates. and will Decomposed on the tangential and normal bases of the channel centerline, the normal component is the lateral offset data. ( (The unit normal vector), the tangential component is the offset data along the direction. ( (as a unit tangential vector), thus ensuring that the offset is consistent with the channel geometry, making it easy to use for direct judgment of operation and maintenance thresholds.

[0071] Step S402: Input the navigation beacon observation feature data and navigation beacon deviation data into the prior constraint state network for time-series state estimation, and output the navigation beacon attitude state data and navigation beacon drift trend data.

[0072] The prior-constrained state network maintains the state vector in a time-series recursive manner. The state vector must contain at least lateral offset data, lateral offset data, tilt amount, and drift velocity. for: ; in, The sampling interval; Drift direction angle for: ; Observation vector The system is constructed by concatenating beacon offset data, light quality timing characteristics, shape and structural characteristics, and radar stability characteristics. The network employs a gated recursive structure to output state increments and state uncertainties. Training is used to ensure the output state fits the actual state obtained from manual annotation or high-precision measurement. The training loss includes state regression loss and consistency loss. The state regression loss is the weighted squared error between the predicted and actual states, while the consistency loss constrains the smoothness of state changes over short periods, preventing the misclassification of observation noise as drift. Beacon attitude state data is formed by the tilt and its rate of change in the state vector. Beacon drift trend data is formed by the drift velocity and drift direction in the state vector, where the drift direction is calculated from the first-order difference direction of the lateral and longitudinal offsets.

[0073] Step S403: Perform prior consistency verification on the beacon drift trend data based on the drift constraint parameter set. If the drift direction distribution data is inconsistent with the beacon drift trend data or the upper limit of the drift speed data is exceeded, reduce the credibility of the beacon state parameter data and mark it as a state that needs to be reviewed.

[0074] Read the drift direction distribution data and drift velocity upper bound data at the same time from the drift constraint parameter set, and calculate the two types of violations.

[0075] Speed ​​violation quantity The mathematical expression for penalizing outputs exceeding the upper bound is: ; Direction violation maps the predicted direction to the sector index. ,in, Let the predicted distribution be the total number of directional sectors into which the navigation beacon drift direction is divided. For frequency statistics within the window, the direction deviation is... for: ; in, Data from drift direction distribution The prior violation is obtained by weighting the velocity violation and the orientation violation based on the upper bound data of drift velocity. And combine multi-source consistency to obtain credibility data. Let the estimated location of the shore base be... The estimated position of the ship is The radar estimated the location as follows Then the cross-modal deviation for: ; Credibility data for: ; ; in, and As weight, The function is a sigmoid function, composed of the root mean square of the differences between shore-based and shipborne position estimates and the radar position estimate. If the prior violation is large or the cross-modal deviation is large, the confidence level is reduced and marked as a state requiring review, which is then used for strategy generation in step S5.

[0076] Among them, for and Let the first In the training samples for historical credibility, the normalized prior violation is: The normalized cross-modal deviation is The trusted labels confirmed by manual review are .in =1 indicates that the navigation mark status assessment has been manually verified and confirmed to be reliable. =0 indicates that the navigation mark status assessment was confirmed as unreliable after manual review.

[0077] Then it is determined through logistic regression training. and : ; in: ; The constraints are >0, ≥0, ≥0, for example, trained based on 200 sets of historical credibility training samples for a certain flight segment. =3.0, =1.8, =1.2, then the credibility data is: ; when =0.1、 When =0.2: = ≈0.93 indicates that the reliability of the navigation mark status assessment is relatively high.

[0078] when =1.0、 When =1.2: = ≈0.44 indicates that the reliability of the navigation mark status assessment is low and needs to be verified.

[0079] Step S404: Combine the beacon offset data, beacon attitude status data, beacon drift trend data, and reliability data to output beacon status parameter data.

[0080] Navigational beacon status parameter data serves as the core data for external output. It is also written into the database and displayed on the operation and maintenance dashboard. Among them, the reliability data is used to explain the reliability of each abnormal conclusion and to adjust the collection frequency at the control end.

[0081] Step S5 includes the following sub-steps: Step S501: Generate anomaly discrimination feature data based on beacon status parameter data. The anomaly discrimination feature data includes the over-limit amount of lateral deviation data, the tilt amount of beacon attitude status data, the periodic deviation amount of light quality timing feature data, and the continuity reduction amount of radar stability feature data.

[0082] The definition of lateral deviation data exceeding the limit is ,in, This indicates a preset lateral offset threshold; the tilt measurement is taken from the tilt amount in the beacon attitude state data; and the periodic deviation measurement is taken from... ( To preset the flash cycle, To preset the duty cycle, The weight for the flash period deviation is set to 0.7. The duty cycle deviation weight is set to 0.3, and the continuity decrease is set to... ( For continuity threshold, The scattering wave threshold, For target track continuity data, Reduce the weight of target track continuity. For scattered ( To preset the flash cycle, To preset the duty cycle, The weight for the flash period deviation is set to 0.7. The duty cycle deviation weight is set to 0.3, and the continuity decrease is set to... ( For continuity threshold, The scattering wave threshold, For target track continuity data, Reduce the weight of target track continuity. (Assuming a weight for scattering intensity fluctuations), thus uniformly mapping drift, tilt, lamp quality, and radar stability into discernible anomaly discrimination feature data. This anomaly discrimination feature data is then used in the next sub-step for anomaly type determination and for outputting processing priorities.

[0083] and All weights are non-negative and satisfy the following conditions: ; ; The and Calculated based on historical radar stability samples. Historical radar stability samples include historical target track continuity data, historical scattering intensity fluctuation data, and radar observation anomaly labels verified by manual review. Let the... The target track continuity data in the set of historical radar stability samples are The scattering intensity fluctuation data is The continuity threshold is The scattering wave threshold is Then the first The continuity deficiency and scattering fluctuation excess corresponding to the historical radar stability samples are as follows: ; ; Let the first The radar observation anomaly labels for the group of historical radar stability samples are: ,in = indicates that an anomaly in radar observations has been confirmed through manual verification. =0 indicates that no radar observation anomalies have been confirmed by manual verification, and the result is determined using the constrained least squares method. and : ; The constraints are: ; When the number of historical radar stability samples is less than the preset sample size threshold, the initial equalization weight is used. = =0.5. When the number of historical radar stability samples reaches the preset sample size threshold, the constrained least squares method described above is used to recalculate. and For example, in 120 sets of historical radar stability samples for a certain flight segment, if calculated as follows... =0.65, =0.35 indicates that radar observation anomalies in this segment are more easily reflected by a decrease in target track continuity. Therefore, in terms of the amount of continuity decrease... The proportion of the impact of increasing the continuity of target track data in the calculation.

[0084] Step S502: Based on the anomaly discrimination feature data and the confidence data, the abnormal navigation mark status data is judged. The abnormal navigation mark status data includes drift anomaly data, tilt anomaly data, light quality anomaly data and structural anomaly data.

[0085] Anomaly detection features and confidence level data are used to identify and classify anomalous beacon status data. Drift anomalies are triggered when the lateral deviation exceeds the drift threshold and the confidence level is not less than the confidence threshold. Tilt anomalies are triggered when the tilt exceeds the tilt threshold and the confidence level is not less than the confidence threshold. Light quality anomalies are triggered when the period deviation exceeds the light quality threshold. Structural anomalies are triggered when the outline integrity is below the integrity threshold or the reflectivity is below the reflectivity threshold and the confidence level is not less than the confidence threshold. For cases with low confidence level data but significant anomaly detection features, the system does not directly output a confirmed anomaly but instead outputs a beacon status data requiring verification to ensure that false alarms are controllable.

[0086] Step S503: Generate monitoring and handling strategy data based on abnormal navigation mark status data and credibility data. The monitoring and handling strategy data includes review collection strategy data and collection frequency adjustment strategy data.

[0087] In this embodiment, the monitoring and handling strategy data is divided into two parts: verification acquisition strategy data and acquisition frequency adjustment strategy data. If there is drift anomaly data or tilt anomaly data and the confidence level is high, the verification acquisition strategy data is output and the acquisition frequency is adjusted to a high-frequency acquisition frequency. If there is light quality anomaly data or structural anomaly data but the confidence level is medium, the verification acquisition strategy data is output and the acquisition frequency is adjusted to a medium-frequency acquisition frequency. If there is no abnormal beacon status data or only a situation that needs to be verified but the confidence level is low, the baseline acquisition frequency is maintained and the verification acquisition strategy data is set to postpone verification or wait for more observations, thereby achieving a balance between risk and cost.

[0088] The logic for generating monitoring and response strategy data is as follows: If abnormal navigation mark status data exists and the credibility data is greater than or equal to the preset credibility threshold, then the data for reviewing the collection strategy will be output and the collection frequency will be adjusted to a high-frequency collection frequency.

[0089] If abnormal navigation mark status data exists and the credibility data is less than the preset credibility threshold, then the data for reviewing the acquisition strategy will be output and the acquisition frequency will be adjusted to the intermediate frequency acquisition frequency.

[0090] If abnormal navigational beacon status data is not available, the acquisition frequency adjustment strategy data is output and the baseline acquisition frequency is maintained.

[0091] The monitoring and response strategy data is encapsulated into data packets that can be parsed by the control end and transmitted. The packets include navigation mark identification, abnormal navigation mark status data, reliability data, and data collection frequency adjustment strategy data, ensuring that the control end can complete the parsing without introducing data terms that have not appeared before.

[0092] The control unit adjusts the acquisition frequency of shore-based video data and shipborne video data according to the acquisition frequency adjustment strategy data. If it is a high-frequency acquisition frequency, the sampling interval is shortened and the time window W is expanded to improve the observability of drift trend. If it is a medium-frequency acquisition frequency, the sampling of key navigation marks is increased while ensuring bandwidth. If the baseline acquisition frequency is maintained, the default sampling strategy is maintained.

[0093] The control terminal triggers the verification acquisition process based on the verification acquisition strategy data. The results of the verification acquisition process are still transmitted back in the form of shore-based video data, shipborne video data, shipborne attitude data and navigation radar echo data and enter step S1 to form a new navigation beacon alignment observation dataset, thereby realizing a closed loop from acquisition to inference to strategy and back to acquisition. All data are used in the data processing and transmission link of this embodiment.

[0094] Example 2, refer to Figure 2 It provides a deep learning-based waterway navigation mark status monitoring system, including a data acquisition and alignment module, a benchmark construction and constraint module, a deep perception and reasoning module, a prior constraint estimation module, an anomaly detection and generation module, and a strategy transmission and control module.

[0095] The data acquisition and alignment module is used to collect shore-based video data, shipborne video data, shipborne attitude data, navigation beacon radar echo data, ship AIS track data, and hydrodynamic environment data, and to perform time synchronization and spatial alignment to form a navigation beacon alignment observation dataset.

[0096] The benchmark construction constraint module is used to construct beacon benchmark data and drift prior constraint data based on the beacon alignment observation dataset and form a drift constraint parameter set.

[0097] The depth perception inference module is used to input the navigation mark alignment observation dataset into the multimodal depth perception model and output navigation mark target data and navigation mark observation feature data.

[0098] The prior constraint estimation module is used to input the navigation mark observation feature data and the drift constraint parameter set into the prior constraint state network and output the navigation mark state parameter data.

[0099] The anomaly detection and generation module is used to detect anomalies based on navigation mark status parameter data and output abnormal navigation mark status data and monitoring and handling strategy data.

[0100] The strategy transmission and control module is used to transmit monitoring and handling strategy data to the control terminal.

[0101] This invention uses time synchronization and spatial alignment of shore-based video data, shipborne video data, shipborne attitude data, navigation beacon radar echo data, ship AIS track data, and hydrodynamic environment data to form a navigation beacon aligned observation dataset. This allows complementary information to be obtained from different modalities even in foggy, rainy, nighttime, strong reflective, and obstructed conditions, reducing missed and false alarms from single visual monitoring.

[0102] Based on the reference coordinate data of the navigation mark, navigation mark deviation data is generated, and the drift constraint parameter set calculated by hydrodynamic environment data and waterway traffic density data is introduced to perform prior consistency verification on the navigation mark drift trend data. The credibility data is generated by combining the cross-modal deviation, thereby avoiding misjudging the hull roll, camera shake or instantaneous radar loss as navigation mark drift.

[0103] The prior constraint state network outputs navigation beacon state parameter data, including lateral deviation, longitudinal deviation, attitude tilt and drift speed trends. In anomaly detection, it uniformly quantifies anomaly detection feature data such as over-limit, periodic deviation and continuity reduction, so as to achieve early warning of gradual anomalies such as slow drift, tilt development trend caused by anchor loosening, and light quality decay.

[0104] Abnormal navigation mark status data is further subdivided into drift anomaly data, tilt anomaly data, light quality anomaly data, and structural anomaly data, and monitoring and handling strategy data is generated, including review and collection strategy data and collection frequency adjustment strategy data, so that the management end can directly trigger close-range review and collection or increase the collection frequency, realizing a closed loop from monitoring to handling.

[0105] In cases of anomalies and uncertainties, reliability data is used for tiered handling. Low reliability data triggers verification first instead of direct alarm, while high reliability data increases the collection frequency and prioritizes the dispatch of inspection tasks. This balances monitoring coverage and maintenance costs in long-channel, large-scale navigation mark scenarios.

[0106] All types of collected data are incorporated into the beacon alignment observation dataset and form a clear data stream in the drift constraint parameter set, beacon observation feature data, beacon status parameter data, and monitoring and handling strategy data. This facilitates integration with existing shore-based video, radar, and AIS platforms and supports subsequent adjustment of thresholds and strategies based on segment differences.

[0107] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media containing computer-usable program code. The storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read Only Memory (EPROM), Programmable Read-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0108] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the protection scope of the present invention.

Claims

1. A method for monitoring the status of navigation aids in waterways based on deep learning, characterized in that, Includes the following steps: Step S1: Collect shore-based video data, shipborne video data, shipborne attitude data, navigation beacon radar echo data, ship AIS track data, and hydrodynamic environment data, and perform time synchronization and spatial alignment to form a navigation beacon aligned observation dataset; Step S2: Based on the beacon alignment observation dataset, construct beacon reference data and drift prior constraint data, and form a drift constraint parameter set; Step S3: Input the navigation mark alignment observation dataset into the multimodal depth perception model, and output navigation mark target data and navigation mark observation feature data; Step S4: Input the navigation mark observation feature data and drift constraint parameter set into the prior constraint state network, and output the navigation mark state parameter data; Step S5: Based on the navigation mark status parameter data, perform anomaly identification and output abnormal navigation mark status data and monitoring and handling strategy data; Step S6: Transmit the monitoring and handling strategy data to the control terminal.

2. The method for monitoring the status of navigation aids in waterways based on deep learning as described in claim 1, characterized in that, Step S1 includes the following sub-steps: Step S101: Collect shore-based video data and shipborne video data, and extract shooting timestamps and camera pose information; Step S102: Collect shipboard attitude data and align it with shipboard video data based on timestamps; Step S103: Collect radar echo data of navigation beacons and extract target range and azimuth sequence; Step S104: Collect ship AIS track data and generate waterway traffic density data; Step S105: Collect hydrodynamic environmental data, including water level data, flow velocity and direction data, and wind speed and direction data; Step S106: Synchronize and spatially align shore-based video data, shipborne video data, shipborne attitude data, navigation beacon radar echo data, ship AIS track data, and hydrodynamic environment data to output a navigation beacon aligned observation dataset.

3. The method for monitoring the status of navigation aids in waterways based on deep learning as described in claim 2, characterized in that, The spatial alignment is as follows: A mapping relationship between shore-based pixel coordinates and channel plane coordinates is established based on camera pose information from shore-based video data and channel electronic reference map data. Attitude compensation is performed on shipborne video data based on shipborne attitude data, and a mapping relationship between shipborne pixel coordinates and channel plane coordinates is established. Coordinate transformation is performed between the target range and azimuth sequence based on navigation radar echo data and the channel plane coordinates. The mapped shore-based video data, shipborne video data, and converted navigation beacon radar echo data are resampled and fused according to a unified waterway plane coordinate system to form a navigation beacon aligned observation dataset with a unified coordinate index.

4. The method for monitoring the status of navigation aids in waterways based on deep learning as described in claim 3, characterized in that, Step S2 includes the following sub-steps: Step S201: Obtain navigation mark reference coordinate data from the electronic reference chart data of the waterway and generate waterway reference grid data; Step S202: Calculate disturbance intensity data based on hydrodynamic environment data and ship AIS track data; Step S203: Establish prior drift constraint data based on disturbance intensity data and hydrodynamic environment data. The prior drift constraint data includes drift direction distribution data and upper bound drift velocity data. Step S204: Combine the drift direction distribution data with the upper bound data of drift velocity to generate a drift constraint parameter set.

5. The method for monitoring the status of navigation aids in waterways based on deep learning as described in claim 4, characterized in that, Step S3 includes the following sub-steps: Step S301: Construct visual feature tensor data based on shore-based video data and shipborne video data, and perform temporal encoding on the visual feature tensor data; Step S302: Construct radar feature sequence data based on navigation beacon radar echo data and perform time-series encoding on the radar feature sequence data; Step S303: Construct environmental traffic feature data based on ship AIS track data and hydrodynamic environment data, and embed the features. Step S304: Input the visual feature tensor data, radar feature sequence data and environmental traffic feature data into the multimodal depth perception model for cross-modal attention fusion, and output navigation target data and navigation observation feature data.

6. The method for monitoring the status of navigation aids in waterways based on deep learning as described in claim 5, characterized in that, The navigational aid observation feature data includes light quality timing feature data, shape and structure feature data, and radar stability feature data; The light quality temporal characteristic data is generated from the brightness time series of the luminous area of ​​the navigation mark in the visual feature tensor data and includes flash period data and on / off duty data. The external structural feature data is generated from the target contour sequence corresponding to the navigation target data and includes contour integrity data and reflectivity data; The radar stability characteristic data is generated from radar characteristic sequence data and includes scattering intensity fluctuation data and target track continuity data.

7. The method for monitoring the status of navigation aids in waterways based on deep learning as described in claim 6, characterized in that, Step S4 includes the following sub-steps: Step S401: Calculate the navigation mark offset data based on the navigation mark reference coordinate data and the navigation mark target data. The navigation mark offset data includes lateral offset data and lateral offset data. Step S402: Input the navigation beacon observation feature data and navigation beacon deviation data into the prior constraint state network for time-series state estimation, and output the navigation beacon attitude state data and navigation beacon drift trend data. Step S403: Perform prior consistency verification on the navigation beacon drift trend data based on the drift constraint parameter set. If the drift direction distribution data is inconsistent with the navigation beacon drift trend data or the upper limit of the drift speed data is exceeded, reduce the credibility of the navigation beacon state parameter data and mark it as a state that needs to be reviewed. Step S404: Combine the beacon offset data, beacon attitude status data, beacon drift trend data, and reliability data to output beacon status parameter data.

8. The method for monitoring the status of navigation aids in waterways based on deep learning as described in claim 7, characterized in that, Step S5 includes the following sub-steps: Step S501: Generate anomaly discrimination feature data based on beacon status parameter data. The anomaly discrimination feature data includes the over-limit amount of lateral deviation data, the tilt amount of beacon attitude status data, the period deviation amount of light quality timing feature data, and the decrease in continuity of radar stability feature data. Step S502: Based on the anomaly discrimination feature data and the confidence data, the abnormal navigation mark status data is discriminated. The abnormal navigation mark status data includes drift anomaly data, tilt anomaly data, light quality anomaly data and structural anomaly data. Step S503: Generate monitoring and handling strategy data based on abnormal navigation mark status data and credibility data. The monitoring and handling strategy data includes review collection strategy data and collection frequency adjustment strategy data.

9. The method for monitoring the status of navigation aids in waterways based on deep learning as described in claim 8, characterized in that, The logic for generating the monitoring and response strategy data is as follows: If abnormal navigation mark status data exists and the credibility data is greater than or equal to the preset credibility threshold, then the verification collection strategy data will be output and the collection frequency will be adjusted to a high-frequency collection frequency. If abnormal navigation mark status data exists and the credibility data is less than the preset credibility threshold, then the verification collection strategy data will be output and the collection frequency will be adjusted to the medium frequency collection frequency. If abnormal navigational beacon status data is not available, the acquisition frequency adjustment strategy data is output and the baseline acquisition frequency is maintained.

10. A deep learning-based waterway navigation mark status monitoring system, applied in the deep learning-based waterway navigation mark status monitoring method as described in any one of claims 1-9, characterized in that, It includes a data acquisition and alignment module, a benchmark construction and constraint module, a deep perception and reasoning module, a prior constraint estimation module, an anomaly detection and generation module, and a policy transmission and control module; The data acquisition and alignment module is used to acquire shore-based video data, shipborne video data, shipborne attitude data, navigation beacon radar echo data, ship AIS track data, and hydrodynamic environment data, and to perform time synchronization and spatial alignment to form a navigation beacon alignment observation dataset. The benchmark construction constraint module is used to construct beacon benchmark data and drift prior constraint data based on the beacon alignment observation dataset and form a drift constraint parameter set. The deep perception inference module is used to input the navigation mark alignment observation dataset into the multimodal deep perception model and output navigation mark target data and navigation mark observation feature data. The prior constraint estimation module is used to input the navigation mark observation feature data and the drift constraint parameter set into the prior constraint state network and output the navigation mark state parameter data. The anomaly detection generation module is used to detect anomalies based on navigation mark status parameter data and output abnormal navigation mark status data and monitoring and handling strategy data. The strategy transmission and control module is used to transmit monitoring and handling strategy data to the control terminal.

Images