A ship line-of-sight compensation monitoring system based on multi-source perception

Abstract

The present application relates to the technical field of ship line of sight compensation monitoring, and discloses a ship line of sight compensation monitoring system based on multi-source sensing, which comprises: an image acquisition module which acquires global images and generates panoramic angle domain images in combination with attitude data; an image shielding module which shields non-horizon edge interference by using navigation data; a curve generation module which extracts gradient extreme values in the processed images to form three types of curves, i.e., upper bright edges, dark band centers and lower bright edges; a cusp generation module which determines cusps with minimum interval distance by pitch angle slice analysis; a verification module which performs reliability verification according to curve topology and cusp stability through bearing and pitch micro-scan and back-scan operations; and a reference switching module which dynamically adjusts compensation reference according to verification results, realizes geometric alignment, freezing or backtracking, and thus ensures the stability of line of sight compensation.

Description

Technical Field

[0001] This invention relates to the field of ship line-of-sight compensation monitoring technology, and more specifically, to a ship line-of-sight compensation monitoring system based on multi-source sensing. Background Technology

[0002] Currently, ships often rely on sensors such as cameras and infrared imaging to obtain the position of the horizon line during navigation, using it as a stable reference to correct line-of-sight deviations caused by ship movement. This method can maintain image stability and the effectiveness of line-of-sight compensation well under normal atmospheric conditions because the horizon line is usually a clear and unique dividing line that can be reliably identified and utilized by the system.

[0003] However, under unique meteorological conditions at sea, especially when there is a strong temperature inversion in the sea-atmosphere boundary layer, light is significantly refracted during propagation. This causes the observed horizon to no longer be single, but may instead show two bright edges and a dark band sandwiched between them. This anomaly manifests in imaging as a horizon break or the coexistence of multiple edges. When the ship's attitude or observation altitude changes slightly, the reference line captured by the system may abruptly change, resulting in a horizon jump or a false horizon.

[0004] In this environment, traditional line-of-sight compensation monitoring still selects a single edge as a reference according to conventional logic, which can easily lead to random selection among multiple edges, making the compensation reference unstable. When the refraction conditions change slightly, the system will switch between different edges, causing abrupt changes in the monitoring results, or even misjudging it as a failure of the compensation process. Summary of the Invention

[0005] This invention provides a ship line-of-sight compensation monitoring system based on multi-source sensing, which solves the technical problems mentioned in the background art.

[0006] This invention provides a ship line-of-sight compensation monitoring system based on multi-source sensing, comprising:

[0007] The image acquisition module acquires global images and combines them with ship attitude sensing data to map the global images into panoramic angle domain images based on azimuth and pitch angles.

[0008] The image masking module masks non-horizontal edges in panoramic angle domain images based on ship navigation sensor data.

[0009] The curve generation module extracts gradient extrema along the pitch angle in the updated panoramic angle domain image after masking to obtain a curve associated with the azimuth angle; the curve includes: upper bright edge curve, dark band center curve and lower bright edge curve.

[0010] The cusp generation module analyzes multiple elevation angle slices to calculate the azimuth coverage range and interval spacing of the dark band center curve in each slice, and determines the cusp corresponding to the minimum interval spacing.

[0011] The verification module applies a preset azimuth micro-scan step size and a preset pitch micro-scan step size to the observation direction and performs a retrace operation. The verification is based on the topological relationship of the curves before and after the retrace and the stability of the cusp.

[0012] If the reference switching module passes the verification, it performs line-of-sight geometric alignment based on the direction corresponding to the cusp and updates the line-of-sight compensation reference; if the curve exists but the verification fails, it freezes the current line-of-sight compensation reference; if the curve does not exist, it reverts to the non-horizontal reference attitude.

[0013] Furthermore, by acquiring global images and combining them with ship attitude sensing data, the global images are mapped into panoramic angular domain images based on azimuth and pitch angles, including:

[0014] Panoramic images covering the target's field of view are acquired using multi-source image sensors;

[0015] Synchronously read ship attitude sensor data, which includes gimbal attitude matrix and inertial measurement unit attitude matrix;

[0016] Load the intrinsic parameter matrix and the extrinsic parameter matrix from the camera to the ship's coordinate system for each camera;

[0017] Multiply the attitude matrix of the inertial measurement unit by the attitude matrix of the gimbal, and then multiply it by the extrinsic parameter matrix from the camera to the hull to obtain the combined attitude matrix.

[0018] For each pixel in the global image, the inverse of the camera intrinsic parameter matrix is ​​used to perform an inverse transformation on the pixel coordinates to obtain the ray direction vector in the camera coordinate system. The ray direction vector is then transformed into a unit line-of-sight vector by combining the pose matrix.

[0019] The azimuth angle is calculated from the lateral and forward components of the unit line-of-sight vector using the arctangent function, and the pitch angle is calculated from the vertical component of the unit line-of-sight vector using the arcsine function.

[0020] Define discrete grids for azimuth and pitch angles, and establish a mapping relationship between pixels and angle grids. For multiple pixels falling into the same angle grid cell, perform a weighted average based on the reciprocal of the angular distance from the pixel to the center of the angle grid cell to obtain the gray value of the angle grid cell. For angle grid cells where no pixel falls, interpolate the gray values ​​of neighboring angle grid cells to generate a panoramic angle domain image.

[0021] Furthermore, based on ship navigation sensor data, non-horizontal edges in the panoramic angle domain image are masked, including:

[0022] Under the same time reference corresponding to the panoramic angle domain image, ship navigation sensor data is acquired synchronously. The ship navigation sensor data includes the automatic identification system information set, radar detection information set, and electronic chart data set.

[0023] Declare the ship's observation altitude, geographical coordinates, and heading;

[0024] For each target in the Automatic Identification System (AIS) information set, the planar coordinates relative to the ship are calculated from the target's geographic coordinates. The relative distance and azimuth of the planar coordinates relative to the ship are then calculated from the planar coordinates. The target's azimuth coverage area is determined by combining the target's length, beam, and heading. The upper and lower bounds of the pitch angle are calculated based on the ship's observation height and the target's superstructure height to obtain the target's coverage area in the angular domain.

[0025] For each echo cluster in the radar detection information set, the azimuth coverage range of the echo cluster is determined based on the measurement range, measurement azimuth, and beamwidth of the echo cluster; the upper and lower bounds of the pitch angle are calculated based on the ship's observation height and measurement range to obtain the coverage range of the echo cluster in the angular domain.

[0026] For each shoreline segment in the electronic nautical chart dataset, shoreline sampling points are obtained by sampling the shoreline at fixed intervals. The relative distance and azimuth of each shoreline sampling point relative to the ship are calculated, and the pitch angle corresponding to the shoreline sampling point is calculated based on the ship's observation height, thus obtaining the shoreline coverage set in the angular domain.

[0027] The angle domain coverage range or angle domain coverage set obtained from the ship automatic identification system information set, radar detection information set and electronic nautical chart data set are merged to form a masking film.

[0028] The occlusion mask is multiplied bitwise with the panoramic angle domain image to obtain the updated panoramic angle domain image that has been masked.

[0029] Furthermore, in the masked updated panoramic angle domain image, gradient extrema are extracted along the pitch angle to obtain a curve associated with the azimuth angle, including:

[0030] The first-order gradient in the pitch direction is calculated for the updated panoramic angle domain image according to a fixed pitch angle grid step size. The first-order gradient is calculated using the central difference method, that is, the first-order gradient at a certain pitch angle position is equal to the difference between the gray value of the adjacent previous pitch angle position and the gray value of the adjacent next pitch angle position divided by twice the pitch angle grid step size.

[0031] The square of the first gradient is used as the gradient energy, and the second derivative of the gradient energy in the pitch direction is calculated according to the same pitch angle grid step size. The second derivative is calculated using the central difference method. The stationary point of the gradient energy is extracted at each fixed azimuth angle, and the maxima candidate point and minima candidate point are determined by combining the sign of the second derivative.

[0032] Stationary points where the second derivative is less than zero are considered as maxima candidates;

[0033] Stationary points where the second derivative is greater than zero are considered as candidate minimum points;

[0034] At each fixed azimuth angle, select triplets from the maxima and minima that satisfy the order of lower bright edge less than dark band center less than upper bright edge, where the lower bright edge and upper bright edge are maxima and the dark band center is a minima.

[0035] Between adjacent azimuth angles, the sum of the absolute difference in pitch angle between the lower bright edge and the center of the dark band and the upper bright edge in the triplet is used as the cost function. The triplet that minimizes the cost function is selected, and cubic spline interpolation is performed on the obtained discrete triplet sample points along the azimuth direction to obtain the lower bright edge curve, the dark band center curve, and the upper bright edge curve.

[0036] Furthermore, by analyzing multiple elevation angle slices, the azimuth coverage range and interval of the dark band center curve in each slice are calculated, and the cusp corresponding to the minimum interval is determined, including:

[0037] A set of pitch angle slices is generated at intervals of pitch angle grid steps, and each pitch angle slice corresponds to a fixed pitch angle value. For each pitch angle slice, it is determined whether there is an azimuth angle such that the pitch angle value of the lower bright edge curve at the azimuth angle is less than the pitch angle value of the pitch angle slice, and the pitch angle value of the upper bright edge curve at the azimuth angle is greater than the pitch angle value of the pitch angle slice. Pitch angle slices that meet the conditions are marked as valid slices.

[0038] For each valid slice, let the difference function f(ψ) be:

[0039] f(ψ)=y ADZX (ψ)-y YXQP

[0040] Among them, y ADZX (ψ) represents the center curve of the dark band, y YXQP The pitch angle value represents the effective slice, and ψ represents the azimuth angle.

[0041] In a discrete azimuth grid, the sign change of the difference function at the center of two adjacent azimuth cells is detected, and one-dimensional linear interpolation is performed along the azimuth direction between the centers of adjacent azimuth cells where the sign change occurs to obtain the azimuth of all intersection points between the dark band center curve and the effective slice.

[0042] For all intersection azimuths of each valid slice, the smallest azimuth is taken as the left end of the azimuth coverage interval of the valid slice, and the largest azimuth is taken as the right end of the azimuth coverage interval of the valid slice.

[0043] The difference between the right and left ends of the azimuth coverage interval of each effective slice is calculated to obtain the interval spacing of the effective slice. The interval spacing of all effective slices is then arranged in ascending order according to the elevation angle value of their respective elevation angle slices to form an interval spacing sequence.

[0044] The pitch angle value of the pitch angle slice corresponding to the minimum value in the interval spacing sequence is selected as the cusp pitch angle;

[0045] The average of the left and right ends of the azimuth coverage interval corresponding to the pitch angle of the cusp is taken as the center azimuth angle of the cusp, and the minimum value in the interval spacing sequence is taken as the minimum interval spacing corresponding to the cusp.

[0046] Furthermore, a preset azimuth micro-scan step size and a preset elevation micro-scan step size are applied to the observation direction, and a retrace operation is performed. Verification is then conducted based on the topological relationship of the curves before and after the retrace and the stability of the cusps, including:

[0047] S61, the azimuth coverage area corresponding to the pitch angle of the cusp is used as the observation window;

[0048] S62 records the lower bright edge curve, the dark band center curve and the upper bright edge curve before performing micro-scan, as well as the apex pitch angle, center azimuth angle and minimum interval spacing.

[0049] S63, perform a positive azimuth micro-scan on the observation direction, increasing the azimuth of the observation direction by a preset azimuth micro-scan step size; after completing the positive azimuth micro-scan, perform an azimuth retrace, decreasing the azimuth of the observation direction by a preset azimuth micro-scan step size to restore it to the azimuth before the micro-scan; collect the first retrace lower bright edge curve, the first retrace dark band center curve, and the first retrace upper bright edge curve after the retrace, and calculate the corresponding first retrace tip elevation angle, the first retrace center azimuth angle, and the minimum value of the first retrace interval spacing;

[0050] S64, perform a positive pitch angle micro-scan on the observation direction, increasing the pitch angle of the observation direction by a preset pitch micro-scan step size; after completing the positive pitch angle micro-scan, perform a pitch retrace, decreasing the pitch angle of the observation direction by a preset pitch micro-scan step size to restore it to the pitch angle before the micro-scan; collect the second retrace lower bright edge curve, the second retrace dark band center curve, and the second retrace upper bright edge curve, and calculate the corresponding second retrace apex pitch angle, second retrace center azimuth angle, and minimum interval of the second retrace interval;

[0051] S65, within the observation window, verify whether the first scan lower bright edge curve, the first scan dark zone center curve and the first scan upper bright edge curve after azimuth retracement are maintained: the pitch angle value of the first scan lower bright edge curve is less than the pitch angle value of the first scan dark zone center curve, and the pitch angle value of the first scan dark zone center curve is less than the pitch angle value of the first scan upper bright edge curve.

[0052] S66. Within the observation window, verify whether the second sweep of the bright edge curve, the second sweep of the dark zone center curve, and the second sweep of the bright edge curve are maintained: the pitch angle value of the second sweep of the bright edge curve is less than the pitch angle value of the second sweep of the dark zone center curve, and the pitch angle value of the second sweep of the dark zone center curve is less than the pitch angle value of the second sweep of the bright edge curve.

[0053] S67. Compare whether the pitch angle of the first retrace point, the azimuth angle of the first retrace point and the minimum interval of the first retrace point after the azimuth retrace are exactly the same as the pitch angle of the first retrace point, the azimuth angle of the center and the minimum interval of the first retrace before the azimuth microscan is performed.

[0054] S68. Compare the pitch angle of the tip of the second retrace after pitch retrace, the azimuth angle of the center of the second retrace and the minimum interval of the second retrace after pitch retrace with the pitch angle of the tip of the tip, the azimuth angle of the center of the second retrace and the minimum interval of the interval before microscan.

[0055] S69. If the curve order in S65 and S66 remains unchanged, and the corresponding three cusp quantities in S67 and S68 are exactly the same as before the microscan, then the verification passes; otherwise, the verification fails.

[0056] Furthermore, if the verification passes, the line-of-sight geometric alignment is performed based on the direction corresponding to the cusp, and the line-of-sight compensation reference is updated, including:

[0057] The cusp direction vector is calculated based on the cusp pitch angle and the center azimuth angle. The lateral component of the cusp direction vector is equal to the product of the cosine of the cusp pitch angle and the sine of the center azimuth angle. The vertical component of the cusp direction vector is equal to the sine of the cusp pitch angle. The forward component of the cusp direction vector is equal to the product of the cosine of the cusp pitch angle and the cosine of the center azimuth angle.

[0058] Obtain the command line direction vector, calculate the dot product of the cusp direction vector and the command line direction vector, and use the inverse cosine function to obtain the angle between the cusp direction vector and the command line direction vector as the minimum rotation angle; calculate the cross product of the cusp direction vector and the command line direction vector, and normalize the cross product result as the minimum rotation axis.

[0059] The rotation matrix is ​​calculated using the Rodriguez formula based on the minimum rotation angle and minimum rotation axis. The rotation matrix is ​​then multiplied by the current line-of-sight compensation reference attitude to obtain the updated line-of-sight compensation reference attitude, which is then set as the new current line-of-sight compensation reference attitude.

[0060] Furthermore, if the curve exists but the verification fails, the current line-of-sight compensation reference is frozen, including:

[0061] Maintain the current line-of-sight compensation reference posture unchanged.

[0062] Furthermore, if the curve does not exist, the system reverts to a non-horizontal reference attitude, including:

[0063] Replace the current line-of-sight compensation reference attitude with a non-horizontal reference attitude, where the non-horizontal reference attitude is a preset stable reference attitude.

[0064] The beneficial effects of this invention include: by introducing a multi-source sensing mechanism, image acquisition, attitude measurement, navigation data, and intelligent curve analysis and verification mechanisms are organically integrated, enabling reliable extraction and stable identification of the sea-sky boundary in complex maritime environments. This system not only avoids horizon jumps caused by random selection in traditional methods under abnormal conditions such as atmospheric refraction, false horizons, or multiple coexisting edges, but also ensures the continuity and robustness of the line-of-sight compensation reference through cusp generation and retrace verification. This significantly improves the stability of ship monitoring images and the reliability of line-of-sight geometric alignment, reduces misjudgments and abrupt changes, and ensures that ships can obtain stable and reliable line-of-sight compensation references even under complex weather and optical conditions. Attached Figure Description

[0065] Figure 1 This is a block diagram of a ship line-of-sight compensation monitoring system based on multi-source sensing according to the present invention. Detailed Implementation

[0066] The subject matter described herein will now be discussed with reference to exemplary embodiments. It should be understood that these embodiments are discussed only to enable those skilled in the art to better understand and implement the subject matter described herein, and changes may be made to the function and arrangement of the elements discussed without departing from the scope of this specification. Various processes or components may be omitted, substituted, or added as needed in the examples. Furthermore, features described in some examples may be combined in other examples.

[0067] like Figure 1 As shown, a ship line-of-sight compensation monitoring system based on multi-source sensing includes:

[0068] The image acquisition module acquires global images and combines them with ship attitude sensing data to map the global images into panoramic angle domain images based on azimuth and pitch angles.

[0069] The image masking module masks non-horizontal edges in panoramic angle domain images based on ship navigation sensor data.

[0070] The curve generation module extracts gradient extrema along the pitch angle in the updated panoramic angle domain image after masking to obtain a curve associated with the azimuth angle; the curve includes: upper bright edge curve, dark band center curve and lower bright edge curve.

[0071] The cusp generation module analyzes multiple elevation angle slices to calculate the azimuth coverage range and interval spacing of the dark band center curve in each slice, and determines the cusp corresponding to the minimum interval spacing.

[0072] The verification module applies a preset azimuth micro-scan step size and a preset pitch micro-scan step size to the observation direction and performs a retrace operation. The verification is based on the topological relationship of the curves before and after the retrace and the stability of the cusp.

[0073] If the reference switching module passes the verification, it performs line-of-sight geometric alignment based on the direction corresponding to the cusp and updates the line-of-sight compensation reference; if the curve exists but the verification fails, it freezes the current line-of-sight compensation reference; if the curve does not exist, it reverts to the non-horizontal reference attitude.

[0074] It should be noted that the system modules are called in a fixed order, and the data flow is transmitted sequentially. First, the image acquisition module is activated. This module receives the global image and ship attitude sensor data, processes it, and generates a panoramic angle domain image, which is output to the image masking module as part of the data flow. The image masking module receives the panoramic angle domain image and ship navigation sensor data, performs masking processing to generate an updated panoramic angle domain image, and outputs it to the curve generation module as part of the data flow. The curve generation module receives the updated panoramic angle domain image, extracts the upper bright edge curve, the center curve of the dark band, and the lower bright edge curve, and outputs these curve parameters to the cusp generation module as part of the data flow. The cusp generation module calculates the relevant parameters of the cusp based on the received curve parameters and outputs them to the verification module as part of the data flow. The verification module receives the cusp parameters and the current observation direction, performs a micro-scan and retrace operation, generates a verification result, and outputs the result to the reference switching module as part of the data flow. The reference switching module performs an update, freeze, or rollback operation for the line-of-sight compensation reference based on the verification result, completing the closed loop of the entire process.

[0075] It should be noted that the input to the image acquisition module is a global image and ship attitude sensing data. The global image is a series of frame images acquired by multi-source image sensors, and the ship attitude sensing data includes the attitude information of the gimbal and inertial measurement unit. The output is a panoramic angle domain image, which is in the form of a two-dimensional matrix. The rows of the matrix correspond to the azimuth dimension, the columns correspond to the pitch dimension, and each element represents the gray value of the corresponding angle position.

[0076] The input to the image masking module is a panoramic angle domain image and ship navigation sensor data. The ship navigation sensor data includes information from the Automatic Identification System (AIS), radar detection information, and electronic chart data. The output is an updated panoramic angle domain image, which is also in the form of a two-dimensional matrix, with the same dimensions as the input panoramic angle domain image. However, the grayscale values ​​corresponding to non-horizontal edges are masked.

[0077] The curve generation module takes the updated panoramic angle domain image as input and outputs the parameters of three curves. Each curve is presented as an array, with the array index corresponding to the azimuth angle and the array element corresponding to the pitch angle value at that azimuth angle. These correspond to the upper bright edge curve, the dark band center curve, and the lower bright edge curve, respectively.

[0078] The cusp generation module takes as input the parameters of three curves and outputs as cusp parameters, including the cusp pitch angle, cusp center azimuth angle, and minimum interval spacing, all in numerical form.

[0079] The input to the verification module is the cusp parameter and the current observation direction information, which includes the azimuth and elevation angles. The output is the verification result, which is a Boolean value, with true for passing and false for failing.

[0080] The input to the reference switching module is the direction information corresponding to the cusp of the verification result and the current line-of-sight compensation reference; the output is the updated line-of-sight compensation reference or the original reference is maintained. The data is in the form of an attitude matrix, which is used to represent the spatial orientation of the line of sight.

[0081] It should be noted that the non-horizontal reference attitude is the stable attitude reference that the system relies on when it cannot utilize horizontal features. Its sources include three types: first, real-time attitude data provided by the inertial measurement unit (IMU), which is calculated using an inertial navigation algorithm and has short-term stability; second, the output of the platform stabilization loop, which maintains its own stability through mechanical or electronic means, providing a relatively stable attitude reference; and third, the initial calibration attitude, which is the reference attitude determined manually or automatically during system startup, serving as a reference for the system's initial state.

[0082] The data form of non-horizontal reference attitude is either a direction cosine matrix or a quaternion. The direction cosine matrix is ​​a 3x3 matrix that describes the angular relationship between the three coordinate axes through its matrix elements and can be directly used for coordinate transformation; the quaternion consists of four elements and represents rotation in three-dimensional space in a compact form, which is convenient for calculation and storage.

[0083] It should be noted that a three-state machine includes three states: lock-and-hold and rollback.

[0084] The locked state is entered when the verification module outputs a successful verification. In this state, the system updates the line-of-sight compensation reference based on the direction corresponding to the cusp. The locked state is maintained as long as each verification passes; as long as verification continues to pass, the system continues to update the reference.

[0085] The entry condition for the "hold" state is that the curve generation module can extract three curves, but the verification module outputs that the verification failed. In this state, the system keeps the current line-of-sight compensation reference unchanged and does not update it. The "hold" condition is that the curves continue to exist and the verification always fails. If the curves disappear, the system exits this state; if the verification passes, the system switches to the "locked" state.

[0086] The rollback state is entered when the curve generation module fails to extract three curves. In this state, the system switches the line-of-sight compensation reference to a non-horizontal reference attitude. The hold state is entered when the curves remain nonexistent. If the curves reappear and verification passes, the system switches to the locked state; if the curves reappear but verification fails, the system switches to the hold state.

[0087] In one embodiment of the present invention, acquiring a global image and combining it with ship attitude sensing data, mapping the global image into a panoramic angle domain image based on azimuth and pitch angles, includes:

[0088] Panoramic images covering the target's field of view are acquired using multi-source image sensors;

[0089] Synchronously read ship attitude sensor data, which includes gimbal attitude matrix and inertial measurement unit attitude matrix;

[0090] Load the intrinsic parameter matrix and the extrinsic parameter matrix from the camera to the ship's coordinate system for each camera;

[0091] Multiply the attitude matrix of the inertial measurement unit by the attitude matrix of the gimbal, and then multiply it by the extrinsic parameter matrix from the camera to the hull to obtain the combined attitude matrix.

[0092] For each pixel in the global image, the inverse of the camera intrinsic parameter matrix is ​​used to perform an inverse transformation on the pixel coordinates to obtain the ray direction vector in the camera coordinate system. The ray direction vector is then transformed into a unit line-of-sight vector by combining the pose matrix.

[0093] The azimuth angle is calculated from the lateral and forward components of the unit line-of-sight vector using the arctangent function, and the pitch angle is calculated from the vertical component of the unit line-of-sight vector using the arcsine function.

[0094] Define discrete grids for azimuth and pitch angles, and establish a mapping relationship between pixels and angle grids. For multiple pixels falling into the same angle grid cell, perform a weighted average based on the reciprocal of the angular distance from the pixel to the center of the angle grid cell to obtain the gray value of the angle grid cell. For angle grid cells where no pixel falls, interpolate the gray values ​​of neighboring angle grid cells to generate a panoramic angle domain image.

[0095] In detail, the camera intrinsic parameter matrix is ​​obtained through offline calibration. The specific process involves preparing a calibration board with a checkerboard of known dimensions, placing the calibration board at different positions and angles within the camera's field of view, and acquiring multiple sets of images containing the calibration board. Image processing algorithms are used to identify the corner points of the checkerboard in the images. Combined with the actual size of the calibration board, the camera's intrinsic parameter matrix, including parameters such as focal length and principal point coordinates, is calculated using the principle of perspective transformation. After calibration, the intrinsic parameter matrix is ​​stored in the system for use by the image acquisition module.

[0096] The extrinsic parameter matrix from the camera to the ship's coordinate system is obtained through on-site calibration. The process involves setting marker points with known spatial coordinates on the ship's hull, ensuring these marker points appear simultaneously within the camera's field of view and the measurement range of the ship's coordinate system. Images containing these marker points are acquired, and the pixel coordinates of these marker points are identified. Combined with their known coordinates in the ship's coordinate system, the extrinsic parameter matrix is ​​calculated using a coordinate transformation algorithm. This matrix describes the rotational and translational relationship between the camera and ship's coordinate systems. The extrinsic parameter matrix is ​​stored after calibration during the system installation and commissioning phase. If the camera position changes, recalibration is required.

[0097] The calculation of the combined attitude matrix is ​​based on the intrinsic and extrinsic parameter matrices obtained from the above calibration, as well as the gimbal attitude matrix and inertial measurement unit attitude matrix acquired in real time. They are combined sequentially through matrix multiplication to ensure the continuity and accuracy of attitude transition.

[0098] In detail, the unit line-of-sight vector refers to the normalized vector in three-dimensional space, which is the direction vector from the camera's optical center to the target point. Normalization is achieved by calculating the magnitude of the vector, which is the square root of the sum of the squares of its components. Each component is then divided by the magnitude, resulting in a normalized vector with a magnitude of 1.

[0099] When calculating azimuth and pitch angles using trigonometric functions, the input angle unit is radians. This is because the system's internal coordinate transformations and attitude calculations both use radians as the angle unit to ensure unit consistency during the calculation process and avoid errors caused by unit conversion.

[0100] In detail, the azimuth angle ranges from 0 to 360 degrees in radians, covering the entire circumference on the horizontal plane. Using the ship's bow as the reference direction, rotating clockwise one full revolution, the azimuth angle gradually increases from 0 to 360 degrees in radians, ensuring that all directions in the horizontal plane can be uniquely represented.

[0101] The pitch angle ranges from -90 degrees to the radians corresponding to +90 degrees. When the line of sight is horizontal and forward, the pitch angle is 0; when the line of sight is tilted upward, the pitch angle is positive, with a maximum of 90 degrees in radians, pointing towards the zenith; when the line of sight is tilted downward, the pitch angle is negative, with a minimum of -90 degrees in radians, pointing towards the nadir, covering all possible line of sight directions in the vertical direction.

[0102] In detail, when multiple pixels fall into the same angled grid cell, the angular distance from the pixel to the center of the angled grid cell is calculated. To avoid division by zero caused by zero angular distance, the system sets a minimum angular distance threshold. When the angular distance from a pixel to the center is less than this threshold, it is considered equal to the threshold. During weighted averaging, the weight of each pixel is the ratio of this threshold to the angular distance (when the angular distance is greater than or equal to the threshold) or 1 (when the angular distance is less than the threshold). This method ensures that division by zero errors do not occur during weight calculation, while also guaranteeing the rationality of weight allocation, making the grayscale value calculation of the angled grid cell stable and reliable.

[0103] In detail, for angled grid cells where no pixels fall, the system uses bilinear interpolation to obtain the grayscale value. Specifically, it finds the four nearest non-blank grid cells around the blank grid cell and obtains their grayscale values ​​and relative positions.

[0104] Calculate the weights of the blank grid cells based on their distances to their four neighboring grid cells, with closer cells having higher weights. Multiply the gray values ​​of the neighboring grid cells by their respective weights and sum them to obtain the gray value of the blank grid cell.

[0105] For blank grid cells at the boundary, if some neighboring grid cells are outside the angular domain, only the existing neighboring grid cells are used for interpolation to ensure that the gray values ​​at the boundary are calculated reasonably and to avoid distortion of the interpolation results due to boundary issues.

[0106] In one embodiment of the present invention, the non-horizontal edges in a panoramic angle domain image are masked based on ship navigation sensor data, including:

[0107] Under the same time reference corresponding to the panoramic angle domain image, ship navigation sensor data is acquired synchronously. The ship navigation sensor data includes the automatic identification system information set, radar detection information set, and electronic chart data set.

[0108] Declare the ship's observation altitude, geographical coordinates, and heading;

[0109] For each target in the Automatic Identification System (AIS) information set, the planar coordinates relative to the ship are calculated from the target's geographic coordinates. The relative distance and azimuth of the planar coordinates relative to the ship are then calculated from the planar coordinates. The target's azimuth coverage area is determined by combining the target's length, beam, and heading. The upper and lower bounds of the pitch angle are calculated based on the ship's observation height and the target's superstructure height to obtain the target's coverage area in the angular domain.

[0110] For each echo cluster in the radar detection information set, the azimuth coverage range of the echo cluster is determined based on the measurement range, measurement azimuth, and beamwidth of the echo cluster; the upper and lower bounds of the pitch angle are calculated based on the ship's observation height and measurement range to obtain the coverage range of the echo cluster in the angular domain.

[0111] For each shoreline segment in the electronic nautical chart dataset, shoreline sampling points are obtained by sampling the shoreline at fixed intervals. The relative distance and azimuth of each shoreline sampling point relative to the ship are calculated, and the pitch angle corresponding to the shoreline sampling point is calculated based on the ship's observation height, thus obtaining the shoreline coverage set in the angular domain.

[0112] The angle domain coverage range or angle domain coverage set obtained from the ship automatic identification system information set, radar detection information set and electronic nautical chart data set are merged to form a masking film.

[0113] The occlusion mask is multiplied bitwise with the panoramic angle domain image to obtain the updated panoramic angle domain image that has been masked.

[0114] In detail, the Automatic Identification System (AIS) data set contains specific data items for each target. Target coordinates are latitude and longitude, in degrees, accurate to six decimal places, used to determine the target's position on the Earth's surface. Length refers to the straight-line distance between the bow and stern of the target vessel, in meters, accurate to an integer. Beam refers to the maximum transverse width of the target vessel, in meters, accurate to an integer. Heading refers to the direction the target vessel's bow is pointing, in degrees, ranging from 0 to 360, with 0 degrees corresponding to true north, increasing clockwise, accurate to one decimal place. This data is acquired in real time via broadcast signals from the AIS and updated at fixed intervals.

[0115] In detail, the echo cluster in radar detection information refers to a set of spatially close signals among the reflected signals received by the radar, considered as reflections of the same target or area. The measured distance is the straight-line distance from the ship's radar antenna to the center of the echo cluster, measured in meters and accurate to an integer, calculated using the radar signal propagation time. The measured azimuth is the angle of the echo cluster center relative to the ship's bow, measured in degrees, ranging from 0 to 360 degrees, with 0 degrees corresponding to the ship's bow, increasing clockwise, accurate to one decimal place, obtained by measuring the azimuth angle of the radar antenna. The beamwidth is the angular range of the electromagnetic beam emitted by the radar antenna in the horizontal direction, measured in degrees, determined by radar hardware parameters, typically between 1 and 5 degrees, used to determine the coverage area of ​​the echo cluster in the azimuth angle.

[0116] In detail, the coastlines in the electronic nautical chart dataset are stored as polylines, described by continuous feature point coordinates (latitude and longitude). Sampling is performed at fixed intervals of 10 meters, meaning that a sampling point is taken every 10 meters along the coastline, starting from the initial point. The coordinates of the sampling points are latitude and longitude, consistent with the feature point coordinate format of the electronic nautical chart.

[0117] To calculate the relative distance of each sampling point to the ship, the ship's geographical coordinates and the latitude and longitude of the sampling point are first converted to Cartesian coordinates. A Gauss-Krüger projection is used to establish a local coordinate system with the ship's position as the origin. The straight-line distance is then calculated using the formula for the distance between two points, expressed in meters. The relative azimuth is the angle from the ship's bow clockwise to the sampling point, calculated using the coordinate difference in the Cartesian coordinate system, expressed in degrees.

[0118] When projected into the angular domain, the pitch angle is calculated based on the relative distance between the sampling points and the ship's observation height. Specifically, through geometric relationships, the spatial position of the sampling point is converted into a pitch angle relative to the ship's observation point, thereby determining the pitch angle position of the sampling point in the angular domain and forming a set of shoreline coverage in the angular domain.

[0119] In detail, for targets in an Automatic Identification System (AIS), the calculation of the upper and lower bounds of the pitch angle is based on the ship's observation height, the height of the target's superstructure, and the relative distance. The height of the target's superstructure is the vertical distance from the highest point above the target's deck to the sea surface, expressed in meters. During the calculation, through geometric relationships, combined with the difference between the ship's observation height and the height of the target's superstructure, as well as the relative distance to the target, the angles of the line of sight from the ship's observation point to the top and bottom of the target are determined, serving as the upper and lower bounds of the pitch angle, respectively.

[0120] For radar echo clusters, the upper and lower bounds of the elevation angle are calculated based on the ship's observation height and measurement distance. Using geometric relationships, the ship's observation height is considered the vertical distance, and the measurement distance is considered the horizontal distance. The angles from the ship's observation point to the top and bottom of the echo cluster are calculated, with the top corresponding to the upper bound and the bottom corresponding to the lower bound.

[0121] For shoreline sampling points on electronic nautical charts, the pitch angle is calculated based on the ship's observation altitude and the relative distance to the sampling point. Using geometric relationships, the difference between the ship's observation altitude and the vertical distance from the sampling point to the sea surface (considered 0 since the shoreline is at the sea surface) is combined with the relative distance to calculate the angle of the line of sight from the ship's observation point to the sampling point; this angle is the pitch angle corresponding to that sampling point.

[0122] In detail, when merging the angular domain coverage intervals of Automatic Identification System (AIS) radar and electronic chart data, a union method is used. For azimuth coverage intervals, if intervals from different data sources overlap, they are merged into a single continuous interval, with the starting value being the smallest among all intervals and the ending value being the largest among all intervals. If intervals do not overlap but are adjacent, they are also merged into a single continuous interval.

[0123] For pitch angle coverage areas, the processing method is the same as for azimuth angle; overlapping or adjacent areas are merged into a single continuous area.

[0124] When the coverage areas of different data sources conflict, meaning the same angular position is encompassed by the coverage areas of multiple data sources, they are not distinguished and are merged into a single coverage area. After merging, the resulting occlusion mask contains all angular domain positions corresponding to non-horizontal edges. These positions in the mask have a value of 0, while the remaining positions have a value of 1. After bitwise multiplying the mask with the panoramic angular domain image, the grayscale values ​​corresponding to non-horizontal edges are set to 0, achieving the masking effect.

[0125] In one embodiment of the present invention, in the masked updated panoramic angle domain image, the gradient extremum is extracted along the pitch angle to obtain a curve associated with the azimuth angle, including:

[0126] The first-order gradient in the pitch direction is calculated for the updated panoramic angle domain image according to a fixed pitch angle grid step size. The first-order gradient is calculated using the central difference method, that is, the first-order gradient at a certain pitch angle position is equal to the difference between the gray value of the adjacent previous pitch angle position and the gray value of the adjacent next pitch angle position divided by twice the pitch angle grid step size.

[0127] The square of the first gradient is used as the gradient energy, and the second derivative of the gradient energy in the pitch direction is calculated according to the same pitch angle grid step size. The second derivative is calculated using the central difference method. The stationary point of the gradient energy is extracted at each fixed azimuth angle, and the maxima candidate point and minima candidate point are determined by combining the sign of the second derivative.

[0128] Stationary points where the second derivative is less than zero are considered as maxima candidates;

[0129] Stationary points where the second derivative is greater than zero are considered as candidate minimum points;

[0130] At each fixed azimuth angle, select triplets from the maxima and minima that satisfy the order of lower bright edge less than dark band center less than upper bright edge, where the lower bright edge and upper bright edge are maxima and the dark band center is a minima.

[0131] Between adjacent azimuth angles, the sum of the absolute difference in pitch angle between the lower bright edge and the center of the dark band and the upper bright edge in the triplet is used as the cost function. The triplet that minimizes the cost function is selected, and cubic spline interpolation is performed on the obtained discrete triplet sample points along the azimuth direction to obtain the lower bright edge curve, the dark band center curve, and the upper bright edge curve.

[0132] In detail, when using the central difference method for gradient calculation, the central difference cannot be directly applied to the boundary positions of the pitch angle dimension in the panoramic angle domain image, i.e., the first and last pitch angle positions (requiring gray values ​​from adjacent vertical positions). In this case, for the first pitch angle position, forward difference is used to calculate the gradient; that is, the gradient at this position is equal to the difference between the gray values ​​at the second and first pitch angle positions, divided by the pitch angle grid step size. For the last pitch angle position, backward difference is used to calculate the gradient; that is, the gradient at this position is equal to the difference between the gray values ​​at the last and second-to-last pitch angle positions, divided by the pitch angle grid step size. This method ensures that the gradients at all pitch angle positions can be effectively calculated, maintaining the continuity of the gradient at the boundaries.

[0133] In detail, the gradient energy is the square of the first-order gradient. To avoid numerical overflow, the system sets an upper limit on the calculated gradient value. When the gradient value exceeds the upper limit, it is truncated to the upper limit value, and then the square is calculated as the gradient energy to ensure that the value is within the valid range.

[0134] To mitigate noise interference, the updated panoramic angle domain image is smoothed before gradient calculation. A Gaussian smoothing filter is used, with the filter window size set according to the image noise level, typically a 3x3 or 5x5 window. This filtering reduces high-frequency noise, resulting in smoother grayscale value changes, thereby reducing fluctuations in gradient and second-order derivative calculations and improving stability.

[0135] The second derivative is also calculated using the central difference method. To further enhance stability, a threshold is applied to the calculation results. When the absolute value of the second derivative is less than a set threshold, it is considered zero to prevent small fluctuations from being misjudged as extreme point features, thus ensuring the reliability of extreme point selection.

[0136] In detail, the selection of maxima and minima is strictly based on the sign of the second derivative. For stationary points of gradient energy (points with zero first-order gradients), a point is identified as a maxima candidate when its second derivative is less than zero, and as a minima candidate when its second derivative is greater than zero.

[0137] If the second derivative of a stationary point is zero, then that point is not classified as a candidate for maximum or minimum and is considered a non-extremum point. This approach clearly distinguishes between extrema and non-extrema points, avoiding ambiguity that could affect the accuracy of subsequent curve extraction.

[0138] In detail, when selecting triplets from maximal and minimum candidate points at each fixed azimuth angle, the process first checks whether there are at least two maximal candidate points and one minimum candidate point. If so, the triplet is sorted by elevation angle value, and the combination that satisfies the condition that the lower bright edge (maximal candidate point at a smaller elevation angle) is less than the center of the dark band (minimum candidate point) and the center of the dark band is less than the upper bright edge (maximum candidate point at a larger elevation angle) is selected.

[0139] If the number of candidate points is insufficient, i.e., lacking maximal or minima, no triplet will be generated for that azimuth angle. In this case, triplet information from adjacent azimuth angles is used for interpolation to supplement the curve and ensure its continuity at that azimuth angle. During interpolation, the triplet parameters of the left and right adjacent azimuth angles are taken, and the estimated value of the triplet for the current azimuth angle is calculated according to distance weights to avoid curve breaks due to missing local candidate points.

[0140] In detail, when performing cubic spline interpolation along the azimuth direction for discrete triplet sample points, a natural boundary condition is adopted. That is, the second derivative of the interpolation curve is zero at the starting and ending points of the azimuth angle. This boundary condition allows the curve to extend naturally at both ends, avoiding unnecessary sharp bends at the boundaries and ensuring the smoothness and continuity of the curve throughout the entire azimuth angle range. Simultaneously, the natural boundary condition requires no additional boundary constraint information; the interpolation curve can be determined solely from the sample points themselves, simplifying the calculation process while ensuring that the curve shape conforms to the actual characteristic distribution.

[0141] In one embodiment of the present invention, by analyzing multiple elevation angle slices, calculating the azimuth coverage range and interval spacing of the dark band center curve in each slice, and determining the cusp corresponding to the minimum interval spacing, the method includes:

[0142] A set of pitch angle slices is generated at intervals of pitch angle grid steps, and each pitch angle slice corresponds to a fixed pitch angle value. For each pitch angle slice, it is determined whether there is an azimuth angle such that the pitch angle value of the lower bright edge curve at the azimuth angle is less than the pitch angle value of the pitch angle slice, and the pitch angle value of the upper bright edge curve at the azimuth angle is greater than the pitch angle value of the pitch angle slice. Pitch angle slices that meet the conditions are marked as valid slices.

[0143] For each valid slice, let the difference function f(ψ) be:

[0144] f(ψ)=y ADZX (ψ)-y YXQP

[0145] Among them, y ADZX (ψ) represents the center curve of the dark band, y YXQP The pitch angle value represents the effective slice, and ψ represents the azimuth angle.

[0146] In a discrete azimuth grid, the sign change of the difference function at the center of two adjacent azimuth cells is detected, and one-dimensional linear interpolation is performed along the azimuth direction between the centers of adjacent azimuth cells where the sign change occurs to obtain the azimuth of all intersection points between the dark band center curve and the effective slice.

[0147] For all intersection azimuths of each valid slice, the smallest azimuth is taken as the left end of the azimuth coverage interval of the valid slice, and the largest azimuth is taken as the right end of the azimuth coverage interval of the valid slice.

[0148] The difference between the right and left ends of the azimuth coverage interval of each effective slice is calculated to obtain the interval spacing of the effective slice. The interval spacing of all effective slices is then arranged in ascending order according to the elevation angle value of their respective elevation angle slices to form an interval spacing sequence.

[0149] The pitch angle value of the pitch angle slice corresponding to the minimum value in the interval spacing sequence is selected as the cusp pitch angle;

[0150] The average of the left and right ends of the azimuth coverage interval corresponding to the pitch angle of the cusp is taken as the center azimuth angle of the cusp, and the minimum value in the interval spacing sequence is taken as the minimum interval spacing corresponding to the cusp.

[0151] In detail, the difference function is the pitch angle value of the dark band center curve at a certain azimuth angle minus the pitch angle value of the effective slice. Sign change detection refers to comparing the difference function results at the centers of two adjacent cells in the discrete azimuth grid. If one is positive and the other negative, a sign change is determined. This change indicates that the dark band center curve crosses the pitch angle value of the effective slice between these two adjacent azimuth cells, i.e., an intersection point exists.

[0152] In detail, adjacent azimuth angles refer to the azimuth angles corresponding to the centers of two adjacent cells in a discrete azimuth grid. The discrete azimuth grid is divided with a fixed step size, and each cell has a center azimuth angle. The difference between the center azimuth angles of adjacent cells is equal to the grid step size, and they are arranged in an adjacent relationship in the azimuth dimension.

[0153] In detail, when calculating the intersection point using linear interpolation, the difference function values ​​at the centers of two adjacent azimuth units and their corresponding azimuth angles are first obtained. Let the two azimuth angles be the left azimuth angle and the right azimuth angle, and the corresponding difference function values ​​be the left value and the right value, respectively. Based on the coordinates of these two points, the azimuth angle at which the difference function is zero is determined through linear fitting; this is the azimuth angle of the intersection point.

[0154] Regarding endpoint processing, if the sign change occurs at the beginning or end of the azimuth grid, i.e., when there are only azimuth elements on one side, interpolation is performed using only the existing elements and the virtual elements at the endpoints. The difference function value of the virtual elements is extrapolated according to the edge trend to ensure that the intersection calculation does not exceed the effective azimuth range.

[0155] In detail, when multiple disconnected intersection points exist, an outer-bound interval strategy is adopted. That is, disregarding discontinuities between intersection points, the smallest azimuth among all intersection points is taken as the left end of the effective slice azimuth coverage interval, and the largest azimuth among all intersection points is taken as the right end. This method can completely cover the azimuth range of the center curve of the dark band and all intersection points of the slice, avoiding incomplete coverage due to local discontinuities.

[0156] In detail, when multiple slices have the same interval spacing, the priority strategy is to select the slice whose elevation angle value is closest to the average elevation angle of the center curve of the dark band. If there are still multiple slices that meet the criteria, then select the slice whose center azimuth angle of the azimuth coverage interval is closest to the average center azimuth angle of all slices, thus ensuring the uniqueness and rationality of the cusp.

[0157] In detail, when slices have no intersection points, they are skipped directly, not marked as valid slices, and not included in the calculation of interval spacing. Such slices will not affect the generation of subsequent cusps, ensuring that only slices with valid intersection points enter the subsequent processing flow, avoiding invalid data from interfering with the results.

[0158] In one embodiment of the present invention, a preset azimuth micro-scan step size and a preset pitch micro-scan step size are applied to the observation direction and a retrace operation is performed. Verification is then performed based on the topological relationship of the curves before and after the retrace and the stability of the cusps, including:

[0159] S61, the azimuth coverage area corresponding to the pitch angle of the cusp is used as the observation window;

[0160] S62 records the lower bright edge curve, the dark band center curve and the upper bright edge curve before performing micro-scan, as well as the apex pitch angle, center azimuth angle and minimum interval spacing.

[0161] S63, perform a positive azimuth micro-scan on the observation direction, increasing the azimuth of the observation direction by a preset azimuth micro-scan step size; after completing the positive azimuth micro-scan, perform an azimuth retrace, decreasing the azimuth of the observation direction by a preset azimuth micro-scan step size to restore it to the azimuth before the micro-scan; collect the first retrace lower bright edge curve, the first retrace dark band center curve, and the first retrace upper bright edge curve after the retrace, and calculate the corresponding first retrace tip elevation angle, the first retrace center azimuth angle, and the minimum value of the first retrace interval spacing;

[0162] S64, perform a positive pitch angle micro-scan on the observation direction, increasing the pitch angle of the observation direction by a preset pitch micro-scan step size; after completing the positive pitch angle micro-scan, perform a pitch retrace, decreasing the pitch angle of the observation direction by a preset pitch micro-scan step size to restore it to the pitch angle before the micro-scan; collect the second retrace lower bright edge curve, the second retrace dark band center curve, and the second retrace upper bright edge curve, and calculate the corresponding second retrace apex pitch angle, second retrace center azimuth angle, and minimum interval of the second retrace interval;

[0163] S65, within the observation window, verify whether the first scan lower bright edge curve, the first scan dark zone center curve and the first scan upper bright edge curve after azimuth retracement are maintained: the pitch angle value of the first scan lower bright edge curve is less than the pitch angle value of the first scan dark zone center curve, and the pitch angle value of the first scan dark zone center curve is less than the pitch angle value of the first scan upper bright edge curve.

[0164] S66. Within the observation window, verify whether the second sweep of the bright edge curve, the second sweep of the dark zone center curve, and the second sweep of the bright edge curve are maintained: the pitch angle value of the second sweep of the bright edge curve is less than the pitch angle value of the second sweep of the dark zone center curve, and the pitch angle value of the second sweep of the dark zone center curve is less than the pitch angle value of the second sweep of the bright edge curve.

[0165] S67. Compare whether the pitch angle of the first retrace point, the azimuth angle of the first retrace point and the minimum interval of the first retrace point after the azimuth retrace are exactly the same as the pitch angle of the first retrace point, the azimuth angle of the center and the minimum interval of the first retrace before the azimuth microscan is performed.

[0166] S68. Compare the pitch angle of the tip of the second retrace after pitch retrace, the azimuth angle of the center of the second retrace and the minimum interval of the second retrace after pitch retrace with the pitch angle of the tip of the tip, the azimuth angle of the center of the second retrace and the minimum interval of the interval before microscan.

[0167] S69. If the curve order in S65 and S66 remains unchanged, and the corresponding three cusp quantities in S67 and S68 are exactly the same as before the microscan, then the verification passes; otherwise, the verification fails.

[0168] In detail, the settings for the preset azimuth and pitch micro-scan step sizes are related to the angle domain grid resolution and the minimum coding step size of the actuator. The angle domain grid resolution is a fixed interval for dividing the azimuth and pitch angles. The preset micro-scan step size must not be less than this resolution to ensure that the micro-scanning action can be accurately identified in the angle domain. The minimum coding step size of the actuator is the smallest angle adjustment it can achieve. The preset micro-scan step size must be an integer multiple of this minimum coding step size to ensure that the actuator can accurately perform the micro-scanning action and avoid action errors caused by step size mismatch.

[0169] In detail, after the retracement, the acquisition of the bright edge curve in the first retracement, the center curve of the dark band in the first retracement, and the bright edge curve in the first retracement requires re-executing the curve generation method described above. That is, for the image acquired after the retracement, the gradient is calculated according to a fixed pitch angle grid step size, extreme points are extracted, triplet pairs are selected, and cubic spline interpolation is performed to generate three curves.

[0170] To obtain the minimum values ​​of the first retracement cusp elevation angle, the first retracement center azimuth angle, and the first retracement interval spacing, the cusp generation method described above needs to be re-executed. Specifically, for the three curves generated after the retracement, elevation angle slices are generated, the azimuth coverage interval and spacing of the effective slices are calculated, and the cusp parameters are determined. The acquisition methods for the relevant curves and cusp quantities in the second retracement are the same.

[0171] In detail, when verifying the order relationship of curves within the observation window, it is necessary to check each azimuth angle. The observation window is the azimuth coverage interval corresponding to the pitch angle of the cusp, encompassing all discrete azimuth angle grid cells within this interval. For each azimuth angle cell, the pitch angle values ​​of the lower bright edge curve, the center curve of the dark band, and the upper bright edge curve in the first sweep are extracted at that azimuth angle. It is then determined whether the pitch angle value of the lower bright edge curve is less than the pitch angle value of the center curve of the dark band, and the pitch angle value of the center curve of the dark band is less than the pitch angle value of the upper bright edge curve. The same logic is used to verify the order relationship after the pitch sweep, checking the order of the pitch angle values ​​of the three curves in the second sweep angle by azimuth angle.

[0172] In detail, "completely identical" means that the cusp parameters obtained after the retracement are numerically identical to those before the microscan, within the same discrete grid. The discrete grid consists of azimuth and elevation angle grids defined by the system, with each grid cell having a fixed precision for its parameter values. Specifically, the cusp elevation angle of the first retracement is numerically identical to the cusp elevation angle before the microscan within the same elevation angle grid cell; the center azimuth angle of the first retracement is numerically identical to the center azimuth angle before the microscan within the same azimuth angle grid cell; and the minimum interval spacing of the first retracement is numerically identical to the minimum interval spacing before the microscan. The comparison standard for the cusp parameters after the elevation retracement is the same as before the microscan.

[0173] In detail, when the curve is broken, that is, when the curve generated after retracement shows discontinuous or missing parts of the azimuth angle corresponding to the elevation angle value in the observation window, the verification is deemed to have failed.

[0174] When the cusp disappears, that is, when the cusp generation method cannot obtain valid cusp parameters after retracement, the verification is deemed to have failed.

[0175] If the retrace fails, that is, after performing the retrace operation, the observation direction cannot be restored to the azimuth or elevation angle before the microscan, the verification process is terminated and the verification is deemed to have failed.

[0176] In one embodiment of the present invention, if the verification passes, then line-of-sight geometric alignment is performed based on the direction corresponding to the cusp, and the line-of-sight compensation reference is updated, including:

[0177] The cusp direction vector is calculated based on the cusp pitch angle and the center azimuth angle. The lateral component of the cusp direction vector is equal to the product of the cosine of the cusp pitch angle and the sine of the center azimuth angle. The vertical component of the cusp direction vector is equal to the sine of the cusp pitch angle. The forward component of the cusp direction vector is equal to the product of the cosine of the cusp pitch angle and the cosine of the center azimuth angle.

[0178] Obtain the command line direction vector, calculate the dot product of the cusp direction vector and the command line direction vector, and use the inverse cosine function to obtain the angle between the cusp direction vector and the command line direction vector as the minimum rotation angle; calculate the cross product of the cusp direction vector and the command line direction vector, and normalize the cross product result as the minimum rotation axis.

[0179] The rotation matrix is ​​calculated using the Rodriguez formula based on the minimum rotation angle and minimum rotation axis. The rotation matrix is ​​then multiplied by the current line-of-sight compensation reference attitude to obtain the updated line-of-sight compensation reference attitude, which is then set as the new current line-of-sight compensation reference attitude.

[0180] In detail, the coordinate system for the cusp direction vector is based on the ship itself. The lateral direction refers to the horizontal direction perpendicular to the ship's bow and stern, i.e., the port and starboard direction, with the positive direction being from midships to starboard. The vertical direction refers to the direction perpendicular to the horizontal plane, with the positive direction being upwards. The forward direction refers to the bow and stern direction, with the positive direction being from stern to bow. These three directions are mutually perpendicular, forming a right-handed coordinate system, ensuring that the calculation of the direction vector conforms to the spatial logic of ship motion.

[0181] In detail, when calculating the dot product of the cusp direction vector and the command line direction vector, due to potential errors in numerical calculation, the dot product result may slightly exceed the theoretical range of -1 to 1. In this case, the system performs numerical clamping on the result: if the dot product result is greater than 1, it is truncated to 1; if it is less than -1, it is truncated to -1. This ensures the validity of the subsequent inverse cosine function calculation and avoids errors in angle calculation due to numerical overflow.

[0182] In detail, after calculating the cross product of the cusp direction vector and the command line direction vector, the result needs to be normalized to obtain a unit vector as the minimum rotation axis. When the two vectors are collinear (same direction) or opposite (opposite directions), the cross product result is a zero vector and cannot be directly normalized. In this case, the system defaults to taking the vertical direction as the rotation axis to ensure that the rotation matrix calculation has a clear reference axis and avoid attitude update failure due to an uncertain rotation axis.

[0183] In detail, the Rodriguez formula is used to convert rotation angles and rotation axes into rotation matrices. Its matrix form is achieved as follows: using the smallest rotation axis as the unit vector, combined with the smallest rotation angle, an antisymmetric matrix is ​​constructed. Then, the antisymmetric matrix is ​​combined with the identity matrix using a formula to generate the rotation matrix. The rotation direction follows the right-hand rule, i.e., the right thumb points in the positive direction of the rotation axis, and the direction of the curled fingers indicates the rotation direction, ensuring consistency with the logic of ship attitude adjustment.

[0184] In detail, the multiplication of the rotation matrix with the current line-of-sight compensation reference attitude uses a right-multiplication convention, meaning the updated line-of-sight compensation reference attitude is equal to the rotation matrix multiplied by the current line-of-sight compensation reference attitude on the right. This order ensures that the rotation operation is performed based on the current reference attitude, rotating by the smallest angle along the smallest rotation axis, which conforms to the conventional logic of rotation before translation in coordinate transformation, guaranteeing the accuracy of attitude updates.

[0185] In detail, once the verification is successful, the system immediately performs a multiplication operation between the rotation matrix and the current reference attitude to generate an updated line-of-sight compensation reference attitude. This updated reference attitude is then written back to the system in the next system control cycle after the calculation is completed, serving as the new current line-of-sight compensation reference attitude. This timing arrangement ensures the real-time nature of the attitude update, enabling line-of-sight compensation to respond quickly to the verification results and maintain the dynamic stability of the system.

[0186] In one embodiment of the present invention, if a curve exists but the verification fails, the current line-of-sight compensation reference is frozen, including:

[0187] Maintain the current line-of-sight compensation reference posture unchanged.

[0188] In detail, the existence of a curve is determined based on the integrity of the three curves. Specifically, it is necessary to check that the upper bright edge curve, the center curve of the dark band, and the lower bright edge curve have corresponding valid elevation angle values ​​in all azimuth angles covered by the observation window, and that the curves are continuous without obvious breaks. That is, the elevation angle values ​​of the center of the dark band and the upper bright edge can be extracted from each azimuth angle unit, and the order of the three should maintain the relationship that the lower bright edge is smaller than the center of the dark band and the center of the dark band is smaller than the upper bright edge in most azimuth angles, without large-scale missing or disordered values. At this point, the curve is determined to exist.

[0189] In detail, the criteria for determining verification failure follow the failure conditions of claim 6. After performing azimuth and elevation micro-scan retracements, verification is deemed to have failed if any of the following conditions are met: 1) The three curves after azimuth or elevation retracement do not maintain the order of the lower bright edge being smaller than the center of the dark band and the center of the dark band being smaller than the upper bright edge within the observation window; 2) The cusp parameters after azimuth retracement are not entirely the same as the cusp parameters before micro-scanning; 3) The cusp parameters after elevation retracement are not entirely the same as the cusp parameters before micro-scanning. If any of the above conditions are met, verification is considered to have failed.

[0190] In detail, the frozen state means that the current line-of-sight compensation reference attitude remains unchanged. During the frozen state, the system continuously calculates and updates the upper bright edge curve, the center curve of the dark band, and the lower bright edge curve, as well as the corresponding cusp pitch angle, center azimuth angle, and minimum interval spacing. These calculation results are used to monitor the state changes of the curves and cusps in real time; only the line-of-sight compensation reference attitude is not updated.

[0191] In detail, there are two conditions for exiting the frozen state. The first is that the system re-executes the verification process during the freeze period. If the verification passes, all passing conditions are met, and the system exits the frozen state, enters the locked state, and updates the line-of-sight compensation reference attitude. The second is that during the freeze period, the curves no longer meet the existence criteria, i.e., there is a large-scale missing or disordered appearance of the three curves. In this case, the system exits the frozen state, enters the rollback state, and switches to a non-horizontal reference attitude.

[0192] In detail, during the freeze period, the system's output control of the actuators is zero-incremental. That is, the actuators maintain their current attitude output, do not receive any new adjustment commands, and retain the last attitude output value before the freeze. This control method ensures that the line-of-sight compensation reference attitude remains stable during the freeze period, avoiding line-of-sight jitter caused by frequent adjustments.

[0193] In one embodiment of the present invention, if the curve does not exist, the process reverts to a non-horizontal reference attitude, including:

[0194] Replace the current line-of-sight compensation reference attitude with a non-horizontal reference attitude, where the non-horizontal reference attitude is a preset stable reference attitude.

[0195] In detail, the determination of a non-existent curve is based on the generation status of the three curves within the observation window. The observation window is the azimuth coverage area corresponding to the cusp, and the three curves refer to the upper bright edge curve, the center curve of the dark band, and the lower bright edge curve. When the curve generation process cannot extract the complete three curves within this area, i.e., the elevation angle values ​​of the center of the lower bright edge and dark band or the upper bright edge are missing in most azimuth angles, or the number of extracted extreme points is insufficient to form a continuous curve, or the curve has a large-scale break that cannot be supplemented by interpolation, the curve is determined to be non-existent.

[0196] In detail, there are three sources of non-horizontal reference attitude. The first is attitude data provided by the inertial measurement unit (IMU), which calculates the ship's spatial attitude in real time using internal sensors and exhibits short-term stability. The second is the output of the platform stabilization loop, which maintains its own attitude stability through mechanical or electronic means, and its output attitude is unaffected by short-term hull swaying. The third is the startup calibration attitude, which is the initial reference attitude calibrated manually or automatically during system startup and serves as the initial reference after system startup. Attitude data from all three sources are pre-stored in the system, and one is selected as the non-horizontal reference based on the actual situation.

[0197] In detail, the replacement operation adopts a direct replacement method. That is, when the determination curve does not exist, the system immediately replaces the current line-of-sight compensation reference attitude directly with the non-horizontal reference attitude, without performing any rotation, superposition, or transition processing. During the replacement process, the current attitude is completely covered by the non-horizontal reference attitude, ensuring the determinism of attitude switching and avoiding attitude fluctuations caused by transition processing.

[0198] In detail, the trigger condition for transitioning from the frozen state to the rollback state is that the curve is detected as non-existent during the frozen period. The trigger condition for transitioning from the normal state (i.e., the locked state) to the rollback state is that the curve is determined to be non-existent after the curve generation process is executed in the locked state.

[0199] The exit condition for the rollback state is that the curves are restored and the verification is passed. That is, the system continuously monitors the curve generation status during the rollback period. When all three curves are regenerated within the observation window and the verification process is passed, the system exits the rollback state, enters the locked state, and updates the line-of-sight compensation reference attitude.

[0200] In detail, when a rollback action occurs, the system automatically records relevant information, including a timestamp of the rollback occurrence, accurate to milliseconds; the reason for the rollback, explicitly marked as the curve not existing; and the line-of-sight compensation reference posture data before and after the rollback.

[0201] During the reversal, the actuator's control logic maintains the non-horizontal reference attitude unchanged. That is, the actuator receives the non-horizontal reference attitude as an output command without making any additional adjustments, ensuring the ship's line of sight remains stable on the reference attitude and avoiding safety risks caused by frequent attitude changes. All recorded information will be saved to the system log.

[0202] The embodiments of this example have been described above. However, this example is not limited to the specific implementation methods described above. The specific implementation methods described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms based on the guidance of this example, and all of them are within the protection scope of this example.

Claims

1. A ship line-of-sight compensation monitoring system based on multi-source sensing, characterized in that, include: The image acquisition module acquires global images and combines them with ship attitude sensing data to map the global images into panoramic angle domain images based on azimuth and pitch angles. The image masking module masks non-horizontal edges in panoramic angle domain images based on ship navigation sensor data. The curve generation module extracts gradient extrema along the pitch angle in the updated panoramic angle domain image after masking to obtain a curve associated with the azimuth angle; the curve includes: upper bright edge curve, dark band center curve and lower bright edge curve. The cusp generation module analyzes multiple elevation angle slices to calculate the azimuth coverage range and interval spacing of the dark band center curve in each slice, and determines the cusp corresponding to the minimum interval spacing. The verification module applies a preset azimuth micro-scan step size and a preset pitch micro-scan step size to the observation direction and performs a retrace operation. The verification is based on the topological relationship of the curves before and after the retrace and the stability of the cusp. If the reference switching module passes the verification, it performs line-of-sight geometric alignment based on the direction corresponding to the cusp and updates the line-of-sight compensation reference; if the curve exists but the verification fails, it freezes the current line-of-sight compensation reference; if the curve does not exist, it reverts to the non-horizontal reference attitude.

2. The ship line-of-sight compensation monitoring system based on multi-source sensing according to claim 1, characterized in that, By acquiring a global image and combining it with ship attitude sensing data, the global image is mapped into a panoramic angle domain image based on azimuth and pitch angles, including: Panoramic images covering the target's field of view are acquired using multi-source image sensors; Synchronously read ship attitude sensor data, which includes gimbal attitude matrix and inertial measurement unit attitude matrix; Load the intrinsic parameter matrix and the extrinsic parameter matrix from the camera to the ship's coordinate system for each camera; Multiply the attitude matrix of the inertial measurement unit by the attitude matrix of the gimbal, and then multiply it by the extrinsic parameter matrix from the camera to the hull to obtain the combined attitude matrix. For each pixel in the global image, the inverse of the camera intrinsic parameter matrix is ​​used to perform an inverse transformation on the pixel coordinates to obtain the ray direction vector in the camera coordinate system. The ray direction vector is then transformed into a unit line-of-sight vector by combining the pose matrix. The azimuth angle is calculated from the lateral and forward components of the unit line-of-sight vector using the arctangent function, and the pitch angle is calculated from the vertical component of the unit line-of-sight vector using the arcsine function. Define discrete grids for azimuth and pitch angles, and establish a mapping relationship between pixels and angle grids. For multiple pixels falling into the same angle grid cell, perform a weighted average based on the reciprocal of the angular distance from the pixel to the center of the angle grid cell to obtain the gray value of the angle grid cell. For angle grid cells where no pixel falls, interpolate the gray values ​​of neighboring angle grid cells to generate a panoramic angle domain image.

3. A ship line-of-sight compensation monitoring system based on multi-source sensing according to claim 2, characterized in that, Masking of non-horizontal edges in panoramic angle domain images based on ship navigation sensor data includes: Under the same time reference corresponding to the panoramic angle domain image, ship navigation sensor data is acquired synchronously. The ship navigation sensor data includes the automatic identification system information set, radar detection information set, and electronic chart data set. Declare the ship's observation altitude, geographical coordinates, and heading; For each target in the Automatic Identification System (AIS) information set, the planar coordinates relative to the ship are calculated from the target's geographic coordinates. The relative distance and azimuth of the planar coordinates relative to the ship are then calculated from the planar coordinates. The target's azimuth coverage area is determined by combining the target's length, beam, and heading. The upper and lower bounds of the pitch angle are calculated based on the ship's observation height and the target's superstructure height to obtain the target's coverage area in the angular domain. For each echo cluster in the radar detection information set, the azimuth coverage range of the echo cluster is determined based on the measurement range, measurement azimuth, and beamwidth of the echo cluster; the upper and lower bounds of the pitch angle are calculated based on the ship's observation height and measurement range to obtain the coverage range of the echo cluster in the angular domain. For each shoreline segment in the electronic nautical chart dataset, shoreline sampling points are obtained by sampling the shoreline at fixed intervals. The relative distance and azimuth of each shoreline sampling point relative to the ship are calculated, and the pitch angle corresponding to the shoreline sampling point is calculated based on the ship's observation height, thus obtaining the shoreline coverage set in the angular domain. The angle domain coverage range or angle domain coverage set obtained from the ship automatic identification system information set, radar detection information set and electronic nautical chart data set are merged to form a masking film. The occlusion mask is multiplied bitwise with the panoramic angle domain image to obtain the updated panoramic angle domain image that has been masked.

4. A ship line-of-sight compensation monitoring system based on multi-source sensing according to claim 3, characterized in that, In the updated panoramic angle domain image after masking, gradient extrema are extracted along the pitch angle to obtain a curve associated with the azimuth angle, including: The first-order gradient in the pitch direction is calculated for the updated panoramic angle domain image according to a fixed pitch angle grid step size. The first-order gradient is calculated using the central difference method, that is, the first-order gradient at a certain pitch angle position is equal to the difference between the gray value of the adjacent previous pitch angle position and the gray value of the adjacent next pitch angle position divided by twice the pitch angle grid step size. The square of the first gradient is used as the gradient energy, and the second derivative of the gradient energy in the pitch direction is calculated according to the same pitch angle grid step size. The second derivative is calculated using the central difference method. The stationary point of the gradient energy is extracted at each fixed azimuth angle, and the maxima candidate point and minima candidate point are determined by combining the sign of the second derivative. Stationary points where the second derivative is less than zero are considered as maxima candidates; Stationary points where the second derivative is greater than zero are considered as candidate minimum points; At each fixed azimuth angle, select triplets from the maxima and minima that satisfy the order of lower bright edge less than dark band center less than upper bright edge, where the lower bright edge and upper bright edge are maxima and the dark band center is a minima. Between adjacent azimuth angles, the sum of the absolute difference in pitch angle between the lower bright edge and the center of the dark band and the upper bright edge in the triplet is used as the cost function. The triplet that minimizes the cost function is selected, and cubic spline interpolation is performed on the obtained discrete triplet sample points along the azimuth direction to obtain the lower bright edge curve, the dark band center curve, and the upper bright edge curve.

5. A ship line-of-sight compensation monitoring system based on multi-source sensing according to claim 4, characterized in that, By analyzing multiple elevation angle slices, the azimuth coverage range and interval of the dark band center curve in each slice are calculated, and the cusp corresponding to the minimum interval is determined, including: A set of pitch angle slices is generated at intervals of pitch angle grid steps, and each pitch angle slice corresponds to a fixed pitch angle value. For each pitch angle slice, it is determined whether there is an azimuth angle such that the pitch angle value of the lower bright edge curve at the azimuth angle is less than the pitch angle value of the pitch angle slice, and the pitch angle value of the upper bright edge curve at the azimuth angle is greater than the pitch angle value of the pitch angle slice. Pitch angle slices that meet the conditions are marked as valid slices. For each valid slice, let the difference function f(ψ) be: f(ψ)=y ADZX (ψ)-y YXQP Among them, y ADZX (ψ) represents the center curve of the dark band, y YXQP The pitch angle value represents the effective slice, and ψ represents the azimuth angle. In a discrete azimuth grid, the sign change of the difference function at the center of two adjacent azimuth cells is detected, and one-dimensional linear interpolation is performed along the azimuth direction between the centers of adjacent azimuth cells where the sign change occurs to obtain the azimuth of all intersection points between the dark band center curve and the effective slice. For all intersection azimuths of each valid slice, the smallest azimuth is taken as the left end of the azimuth coverage interval of the valid slice, and the largest azimuth is taken as the right end of the azimuth coverage interval of the valid slice. The difference between the right and left ends of the azimuth coverage interval of each effective slice is calculated to obtain the interval spacing of the effective slice. The interval spacing of all effective slices is then arranged in ascending order according to the elevation angle value of their respective elevation angle slices to form an interval spacing sequence. The pitch angle value of the pitch angle slice corresponding to the minimum value in the interval spacing sequence is selected as the cusp pitch angle; The average of the left and right ends of the azimuth coverage interval corresponding to the pitch angle of the cusp is taken as the center azimuth angle of the cusp, and the minimum value in the interval spacing sequence is taken as the minimum interval spacing corresponding to the cusp.

6. A ship line-of-sight compensation monitoring system based on multi-source sensing according to claim 5, characterized in that, A preset azimuth microscan step size and a preset elevation microscan step size are applied to the observation direction, and a retrace operation is performed. Verification is then conducted based on the topological relationship of the curves before and after the retrace and the stability of the cusps, including: S61, the azimuth coverage area corresponding to the pitch angle of the cusp is used as the observation window; S62 records the lower bright edge curve, the dark band center curve and the upper bright edge curve before performing micro-scan, as well as the apex pitch angle, center azimuth angle and minimum interval spacing. S63, perform a positive azimuth micro-scan on the observation direction, increasing the azimuth of the observation direction by a preset azimuth micro-scan step size; after completing the positive azimuth micro-scan, perform an azimuth retrace, decreasing the azimuth of the observation direction by a preset azimuth micro-scan step size to restore it to the azimuth before the micro-scan; collect the first retrace lower bright edge curve, the first retrace dark band center curve, and the first retrace upper bright edge curve after the retrace, and calculate the corresponding first retrace tip elevation angle, the first retrace center azimuth angle, and the minimum value of the first retrace interval spacing; S64, perform a positive pitch angle micro-scan on the observation direction, increasing the pitch angle of the observation direction by a preset pitch micro-scan step size; after completing the positive pitch angle micro-scan, perform a pitch retrace, decreasing the pitch angle of the observation direction by a preset pitch micro-scan step size to restore it to the pitch angle before the micro-scan; collect the second retrace lower bright edge curve, the second retrace dark band center curve, and the second retrace upper bright edge curve, and calculate the corresponding second retrace apex pitch angle, second retrace center azimuth angle, and minimum interval of the second retrace interval; S65, within the observation window, verify whether the first scan lower bright edge curve, the first scan dark zone center curve and the first scan upper bright edge curve after azimuth retracement are maintained: the pitch angle value of the first scan lower bright edge curve is less than the pitch angle value of the first scan dark zone center curve, and the pitch angle value of the first scan dark zone center curve is less than the pitch angle value of the first scan upper bright edge curve. S66. Within the observation window, verify whether the second sweep of the bright edge curve, the second sweep of the dark zone center curve, and the second sweep of the bright edge curve are maintained: the pitch angle value of the second sweep of the bright edge curve is less than the pitch angle value of the second sweep of the dark zone center curve, and the pitch angle value of the second sweep of the dark zone center curve is less than the pitch angle value of the second sweep of the bright edge curve. S67. Compare whether the pitch angle of the first retrace point, the azimuth angle of the first retrace point and the minimum interval of the first retrace point after the azimuth retrace are exactly the same as the pitch angle of the first retrace point, the azimuth angle of the center and the minimum interval of the first retrace before the azimuth microscan is performed. S68. Compare the pitch angle of the tip of the second retrace after pitch retrace, the azimuth angle of the center of the second retrace and the minimum interval of the second retrace after pitch retrace with the pitch angle of the tip of the tip, the azimuth angle of the center of the second retrace and the minimum interval of the interval before microscan. S69. If the curve order in S65 and S66 remains unchanged, and the corresponding three cusp quantities in S67 and S68 are exactly the same as before the microscan, then the verification passes; otherwise, the verification fails.

7. A ship line-of-sight compensation monitoring system based on multi-source sensing according to claim 6, characterized in that, If the verification passes, then perform line-of-sight geometric alignment based on the direction corresponding to the cusp, and update the line-of-sight compensation reference, including: The cusp direction vector is calculated based on the cusp pitch angle and the center azimuth angle. The lateral component of the cusp direction vector is equal to the product of the cosine of the cusp pitch angle and the sine of the center azimuth angle. The vertical component of the cusp direction vector is equal to the sine of the cusp pitch angle. The forward component of the cusp direction vector is equal to the product of the cosine of the cusp pitch angle and the cosine of the center azimuth angle. Obtain the command line direction vector, calculate the dot product of the cusp direction vector and the command line direction vector, and use the inverse cosine function to obtain the angle between the cusp direction vector and the command line direction vector as the minimum rotation angle; calculate the cross product of the cusp direction vector and the command line direction vector, and normalize the cross product result as the minimum rotation axis. The rotation matrix is ​​calculated using the Rodriguez formula based on the minimum rotation angle and minimum rotation axis. The rotation matrix is ​​then multiplied by the current line-of-sight compensation reference attitude to obtain the updated line-of-sight compensation reference attitude, which is then set as the new current line-of-sight compensation reference attitude.

8. A ship line-of-sight compensation monitoring system based on multi-source sensing according to claim 7, characterized in that, If the curve exists but the verification fails, freeze the current line-of-sight compensation reference, including: Maintain the current line-of-sight compensation reference posture unchanged.

9. A ship line-of-sight compensation monitoring system based on multi-source sensing according to claim 8, characterized in that, If the curve does not exist, the system will revert to a non-horizontal reference attitude, including: Replace the current line-of-sight compensation reference attitude with a non-horizontal reference attitude, where the non-horizontal reference attitude is a preset stable reference attitude.

Images