Ice layer thickness identification and stress sensor cooperative navigation monitoring method and system

Abstract

The application provides a kind of ice layer thickness identification and stress sensing cooperative navigation monitoring method and system, related to ship navigation safety monitoring technical field.The method comprises: real-time acquisition ship around sea ice image and pre-processing;Through marking, a self-built dataset is made, a deep learning model is selected for segmentation, the segmentation result of snow layer and ice layer is obtained, the mask pixel value of snow layer and ice layer is obtained, and the actual thickness is calculated;Real-time stress data is acquired;With timestamp synchronization mechanism, the sea ice image and stress data corresponding to the time frame are matched;And the sea ice image and stress data are normalized, the normalized data is weighted and fused, and the comprehensive risk coefficient is calculated;And according to the interval where the coefficient is located, the risk warning level is output;When the coefficient exceeds the set threshold, sound and light alarm and interface warning information are sent out.The application improves the navigation safety of ship in polar and sea ice area through multi-dimensional data cooperative analysis and risk assessment.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of ship navigation safety monitoring, in particular to an ice layer thickness identification and stress sensing cooperative navigation monitoring method and system. BACKGROUND

[0002] With the increase of polar navigation, sea ice poses a great threat to ship safety. Especially during the navigation process of icebreaker or polar research ship, real-time monitoring of sea ice thickness is the key to ensure ship safety. At present, the common sea ice monitoring methods mainly rely on manual observation, satellite remote sensing and sea ice buoy, etc. Although these methods have certain effect, they have great defects in accuracy under complex weather conditions and are difficult to respond to the ship navigation state in real time.

[0003] Traditional ship stress monitoring mostly focuses on static mechanics analysis, ignoring the dynamic influence of sea ice. Due to the complex relationship between sea ice thickness and ship stress, the existing stress monitoring system is difficult to achieve comprehensive evaluation of the dynamic impact force caused by sea ice.

[0004] At present, when the ship is sailing in the ice area, the crew's visual observation and radar ice measurement lack accurate perception of the actual load on the ship structure. The existing monitoring system is mostly single parameter alarm, which cannot comprehensively evaluate the dynamic risk of ice conditions and ship stress, resulting in delayed early warning or high false alarm rate.

[0005] Therefore, a cooperative perception, real-time intelligent monitoring safety system is needed to comprehensively evaluate the influence of surrounding sea ice on ship navigation and timely warning. SUMMARY

[0006] In view of the technical problems that the existing sea ice monitoring method has poor accuracy under complex weather conditions and is difficult to respond in real time, the ship stress monitoring is difficult to comprehensively evaluate the dynamic impact force, and there is a lack of dynamic risk evaluation of ice conditions and ship stress. The present application proposes an ice layer thickness identification and stress sensing cooperative navigation monitoring method and system to ensure the safety of ship navigation in the ice area.

[0007] To achieve the above purpose, the technical scheme adopted by the present application is:

[0008] An ice layer thickness identification and stress sensing cooperative navigation monitoring method, the method comprising:

[0009] The shipboard camera acquires sea ice images around the ship in real time and performs preprocessing;

[0010] The preprocessed sea ice images are labeled to form a self-built data set, a deep learning model is selected for segmentation, the segmentation results of the snow layer and the ice layer are obtained, the mask pixel values of each snow layer and ice layer are obtained, and the actual thickness is calculated;

[0011] Real-time stress data is acquired from stress sensors at various positions of the ship body;

[0012] The sea ice image and the stress data corresponding to the frame are matched by using a timestamp synchronization mechanism, the sea ice image and the stress data are normalized, the normalized data are weighted and fused, a comprehensive risk coefficient R is calculated, and a corresponding risk warning level is output according to an interval in which the R value is located;

[0013] When the risk coefficient exceeds a set threshold value, the system sends an audible and visual alarm and interface warning information to the ship control center.

[0014] In another aspect, the application also provides a system for the above-mentioned ice layer thickness identification and stress sensing cooperative navigation monitoring method, the system comprising:

[0015] An image acquisition module acquires images of sea ice around the ship by using a ship-mounted camera to obtain sea ice images around the ship;

[0016] An ice identification module pre-processes the acquired sea ice images, and then uses a deep learning model to segment and analyze the images to identify sea ice characteristics;

[0017] A ship body stress monitoring module acquires data by using stress sensors installed at key positions of the ship body, extracts stress characteristic parameters after pre-processing the acquired data, and monitors the stress condition of the ship body in real time;

[0018] A data fusion and risk assessment module time-aligns and fuses the sea ice characteristic parameters output by the ice identification module and the stress characteristic parameters output by the ship body stress monitoring module, uses a weighted fusion algorithm, calculates a comprehensive risk coefficient according to the sea ice thickness and the stress data, and performs risk assessment;

[0019] A navigation safety warning module triggers a warning when the risk value exceeds a certain threshold value, and notifies relevant staff in different warning modes according to different degrees of risk;

[0020] A data recording and back transmission module records and uploads image data, sensor data and evaluation effect data acquired in real time to a database for storage.

[0021] Compared with the prior art, the application has the following beneficial effects:

[0022] 1. Real-time and accuracy improvement: Real-time acquisition of sea ice images through on-board cameras and rapid processing using deep learning algorithms to obtain the actual thickness of sea ice, solving the problem of insufficient accuracy and poor real-time performance of traditional methods (such as manual observation and satellite remote sensing) in complex weather conditions. And through the installation of stress sensors at key positions on the ship body, real-time monitoring of the stress on the ship body, including maximum stress, stress rate of change and other key indicators, overcoming the limitations of traditional static mechanics analysis.

[0023] 2. Multi-source data fusion and comprehensive risk assessment: The invention innovatively aligns and weights the ice thickness data and ship body stress data in time sequence, calculates a comprehensive risk coefficient, and realizes comprehensive assessment of navigation risk. And according to the real-time data, the risk coefficient is dynamically adjusted, and the change of the navigation environment is timely reflected, providing more accurate risk assessment results for the driver. This method is more accurate and comprehensive than a single parameter alarm system.

[0024] 3. Improved early warning mechanism and specific operation suggestions: The system sets different early warning thresholds according to the risk coefficient, and triggers the corresponding level of early warning when the risk value exceeds the threshold, including sound and light alarm, interface warning information, etc., effectively improving the timeliness and pertinence of early warning. The early warning information contains specific operation suggestions generated based on the current sea ice thickness and ship body stress data, such as reducing speed, adjusting heading, etc., providing clear decision support for the driver.

[0025] 4. Convenient system deployment and high cost-effectiveness: The system does not need to modify the main structure of the ship, the core is to intelligently fuse the data of existing or easily installed sensors and innovate algorithms, reducing the deployment difficulty and cost. Avoid relying on expensive high-performance sensors or hardware systems, providing a highly cost-effective safety enhancement means for most ships, especially those with little experience in polar navigation.

[0026] 5. Data recording and backhaul function improvement: The invention can record all collected image data, sensor data and evaluation results in real time, and regularly upload data to the database through satellite communication or wireless network, supporting data playback and historical trend analysis. The recorded data can be used for subsequent model optimization or accident tracing, helping to continuously improve the accuracy and reliability of the system.

[0027] Other features and advantages of the embodiments of the invention will be described in detail in the subsequent specific embodiments. BRIEF DESCRIPTION OF DRAWINGS

[0028] Figure 1 is a flow chart of the ice thickness identification and stress sensor coordinated navigation monitoring method according to the invention;

[0029] Figure 2 is a risk assessment flow chart according to the invention;

[0030] Figure 3 is a navigation monitoring system architecture diagram of ice layer thickness identification and stress sensing cooperation according to the application. DETAILED DESCRIPTION

[0031] In order to make the technical personnel in the art better understand the technical solutions in the present application, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application, so that the purpose, characteristics and advantages of the present application can be more clearly understood. It should be understood that the embodiments shown in the drawings are not a limitation on the scope of the present application, but only to illustrate the essential spirit of the technical solutions of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, not all. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor should belong to the scope of protection of the present application.

[0032] Unless the context clearly requires otherwise, throughout the description and the claims, the words "comprise", "comprising", and the like are to be construed in an inclusive sense as opposed to an exclusive or exhaustive sense; that is to say, in the sense of "including, but not limited to".

[0033] Reference throughout this specification to "one embodiment" or "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment. Thus, the appearances of the phrase "in one embodiment" or "in an embodiment" in various places throughout this specification are not necessarily all referring to the same embodiment. Furthermore, the particular features, structures, or characteristics can be combined in any suitable manner in one or more embodiments.

[0034] As used in this specification and the appended claims, the singular forms "a", "an", and "the" include plural referents unless the content clearly dictates otherwise. It should be noted that the term "comprising" or "comprises" is generally used in the sense of "including" or "includes" but not limited to, as antecedent to recitations of elements of the application.

[0035] In the following description, for the purpose of clarity, directional terms such as "front", "back", "left", "right", "upper", "lower", "inner", "outer", "over", "under", and the like are used with reference to the orientation of the illustration as placed on the page (for example, as shown in the drawings). However, it is to be understood that the application can assume various alternative orientations and, accordingly, such terms are not to be understood as limiting.

[0036] The implementation details of the embodiments of the present application will be specifically described below with reference to the drawings. The following content provides implementation details for the convenience of understanding, and is not essential for implementing the present solution.

[0037] The existing sea ice monitoring means and ship body stress monitoring methods have many limitations. The common sea ice monitoring method has poor accuracy under complex weather conditions, and it is difficult to accurately reflect the sea ice situation in real time under the ship navigation state; the traditional ship body stress monitoring focuses on static analysis, and cannot comprehensively evaluate the dynamic impact force caused by sea ice; and the existing monitoring system is mainly single parameter alarm, and lacks comprehensive dynamic risk assessment of ice conditions and ship body stress, which leads to untimely warning or high false alarm rate.

[0038] Therefore, in order to effectively protect the safe navigation of the ship in the polar environment, a new method of cooperative perception and real-time intelligent monitoring of navigation safety is urgently needed. The present application is based on this demand, and proposes a navigation monitoring method of ice layer thickness identification and stress sensing cooperation, which aims to comprehensively evaluate the influence of surrounding sea ice on ship navigation and timely warning.

[0039] The present application uses the sea ice image collected by the shipborne camera and the ship body stress data collected by the stress sensor during the ship navigation process, evaluates the navigation risk and provides warning by using deep learning and data fusion technology, and ensures the safe navigation of the ship in the polar environment. As shown in the figure, the specific method includes the following steps: Figure 1

[0040] S1: The shipborne camera acquires the sea ice image around the ship in real time and performs preprocessing, which includes:

[0041] S1a: Collecting sea ice images near the ship by using a shipborne camera;

[0042] During the ship navigation, a high-resolution and high-sensitivity shipborne camera is installed at a suitable position of the ship body to ensure that its field of view can cover the sea ice situation of the key area around the ship. The camera continuously collects sea ice images near the ship at a preset frame rate, which is determined comprehensively according to the ship navigation speed, sea ice change frequency and other factors, so as to ensure that the dynamic change details of the sea ice can be captured. For example, when the ship navigation speed is fast or the sea ice movement is more violent, the frame rate can be appropriately increased; otherwise, the frame rate can be appropriately reduced to reduce the data storage and processing pressure. The collected images are stored in the original format, and the related information such as the collection time and the ship position is recorded, which provides complete background information for subsequent data analysis and processing.

[0043] S1b: Obtain the intrinsic matrix and distortion coefficient of the camera through calibration, and establish the mapping relationship between the pixel coordinates and the actual space coordinates;

[0044] ​During the imaging process, due to factors such as lens manufacturing process, the camera will produce a certain distortion, resulting in the deviation of the shape and position of the objects in the image from the actual scene. In order to eliminate this distortion and accurately obtain the actual size and position information of sea ice, the camera needs to be calibrated. Professional calibration boards can be used to shoot multiple sets of calibration board images at different angles and positions, and image processing algorithms can be used to extract the feature point coordinates on the calibration board. Combined with the actual physical size of the calibration board, the intrinsic matrix of the camera (including focal length, principal point coordinates, etc.) and the distortion coefficient (radial distortion coefficient and tangential distortion coefficient) can be calculated through specific mathematical models and algorithms. Based on these parameters, the mapping relationship between pixel coordinates and actual spatial coordinates is established , which provides a theoretical basis for accurately measuring the thickness and position of sea ice from images in the future.

[0045] S1c: Preprocessing such as distortion correction is performed on the collected images.

[0046] According to the camera intrinsic matrix and distortion coefficient calculated above, the collected sea ice images are subjected to distortion correction. Suitable distortion correction algorithms such as Brown-Conrady distortion model are used to perform coordinate transformation on each pixel in the image, eliminating the effects of lens distortion and restoring the shape and position of objects in the image to their true state. At the same time, in order to improve the quality of the image and the accuracy of subsequent processing, other preprocessing operations can also be performed on the corrected image, such as grayscale processing (converting color images to grayscale images to reduce data volume and highlight brightness information), image enhancement (enhancing the contrast and clarity of the image through histogram equalization, contrast stretching, etc.), filter denoising (using median filter, Gaussian filter, etc. to remove noise interference in the image), etc. After these preprocessing operations, clear and stable sea ice image data with high quality is finally formed, providing reliable data support for subsequent sea ice thickness identification and risk assessment.

[0047] S2: The preprocessed sea ice image is made into a data set by labeling, a deep learning model is selected for segmentation, the segmentation results of the snow layer, ice layer and background are obtained, the pixel value of each mask is obtained, and the actual thickness is calculated. Specifically, it includes:

[0048] S2a: The preprocessed sea ice image is selected and labeled with labelme to build a self-built data set;

[0049] After the preprocessing operation, the sea ice images obtained have improved overall quality, but not all images are suitable for subsequent sea ice thickness identification and risk assessment. Therefore, it is necessary to select them. In the selection process, a comprehensive evaluation is made according to the image clarity, integrity, and obviousness of sea ice features. Preferably, images with clear sea ice edges, rich texture features, no obvious obstructions, and suitable lighting conditions are selected to ensure accurate reflection of the true characteristics of sea ice. Images with blurring, severe shadow interference, or incomplete sea ice features are rejected.

[0050] After the image selection is completed, the selected images are labeled using the image labeling tool labelme to build a data set suitable for deep learning model training. During the labeling process, a professional person accurately outlines the ice layer, snow layer, and background area in the image according to the actual shape and features of sea ice. For the ice layer area, the boundary is labeled to ensure that all parts of the ice layer are covered; the snow layer area is also labeled to distinguish the range of snow cover; and the background area is labeled as other parts except the ice layer and snow layer. During the labeling process, the operation is carried out in accordance with the pre-set standards and specifications to ensure the accuracy and consistency of the labeling. After the labeling is completed, the labeling information is saved in a specific format, such as JSON format, so that the subsequent deep learning model can read and use it. The labeled images and corresponding labeling information are integrated together to form a self-built data set, providing data support for subsequent deep learning model training.

[0051] S2b: Select a deep learning model to perform semantic segmentation on the sea ice image to extract the key region mask;

[0052] After the self-built data set is constructed, a deep learning model is used to perform semantic segmentation on the sea ice image to accurately extract the mask of the ice layer, snow layer, and background area. Currently, there are various deep learning models available in the field of semantic segmentation, such as U-Net, DeepLab series, PSPNet, etc.

[0053] The self-built data set is divided into a training set and a test set according to a certain ratio, such as a ratio of 9:1 for the training set and the test set. The training set is used to train the selected deep learning model with semantic segmentation algorithm. During the training process, appropriate loss functions and optimization algorithms are used, such as cross-entropy loss function and stochastic gradient descent (SGD) algorithm or its improved algorithm (such as Adam algorithm), to continuously adjust the parameters of the model, so that the model gradually learns the features of different regions in the sea ice image. The neural network model is verified by the data in the test set, and the hyperparameters of the model, such as learning rate, batch size, etc., are adjusted according to the verification result to improve the performance and generalization ability of the model. After multiple iterations of training, until the verification accuracy meets the requirements, the training is completed, and a pixel-level sea ice thickness model is formed.

[0054] S2c: Extract the RGB value range of each mask to divide the clear ice layer and snow layer;

[0055] When the deep learning model is trained, the model is used to perform semantic segmentation on the sea ice images in the test set to obtain the masks of the ice layer, snow layer, and background area. To further clearly divide the ice layer cross-section and snow layer cross-section masks, the RGB value range of each mask needs to be extracted. The RGB value is the color intensity value of each pixel in the image in the red (R), green (G), and blue (B) channels. Different objects and regions have different distribution characteristics in the RGB space.

[0056] For each segmented mask region, all pixel points in the region are traversed, and the distribution of their RGB values is counted. By calculating statistical quantities such as mean and variance, the approximate range of the mask region in the RGB space is determined. For example, due to its physical properties, the ice layer region usually exhibits specific color characteristics in the RGB image, which may be biased towards blue or white, and its RGB value range is relatively concentrated. The snow layer region may have brighter colors due to factors such as light reflection, and its RGB value range is different from that of the ice layer. By extracting the RGB value range of each mask, the ice layer and snow layer can be more accurately distinguished, providing a reliable basis for subsequent sea ice thickness calculation. At the same time, the extracted RGB value range is visualized to intuitively observe the color characteristic differences of different regions, further verifying the accuracy of the extraction results.

[0057] S2d: Draw the largest inscribed circle on the clear mask;

[0058] After accurately extracting the ice layer cross-section and snow layer cross-section masks and determining their RGB value ranges, the largest inscribed circle needs to be drawn on the clear mask. The largest inscribed circle can reflect the maximum effective size of the mask region, which is important for calculating the sea ice thickness.

[0059] An advanced image processing algorithm and geometric calculation method are used to realize the drawing of the maximum inscribed circle. First, the edge detection of the mask area is performed to obtain its edge information. Common edge detection algorithms include Canny algorithm, Sobel algorithm, etc., which can effectively extract the edge features in the image. After obtaining the edge information, the maximum inscribed circle in the mask area is found by using geometric calculation method. In the specific implementation process, the center point of the mask area can be started, and the radius of the circle can be gradually expanded, while checking the intersection points of the circle and the mask boundary. When the radius of the circle is expanded to be tangent to the mask boundary, the circle at this time is the maximum inscribed circle. In order to ensure the accuracy of the drawing, multiple iterations and optimization methods are used to check and correct the drawing results. For example, by changing the position of the initial center point or adjusting the search strategy, the local optimal solution can be avoided, so that a more accurate maximum inscribed circle can be obtained.

[0060] S2e: Output the diameter pixel value of the maximum inscribed circle, which is the thickness pixel value of the ice layer and the snow layer;

[0061] After successfully drawing the maximum inscribed circle, the diameter pixel value of the maximum inscribed circle needs to be accurately output, which is the thickness pixel value of the ice layer cross section and the thickness pixel value of the snow layer cross section. The diameter of the maximum inscribed circle is calculated by image processing algorithm, and in the calculation process, the accuracy of the calculation result is ensured by considering factors such as the coordinate system of the image and the pixel size.

[0062] The calculated diameter pixel value is recorded and stored, and is associated with the corresponding sea ice image and mask area information. In order to facilitate subsequent processing and analysis, the diameter pixel value can be integrated with other related data (such as image acquisition time, ship position, etc.) to form a complete data record. The output diameter pixel value is visualized and displayed, for example, the maximum inscribed circle and its diameter information are marked on the image, so that the operator can intuitively understand the pixel representation of the sea ice thickness, and provide clear data source for subsequent actual thickness conversion.

[0063] S2f: Calculate the proportion coefficient by comprehensively considering the factors affecting the ice layer thickness, and convert the thickness pixel value to a more accurate actual thickness.

[0064] Based on the above S1b, the mapping relationship between the pixel coordinates and the actual space coordinates is established , according to the actual thickness of the known object, combined with the pixel thickness obtained in the image, the initial proportion coefficient is calculated by the following formula :

[0065]

[0066] Among them: is the actual thickness of the known object, is the pixel thickness obtained from the known object image.

[0067] Since the ice surface may have a certain inclination angle, the angle between the camera and the ice surface will affect the measurement of thickness, at this time the attitude sensor of the ship body (such as IMU sensor) is used to estimate the included angle between the ship body and the ice surface. The angle correction can be calculated by the following formula:

[0068]

[0069] Wherein: is the actual thickness of the identified object after correction, is the pixel thickness of the identified object measured after training the deep learning model, is the included angle between the camera and the ice surface. is a factor used to correct the measurement error caused by the angle. When the shooting angle is larger, the measured thickness or force data is smaller, which needs to be restored to the actual value through this correction factor.

[0070] Considering environmental factors such as temperature, humidity, etc. during the ship's voyage, which will also affect the actual thickness of sea ice. In order to more accurately calculate the ice thickness, real-time environmental data can be collected by sensors, and regression models such as linear regression, support vector machine regression or neural network can be used to model these data to obtain the correction coefficient of environmental factors. The thickness correction is carried out according to the following formula:

[0071]

[0072] Wherein, is calculated from environmental sensor data and regression model, indicating the correction coefficient caused by environmental factors.

[0073] The sea ice surface is not uniform, to solve this problem, a multi-scale convolutional neural network is used to process the sea ice image, so as to extract local features at different scales. By using multiple convolutional layers with different image resolutions, the details of the ice layer are captured, and the feature information of each local region is obtained, and then the local adjustment factor is calculated. This factor is learned from the local image features by machine learning models (such as KNN, random forest, support vector regression, etc.), and is adjusted in real time to ensure that the thickness calculation of different ice layers is more accurate.

[0074] All correction factors are combined to obtain the final actual thickness through weighted fusion:

[0075]

[0076] In this formula, is a proportional coefficient considering environmental factors such as temperature, density, etc., is a proportional coefficient based on local features of the image.

[0077] This method combines pose correction, environmental adjustment and multi-scale feature extraction, and can provide more accurate thickness calculation under different sea ice conditions.

[0078] In order to deal with the changes of sea ice environment and ship state, this method supports dynamic adjustment of proportional coefficient. Through incremental learning algorithm or online learning method, the value of , can be automatically optimized as the system collects more sea ice images and environmental data, ensuring the stability and adaptability of calculation accuracy in long-term use.

[0079] In the process of sea ice thickness measurement, first, the intrinsic matrix and distortion coefficient of the camera are obtained through camera calibration, and the projection error caused by the change of ship angle is corrected through the geometric correction coefficient . Then, according to the real-time sensor data (such as temperature, humidity, ship vibration, etc.), dynamic correction coefficient and image quality correction coefficient are calculated together and applied to thickness calculation. All correction coefficients are determined according to calibration data and experimental results at system initialization, and are dynamically updated according to real-time monitoring data to ensure the accuracy of each measurement. Each correction coefficient is only applied to its specific error source to avoid repeated calculation.

[0080] After obtaining the thickness pixel value of ice layer and snow layer, the pixel thickness in the image is converted to the actual physical thickness through the above method.

[0081] After conversion, the actual thickness value D of ice layer and snow layer is obtained, with unit of meter. The actual thickness value is associated with the corresponding sea ice image, collection time, ship position and other information for storage, providing accurate basic data for subsequent data analysis and risk assessment. At the same time, the conversion result is verified and calibrated, compared with the actual measurement data (such as sea ice thickness measured by laser range finder and other equipment), the accuracy of conversion is evaluated, and adjustment and optimization are made as needed to ensure that the final actual thickness value is reliable and accurate.

[0082] S3: Obtain real-time stress data from stress sensors at each position of the ship body, filter and denoise the collected signals, and obtain key indicators such as maximum stress , stress change rate , stress fluctuation amplitude The processed data is transmitted to the data fusion and risk assessment module. The output results include a timestamp t, a sensor number ID sen, and a real-time stress value .

[0083] S4: The sea ice image and stress data corresponding to the frame are matched using a timestamp synchronization mechanism. The two types of data are normalized based on the stress that the ship body can withstand and the ice class. The normalized data is weighted and fused to calculate the comprehensive risk coefficient R. According to the interval in which the R value is located, the corresponding risk warning level is output. Specifically, it includes:

[0084] S4a: Based on the unified timestamp, the ice layer thickness data and the ship body stress data are synchronized and aligned to form a corresponding multi-source data set.

[0085] During system operation, the ship-mounted camera of the image acquisition module and the stress sensor of the ship body stress monitoring module independently collect sea ice image data and ship body stress data. Since the collection of the two is asynchronous, there may be a mismatch in the time dimension of the data, which will affect the accuracy of subsequent data fusion and risk assessment. Therefore, it is necessary to synchronize and align the two types of data based on a unified timestamp.

[0086] In specific implementation, while the ship-mounted camera collects sea ice images, the exact collection time is recorded, and a unique timestamp is generated. Similarly, when the stress sensor collects ship body stress data, it also records the corresponding collection time and generates a timestamp. Then, the timestamp is used as an association identifier to match the ice layer thickness data and the stress data. For example, based on the timestamp of the stress data, the data within the allowed error range in the ice layer thickness data is searched, and they are considered as data collected at the same time, thereby forming a corresponding multi-source data set.

[0087] To ensure the accuracy and consistency of the timestamp, the system uses high-precision clock synchronization technology. The ship-mounted camera and the stress sensor are synchronized with a high-precision clock source, which can be an accurate time provided by the Global Positioning System (GPS) or a special high-precision clock device on the ship. In this way, the data collected by different modules can be accurately corresponded in the time dimension, providing a reliable foundation for subsequent data fusion.

[0088] After the unified timestamp, the Z-score, Isolation Forest algorithm, or deep learning autoencoder is used to detect outliers in the stress data, and the outliers are corrected through weighted average method or interpolation method to ensure the smoothness and continuity of the data.

[0089] The Z-score method determines data points exceeding 3 times the standard deviation as outliers by calculating the standard deviation of the data. The Isolation Forest method determines outliers by building multiple trees to score data points, with high-scoring data points being determined as outliers. The Autoencoder method detects errors in data points by learning the reconstruction error of normal data, with data points having larger errors being considered as outliers. For detected outliers, the weighted average method or interpolation method is used for correction. The weighted average method combines the information of the previous and subsequent data points, or the interpolation method restores the value of the abnormal data point. The corrected data will be verified again to ensure the smoothness and continuity of the data, and to avoid the introduction of new outliers.

[0090] S4b: After the spatial and temporal alignment of the sea ice thickness data and the hull stress data, the two parameters are normalized to obtain the sea ice risk factor T and the stress risk factor S;

[0091] Due to the large difference in the dimension and numerical range of the ice layer thickness data and the hull stress data, direct fusion analysis will be affected by the data magnitude, leading to one type of data dominating in the fusion process, thereby affecting the accuracy of risk assessment. Therefore, the synchronized data needs to be normalized to convert them to a unified numerical range.

[0092] For ice layer thickness data, first determine its reasonable range in the current navigation environment and historical data. For example, according to the ice class and route information of the ship, determine the maximum and minimum values of the sea ice thickness that may be encountered. Then, using a linear normalization method, the ice layer thickness data D is normalized according to the formula , where , are the maximum and minimum values of the sea ice thickness, and T is the normalized sea ice risk factor, with a value range of [0, 1]. When the sea ice thickness is close to the maximum value, the sea ice risk factor is close to 1, indicating a high sea ice risk. When the sea ice thickness is close to the minimum value, the sea ice risk factor is close to 0, indicating a low sea ice risk.

[0093] For the hull stress data, the reasonable range is also determined first. According to the design parameters and structural strength of the ship, determine the maximum stress and minimum stress that the hull can withstand at different positions. Then, using a similar linear normalization method, the hull stress data is normalized according to the formula , where and are the maximum and minimum values of the hull stress, and S is the normalized stress risk factor, with a value range of [0, 1]. When the hull stress is close to the maximum value, the stress risk factor is close to 1, indicating a high hull stress risk. When the hull stress is close to the minimum value, the stress risk factor is close to 0, indicating a low hull stress risk.

[0094] S4c: The normalized ice layer thickness data and the hull stress data are weighted and fused using machine learning or configurable weight coefficients to generate a comprehensive risk coefficient R;

[0095] After completing the normalization of the ice layer thickness data and the hull stress data, they need to be weighted and fused to comprehensively consider the influence of sea ice and hull stress on ship navigation safety, and generate a comprehensive risk coefficient R.

[0096] One method is to use machine learning algorithms to determine the weight coefficients. A large amount of historical navigation data is collected, including stress data of the hull under different sea ice conditions and corresponding navigation safety event records. These data are input into the machine learning model, such as neural network, support vector machine, etc. Through training, the model can learn the relationship between ice layer thickness data and hull stress parameters and navigation safety risk, automatically adjust the weight coefficients, so that the comprehensive risk coefficient R can more accurately reflect the actual navigation risk. For example, the neural network model can minimize the error between the predicted risk value and the actual risk value by continuously adjusting the weights and biases, thereby obtaining the optimal weight coefficients.

[0097] Another method is to use configurable weight coefficients. According to the ice class, navigation mode, route characteristics and other factors of the ship, the weight coefficients α and β are manually set by professionals, where α is the weight of the sea ice risk factor and β is the weight of the stress risk factor, and α+β=1. For example, for ships with high ice class, when navigating in areas with thick sea ice, the weight of the sea ice risk factor α can be appropriately increased; and for ships with relatively weak hull structure, in the case of large stress changes, the weight of the stress risk factor β can be increased. In this way, according to different navigation scenarios and needs, the weight coefficients are flexibly adjusted to improve the pertinence and accuracy of risk assessment. According to the determined weight coefficients, the following weighted fusion formula is used to calculate the comprehensive risk coefficient R:

[0098]

[0099] This formula takes into account the influence of sea ice risk and hull stress risk on ship navigation safety, and by adjusting the weight coefficients, the importance of a certain type of risk factor can be highlighted, thereby more accurately assessing the navigation risk of the ship in the current environment.

[0100] S4d: As shown in Figure 2 , according to the comprehensive risk coefficient R, the single risk factor S, T and the preset threshold, the risk is divided into low, medium and high levels. When the risk coefficient is higher than the set threshold, the system will trigger the corresponding warning.

[0101] According to the R value and the key risk factors, the risk degree is divided into multiple levels, and the corresponding early warning signal is triggered.

[0102] R<0.4, that is, all parameters are above the lower limit of the safety threshold, the visual prompt is green, the interface displays "navigation safety", all data are normal, and the response is: keep normal navigation.

[0103] 0.4≤R<0.7, the ice thickness or stress parameter continues to rise, approaches the warning line, is set to a first early warning, low risk, the visual alarm is yellow, the early warning sound is a single prompt sound, the interface related data area flashes yellow light, and displays "attention: sea ice condition intensifies, it is suggested to keep observation". The response is: it is suggested that the driver check the navigation plan and consider reducing the speed in advance.

[0104] 0.7≤R<0.9, S>0.7 or T>0.8, set to a second early warning, medium risk, the visual alarm is orange, the early warning sound is intermittent beeping, an orange alarm box appears on the screen, clearly displays the key over-limit parameters, and gives suggestions such as "warning! High stress! It is suggested to immediately reduce the speed to X knots". The response is: the driver should follow the suggestions and immediately take operations such as reducing the speed and changing the heading.

[0105] R≥0.9, S>0.9 (stress close to the limit) or T>1.0 (ice thickness exceeds the design value), set to a third early warning, high risk, the visual alarm is red, the early warning prompt sound is continuous high-frequency beeping, the screen flashes red full screen, and displays "emergency alert! Hull stress over limit!". Clear voice instructions are issued, such as "stop navigation! Stop navigation!". The response is: the emergency operation must be immediately and unconditionally executed, and external support is prepared to be requested.

[0106] The application adopts a composite judgment logic: first, the weighted comprehensive risk coefficient R is compared with a plurality of preset level thresholds, as a judgment basis. Secondly, each single risk factor S, T is compared with its respective single threshold in parallel. The final risk level will be determined by the highest level in the above two determination results. This design ensures that the system can not only make comprehensive evaluation, but also quickly respond to any single sudden high-risk factor. This design ensures that the system can not only make comprehensive evaluation, but also quickly respond to any single sudden high-risk factor, and provides more reliable protection for ship navigation safety.

[0107] S5: when the risk coefficient exceeds the set threshold, the system issues an audible and visual alarm to the ship control center and interface warning information, displays the sea ice thickness distribution map, stress curve and risk coefficient in real time on the monitoring interface, makes suggestions for navigation according to the risk level, such as reducing the speed or adjusting the heading. At the same time, the sea ice and stress data at this moment are marked as key samples, which are convenient for subsequent data analysis.

[0108] S6: Save all collected image data, stress data, statistical results and early warning records in chronological order, upload data to the database through satellite communication or wireless network regularly, support data playback and historical trend analysis, and use for subsequent model optimization or accident tracing.

[0109] Based on the above ice thickness identification and stress sensing cooperative navigation monitoring method, the application also provides an ice thickness identification and stress sensing cooperative navigation monitoring system, as shown in the figure, the monitoring system comprises: an image acquisition module, an ice identification module, a ship stress monitoring module, a data fusion and risk assessment module, a navigation safety warning module and a data recording and return module. Figure 3

[0110] Specifically, the image acquisition module uses a ship-mounted camera to acquire images of sea ice around the ship to obtain sea ice images around the ship.

[0111] The ice identification module pre-processes the collected sea ice images to ensure image quality, and then uses a deep learning model to segment and analyze the images to identify sea ice characteristics.

[0112] The ship stress monitoring module uses stress sensors installed at key positions on the ship body to collect data, and extracts stress characteristic parameters after pre-processing such as sampling and filtering of the collected data, to monitor the stress of the ship body in real time.

[0113] The data fusion and risk assessment module time-aligns and fuses the sea ice characteristic parameters output by the ice identification module and the stress characteristic parameters output by the ship stress monitoring module, calculates a comprehensive risk coefficient according to the thickness and stress data using a weighted fusion algorithm, and performs risk assessment.

[0114] The navigation safety warning module triggers a warning when the risk value exceeds a certain threshold, and sets different warning methods to notify relevant personnel according to different levels of risk.

[0115] The data recording and return module records and uploads real-time collected image data, sensor data and evaluation effect data to the database for storage.

[0116] The system can real-time collect sea ice images during ship navigation, and calculate a comprehensive risk coefficient by weighted fusion of ship stress data and two kinds of data, thereby realizing real-time monitoring and early warning of navigation safety. The system can efficiently and accurately assess the risks that may be encountered during ship navigation, timely issue warnings, reduce ship damage caused by sea ice environment, significantly improve navigation safety, and is suitable for polar navigation environment.

[0117] ​The ice layer thickness identification and stress sensing cooperative navigation monitoring method and system provided by the application can monitor the sea ice environment and the stress on the ship body in real time through a sea ice identification module and a ship body stress sensor module. Through advanced image processing technology and data fusion algorithms, the system can calculate the actual thickness of the sea ice, and conduct comprehensive risk assessment combined with the ship body stress data, and finally provide accurate navigation safety warning. Compared with the monitoring method in the prior art which only relies on ice layer thickness or stress data, the application realizes multi-dimensional data cooperative analysis and risk assessment, and improves the navigation safety of the ship in the polar region and the sea ice area.

[0118] Although the application has been described in detail with reference to the preferred embodiments, the application is not limited to this. Those skilled in the art can make various equivalent modifications or replacements to the embodiments of the application without departing from the spirit and essence of the application, and these modifications or replacements should be within the scope of the application or any skilled person in the art can easily think of changes or replacements within the technical scope disclosed by the application, and should be covered by the protection scope of the application. Therefore, the protection scope of the application should be subject to the protection scope of the claims.

Claims

1. A navigation monitoring method that combines ice thickness identification and stress sensing, characterized in that, The method includes: The shipborne camera acquires real-time images of sea ice around the ship and performs preprocessing. The preprocessed sea ice images were labeled to create a self-built dataset. A deep learning model was used for segmentation to obtain the segmentation results of the snow layer and ice layer. The mask pixel values ​​of each snow layer and ice layer were obtained, and the actual thickness was calculated. Real-time stress data is obtained from stress sensors at various locations on the hull; The sea ice images and stress data of the corresponding time frames are matched using a timestamp synchronization mechanism; the sea ice images and stress data are normalized, and the normalized data are weighted and fused to calculate the comprehensive risk coefficient R; and the corresponding risk warning level is output according to the interval of R value. When the risk factor exceeds the set threshold, the system sends an audible and visual alarm as well as an interface warning to the ship control center.

2. The method according to claim 1, characterized in that, The method also includes: saving all collected sea ice image data, stress data, statistical results and early warning records in chronological order, and periodically uploading the data to the database via satellite communication or wireless network.

3. The method according to claim 1, characterized in that, The shipborne camera acquires real-time images of sea ice around the ship and performs preprocessing, specifically including: Using a calibration board, multiple sets of images of the calibration board are captured from different angles and positions. Image processing algorithms are used to extract the coordinates of feature points on the calibration board. Combined with the actual physical dimensions of the calibration board, the camera's intrinsic parameter matrix and distortion coefficients are calculated using mathematical models and algorithms. Based on these intrinsic parameter matrices and distortion coefficients, a mapping relationship between pixel coordinates and actual spatial coordinates is established. ; Based on the calculated camera intrinsic parameter matrix and distortion coefficients, coordinate transformation is performed on each pixel in the acquired sea ice image to eliminate the effects of lens distortion and restore the true shape and position of objects in the image. The corrected image is then preprocessed, including grayscale conversion, image enhancement, and filtering for noise reduction.

4. The method according to claim 3, characterized in that, The step of creating a self-built dataset from preprocessed sea ice images through annotation specifically includes: The sea ice images obtained after preprocessing are selected. During the selection process, a comprehensive evaluation is conducted based on multiple key indicators. Priority is given to images with clear sea ice edges, rich texture features, no obvious obstructions, and suitable lighting conditions to ensure that they can accurately reflect the true characteristics of sea ice. After selecting the images, the image annotation tool Labelme was used to annotate the selected images to build a dataset suitable for training deep learning models. During the annotation process, the ice layer, snow layer, and background area in the images were accurately delineated according to the actual shape and characteristics of sea ice. For the ice layer area, its boundary was marked to ensure that all parts of the ice layer were covered. The snow layer area was also marked to distinguish the extent of snow coverage. The background area was marked with other parts except for the ice layer and snow layer. After the annotation was completed, the annotated images and corresponding annotation information were integrated together to form a self-built dataset.

5. The method according to claim 4, characterized in that, The segmentation using a deep learning model to obtain segmentation results for the snow layer and ice layer specifically includes: The self-built dataset was divided into a training set and a test set in a 9:1 ratio. The selected deep learning model with semantic segmentation algorithm was trained using the training set. During training, a loss function and optimization algorithm were used to continuously adjust the model's parameters, allowing the model to gradually learn the features of different regions in the sea ice image. The neural network model was validated using data in the test set, and the model's hyperparameters were adjusted based on the validation results to improve the model's performance and generalization ability. After multiple iterations of training until the validation accuracy met the requirements, the training was completed, resulting in a pixel-level sea ice thickness model. After the deep learning model is trained, it is used to perform semantic segmentation on sea ice images in the test set to obtain masks for ice, snow and background areas. In order to further clearly distinguish the masks of ice and snow sections, the RGB value range of each mask is extracted. The RGB value is the color intensity value of each pixel in the image in the three channels of red (R), green (G) and blue (B). For each segmented mask region, iterate through all pixels within each mask region, statistically analyze the distribution of RGB values, and determine the range of the mask region in the RGB space.

6. The method according to claim 5, characterized in that, The process of obtaining the mask pixel values ​​for each snow and ice layer and then calculating the actual thickness specifically includes: After extracting the ice layer and snow layer cross-section masks and determining their RGB value ranges, image processing algorithms and geometric calculation methods are used to draw the maximum inscribed circle: First, edge detection is performed on the mask area to obtain its edge information; after obtaining the edge information, geometric calculation methods are used to find the maximum inscribed circle within the mask area. In the specific implementation process, starting from the center point of the mask area, the radius of the circle is gradually expanded, while checking the intersection of the circle and the mask boundary; when the radius of the circle expands to be exactly tangent to the mask boundary, the circle at this point is the maximum inscribed circle; multiple iterations and optimizations are used to verify and correct the drawing results. After drawing the largest inscribed circle, output the pixel value of the diameter of the largest inscribed circle, where the pixel value of the diameter is the pixel value of the thickness of the ice layer cross section and the pixel value of the thickness of the snow layer cross section. After obtaining the pixel values ​​of the ice and snow layers thickness, a mapping relationship is used... The pixel thickness in the image is converted to the actual physical thickness by considering the proportional coefficients of influencing factors and the geometric relationship between sea ice and ships. The conversion method is as follows: in: It is the actual thickness of the object after correction. It is the pixel thickness of the identified object after training with a deep learning model. It is the angle between the camera and the ice surface. It is the proportion coefficient of environmental factors. It is a scaling factor based on local image features.

7. The method according to claim 6, characterized in that, The mapping relationship It is obtained by combining the known actual thickness of the object with the pixel thickness obtained from the image, using the following formula: in: The actual thickness of the known object is... It is the pixel thickness obtained from an image of a known object.

8. The method according to claim 7, characterized in that, The process involves using a timestamp synchronization mechanism to match sea ice images and stress data corresponding to specific times; normalizing the sea ice images and stress data; and then weighting and fusing the normalized data to calculate the comprehensive risk coefficient R. Specifically, this includes: When the shipborne camera acquires sea ice image data, the acquisition time is recorded and a unique timestamp is generated; when the stress sensor acquires hull stress data, the corresponding acquisition time is recorded and a timestamp is generated; the timestamp is used as an association identifier to match the sea ice image data and the hull stress data. The synchronized sea ice image data and hull stress data were normalized: For the sea ice image data, its reasonable range within the current navigation environment and historical data was first determined; then, a linear normalization method was used to normalize the ice thickness data D according to the formula... Normalization was performed, where , These represent the maximum and minimum values ​​of sea ice thickness, respectively. The normalized sea ice risk factor has a value range between [0,1]. When the sea ice thickness is close to the maximum value, the sea ice risk factor is close to 1, indicating a higher sea ice risk. When the sea ice thickness is close to the minimum value, the sea ice risk factor is close to 0, indicating a lower sea ice risk. For hull stress data, based on the ship's design parameters and structural strength, the maximum and minimum stresses that the hull can withstand in different parts are determined; then, a linear normalization method is used to process the hull stress data. According to the formula Normalization was performed, where and These represent the maximum and minimum values ​​of the ship's hull stress, respectively. The stress risk factor is the normalized value, ranging from [0,1]. When the hull stress is close to the maximum value, the stress risk factor is close to 1, indicating a high risk of hull stress. When the hull stress is close to the minimum value, the stress risk factor is close to 0, indicating a low risk of hull stress. Based on the ship's ice class, navigation mode, and route characteristics, configurable weighting coefficients α and β are set, where α is the weight of the sea ice risk factor, β is the weight of the stress risk factor, and α+β=1. Based on the determined weighting coefficients, the overall risk coefficient R is calculated using the following weighted fusion formula: 。 9. The method according to claim 8, characterized in that, The step of outputting the corresponding risk warning level based on the range of R values ​​specifically includes: Based on the R-value and key risk factors, the risk level is divided into multiple levels, and corresponding early warning signals are triggered. The classification is as follows: 1) R<0.4 indicates that all parameters are above the lower limit of the safety threshold. The visual prompt is green, the interface displays that the navigation is safe, all data is normal, and the corresponding response to maintain normal navigation is given. 2) If 0.4≤R<0.7, the ice thickness or stress parameter continues to rise and approaches the warning line. This is set as a Level 1 warning, which is low risk. The visual alarm is yellow, the warning sound is a single beep, the relevant data area on the interface flashes yellow light, and the warning information and corresponding response information are displayed. 3) If 0.7≤R<0.9, S>0.7 or T>0.8, set it to Level II warning, medium risk, visual alarm is orange, warning sound is intermittent buzzing, orange alarm box appears on the screen, key out-of-limit parameters are displayed, and suggestions and responses are given; 4) R≥0.9, S>0.9 indicates that the stress is close to the limit or T>1.0 indicates that the ice thickness exceeds the design value. It is set as a level three warning, high risk, with a visual alarm in red, a continuous high-frequency beeping warning sound, and the screen flashing red in the whole screen to display the warning information and corresponding response information; and a clear voice command to stop navigation is issued.

10. A system applicable to the navigation monitoring method combining ice thickness identification and stress sensing as described in any one of claims 1 to 9, characterized in that, The system includes: The image acquisition module uses a shipborne camera to acquire images of the sea ice around the ship. The sea ice identification module preprocesses the acquired sea ice images and then uses a deep learning model to segment and analyze the images in order to identify sea ice characteristics. The hull stress monitoring module uses stress sensors installed at key locations on the hull to collect data. After preprocessing the collected data, stress characteristic parameters are extracted to monitor the hull stress in real time. The data fusion and risk assessment module performs time-series alignment and fusion of the sea ice characteristic parameters output by the sea ice identification module and the stress characteristic parameters output by the hull stress monitoring module. Using a weighted fusion algorithm, it calculates the comprehensive risk coefficient based on the sea ice thickness and stress data and performs a risk assessment. The navigation safety early warning module triggers an early warning when the risk value exceeds a certain threshold, and different warning methods are set according to different levels of risk to notify relevant personnel. The data recording and feedback module records the real-time collected image data, sensor data, and evaluation data, and uploads them to the database for storage.

Images