Ship deck wave detection statistical method based on artificial intelligence

Abstract

The invention discloses a ship deck wave detection statistical method based on artificial intelligence, and relates to the technical field of ship monitoring, and the method comprises the steps: obtaining a deck region video of a ship deck region at a current moment; inputting the deck area video into the deck upward wave detection model, and detecting each frame of deck area image to obtain a time sequence of deck upward wave detection results; the deck rising wave detection model is used for generating a hull mask of each frame of deck area image in the deck area video, and determining a deck rising wave detection result of each frame of deck area image based on contour features and semantic features of the hull mask; and carrying out merging processing on the time sequences of the deck wave detection results, filtering, and carrying out statistics to obtain the wave times of the ship deck area. According to the method, real-time sensing and quantitative evaluation of the ship deck surging event under real ship navigation are realized, and important support is provided for guaranteeing the safety of ships and seamen and subsequent ship type optimization.

Description

Technical Field

[0001] This application relates to the field of ship monitoring technology, and in particular to an artificial intelligence-based method for detecting and statistically analyzing waves on ship decks. Background Technology

[0002] Deck surge is an extreme fluid impact phenomenon generated by the nonlinear interaction between a ship and waves. Its hazards are mainly reflected in three dimensions: (1) instantaneous impact damage, the impact pressure of a single surge can reach 50-200 kPa (equivalent to 2-8 tons / square meter), which is enough to damage the fixed equipment on the deck; (2) continuous water accumulation risk, the design standard of the deck drainage system is to accommodate rainfall of 50-100 mm / hour, but a strong surge can generate 300 mm of water accumulation within 60 seconds, which will cause changes in the ship's GM value and lead to loss of stability, seriously threatening the safety of the ship; (3) personnel operation danger, in the reports of casualties among deck workers in the shipping industry, sudden deck surge is one of the common causes. Therefore, the detection of deck surge is of great significance for the stability of the ship and the safety of the workers.

[0003] Traditional wave detection on deck primarily employs two methods: manual analysis and mechanical sensors. Manual analysis involves installing fixed-view cameras around the hull to record wave activity on the deck and then analyzing the data through human observation. This method requires frame-by-frame manual observation, which is cumbersome, time-consuming, and inefficient when processing long videos or complex sea conditions. Furthermore, it is susceptible to subjective human factors that can lead to misjudgments or omissions, making accuracy difficult to guarantee. Mechanical sensor-based wave detection monitors the frequency and intensity of waves on the deck by installing strain gauge arrays or pressure sensors on the hull or deck surface. This method is significantly affected by external environmental factors, such as human impacts and seawater corrosion, resulting in a high failure rate. Moreover, installing mechanical sensors requires damaging the deck or hull structure, leading to high costs and consequences. Summary of the Invention

[0004] This application addresses the aforementioned problems and technical needs by proposing an artificial intelligence-based method for detecting and statistically analyzing waves on ship decks. The technical solution of this application is as follows: An artificial intelligence-based method for detecting and statistically analyzing waves on a ship's deck includes the following steps: Acquire a video of the ship's deck area at the current moment. The deck area video includes images of the deck area of ​​the ship at the current moment and within a predetermined continuous time period T prior to that moment. The deck area video is input into the deck wave detection model, and the wave detection results are obtained by detecting each frame of the deck area image. The deck wave detection model is used to generate the hull mask of each frame of the deck area image in the deck area video, and determines the wave detection result of each frame of the deck area image based on the contour features of the hull mask of each frame of the deck area image and the semantic features of each frame of the deck area image. The wave detection result indicates the presence of waves in the deck area image. The hull mask of each frame of the deck area image covers the hull structure in the deck area image where there are no waves. The time series of wave detection results on the deck are merged, and the merged time series is filtered and the number of waves on the ship's deck area is statistically obtained.

[0005] The further technical solution is that the deck wave detection model includes a template mask generation module, a real-time mask generation module, and a deck wave detection module that are cascaded in sequence. After the deck area video is input into the deck wave detection model, the template mask generation module generates the hull mask of the first frame of the deck area image in the video as the template mask. The real-time mask generation module sequentially generates the hull mask of each subsequent frame of the deck area image as the real-time mask for each subsequent frame of the deck area image. The deck wave detection module extracts the contour features of the template mask and each real-time mask, as well as the semantic features of each frame of the deck area image. It calculates the difference in contour features between each real-time mask and the template mask, and combines the semantic features of each frame of the deck area image to determine the deck wave detection result for each frame of the deck area image.

[0006] A further technical solution is that the contour features include the ship's hull contour, and the deck wave detection module determines the deck wave detection results for any frame of the deck area image, including: The ship's hull outline is extracted from the template mask and the real-time mask of the deck area image using an edge detection algorithm, and the outline similarity between the template mask and the real-time mask is calculated. When the contour similarity is greater than the first threshold, the deck wave detection result of the deck area image is determined to be an on-deck wave; when the contour similarity is less than the second threshold, the deck wave detection result of the deck area image is determined to be a non-on-deck wave; when the contour similarity is greater than the second threshold and less than the first threshold, the semantic features of the deck area image are obtained, and the deck wave detection result of the deck area image is determined based on the semantic features combined with the real-time mask.

[0007] The further technical solution is that the semantic features include wave semantic features. The Grounding DINO model is used to obtain the wave target box corresponding to the wave semantic features of the deck area image. The wave target box is the smallest bounding rectangle of the area where the waves are located in the deck area image. Based on semantic features and real-time masking, the deck wave detection results in the deck area image include: Calculate the overlap area between the wave target bounding box and the real-time mask. When the overlap area exceeds the area threshold, determine that the wave detection result on the deck area image is a wave on the deck; otherwise, determine that the wave detection result on the deck area image is not a wave on the deck.

[0008] A further technical solution involves merging the time series of wave detection results on the deck, including: Extract all wave intervals in the time series where the wave detection results on the deck are consecutive waves, and merge any two adjacent wave intervals with an interval not exceeding the threshold max_gap into the same wave interval; the interval between two adjacent wave intervals is the number of frames between the first frame of the deck area image of the previous wave interval and the first frame of the deck area image of the next wave interval.

[0009] A further technical solution involves obtaining the number of waves hitting the ship's deck area, including: All wave-climbing intervals with an interval length less than the interval threshold min_region_len in the merged time series are filtered out, and the total number of wave-climbing intervals remaining is counted to obtain the number of wave-climbing intervals in the ship deck area; the interval length of the wave-climbing interval is the number of consecutive frames from the first frame of the deck area image to the last frame of the deck area image in the wave-climbing interval.

[0010] The further technical solution is to determine the threshold max_gap=fps based on the frame rate (fps) of the typical deck area video; and to determine the interval threshold based on the average interval length (avg_region_len) of all wave-climbing intervals in the typical deck area video. .

[0011] The further technical solution is that the template mask generation module uses the SAM model to segment the hull region of the first frame of the deck area image in the deck area video to obtain the hull mask of the first frame of the deck area image.

[0012] The further technical solution is that the real-time mask generation module uses the XMem model to perform target segmentation on each frame of the deck area image after the first frame in the deck area video, and obtains the real-time mask of each frame of the deck area image. The segmentation target is the hull area that does not contain waves.

[0013] The beneficial technical effects of this application are: This application discloses an artificial intelligence-based statistical method for detecting and detecting waves on ship decks. It performs real-time analysis of each frame of a deck area video, constructs template masks for frames without waves and real-time masks for each frame, and detects wave occurrences based on the difference between the real-time mask and the template mask. An improved neighborhood merging strategy is then used to statistically analyze the wave detection results, quantifying wave-related events on the deck. This method enables real-time perception and quantitative assessment of wave-related events on ship decks during actual navigation, thereby achieving risk warning and providing crucial support for ensuring ship and seafarer safety and subsequent ship design optimization.

[0014] Considering that waves cause significant changes to the deck area's contour, wave-related phenomena are identified by analyzing the contour features of each frame of the deck area image in the video. Furthermore, considering subtle contour differences caused by ship vibration or other non-wave-related factors, the semantic features of each frame of the deck area image in the video are analyzed to further determine whether waves have occurred, effectively improving the accuracy of wave detection results. By combining coarse-grained and fine-grained judgment mechanisms, a comprehensive analysis from global geometric characteristics to local visual semantics is achieved, enabling robust detection of wave-related phenomena in various complex scenarios.

[0015] By performing neighborhood merging on the binary time series signal of wave detection results, false interruptions caused by brief fluctuations or speed changes can be effectively merged. Simultaneously, denoising filtering of the merged time series can remove brief false signals generated by accidental disturbances. The neighborhood merging strategy designed in this application can effectively solve the problem of "false interruptions" or "fragmented wave events" caused by short-term fluctuations in the detection results, providing more robust and physically reasonable wave event markers under different sea states and speed conditions. Attached Figure Description

[0016] Figure 1 This is a flowchart of the statistical method for detecting waves on ship decks.

[0017] Figure 2 This is a structural diagram of the statistical method for detecting waves on ship decks.

[0018] Figure 3 This is a UI interface for obtaining a template mask from SAM model segmentation in an instance.

[0019] Figure 4 It is a real-time mask obtained from the segmentation of the XMem model in an instance.

[0020] Figure 5 It is the difference in contour features between the template mask and the real-time mask.

[0021] Figure 6 It is the target bounding box of the waves corresponding to the semantic features of the waves in the deck area image.

[0022] Figure 7 This is a structural diagram of a wave detection system on a ship's deck, as shown in the example.

[0023] Figure 8 This is a schematic diagram of a hybrid camera layout in an example.

[0024] Figure 9 This is a schematic diagram of a hybrid camera layout in another example. Detailed Implementation

[0025] The specific embodiments of this application will be further described below with reference to the accompanying drawings.

[0026] This application discloses an artificial intelligence-based method for detecting and statistically analyzing waves on ship decks. Please refer to [the relevant documentation / reference]. Figure 1 The flowchart shown illustrates the specific steps of this method as follows: Step 1: Obtain a video of the ship's deck area at the current moment. The deck area video includes images of the deck area of ​​the ship at the current moment and within a predetermined continuous time period T prior to that moment.

[0027] The maritime environment is complex and ever-changing, and sudden wave impacts on ships are frequent occurrences. To ensure the safety of ships and crew, real-time monitoring of wave impacts on the ship's deck area is necessary. This requires real-time acquisition of video data of the ship's deck area, including multiple frames of deck area images from the current and historical moments. The specific video duration T to be acquired should be set according to the actual application and must be consistent with the input video duration of the deck wave detection model. Furthermore, to facilitate subsequent adaptation to the deck wave detection model, the first frame of the deck area video should not show any waves on the ship's hull structure.

[0028] Step 2: Input the deck area video into the deck wave detection model, and perform detection on each frame of the deck area image to obtain the time series of deck wave detection results.

[0029] The deck wave detection model is used to generate the hull mask of each frame of the deck area image in the deck area video, and to determine the deck wave detection result of each frame of the deck area image based on the contour features of the hull mask of each frame of the deck area image and the semantic features of each frame of the deck area image. The deck wave detection result indicates the presence of waves in the deck area image; the time series is the deck wave detection result of each frame of the deck area image in the order of time; the hull mask of each frame of the deck area image covers the hull structure in the deck area image where there are no waves.

[0030] During actual ship navigation, the deck wave detection model has been deployed into the ship's deck wave detection system. During actual monitoring, the deck area video is input into the deck wave detection system to obtain the deck wave detection results. Specifically, the deck wave detection model includes a template mask generation module, a real-time mask generation module, and a deck wave detection module that are cascaded in sequence.

[0031] The process of wave detection using a deck wave detection model is as follows: Figure 2 As shown, after the deck area video is input into the deck wave detection model, the template mask generation module generates the hull mask of the first frame of the deck area image in the deck area video as the template mask. The real-time mask generation module sequentially generates the hull masks of each subsequent frame of the deck area image as the real-time masks for each subsequent frame of the deck area image. The deck wave detection module extracts the contour features of the template mask and each real-time mask, as well as the semantic features of each frame of the deck area image, calculates the difference in contour features between each real-time mask and the template mask, and combines the semantic features of each frame of the deck area image to determine the deck wave detection result for each frame of the deck area image. Since the template mask needs to provide a reference for the subsequent real-time masks, it should be noted that the first frame of the deck area image in the deck area video needs to be a wave-free frame where there are no waves on the hull structure. The first frame of the deck area image is usually a deck area image collected when the ship is stationary and has not started. At this time, there are no waves on the ship's deck. In this case, the deck area video starts before the ship starts, and the starting time of the deck area video is a certain historical moment before the ship starts. If the deck area video does not start before the ship starts, then it is necessary to ensure that the first frame of the deck area video is a pre-wave frame, that is, the deck area video is captured from a moment when no wave has occurred.

[0032] The core of this application's deck wave detection method lies in determining the wave phenomenon by analyzing the difference between the hull mask of a wave-affected frame and a wave-free frame. Waves typically cause the deck to be submerged, resulting in a significant difference between the hull mask of that frame and the mask of a normal frame. Based on this characteristic, it is possible to effectively distinguish whether or not waves have occurred.

[0033] Since the template mask is the basis for wave detection in subsequent real-time masks, accurate segmentation of the template mask is crucial for the detection results. This application investigated efficient hull mask segmentation models. Considering that the SAM segmentation model is a general image segmentation model proposed by Meta AI in 2023, which can segment any object in an image without fine-tuning for a specific task and supports different forms of "hint" input, such as points, boxes, and masks, and has been tested and verified for its feasibility in hull segmentation, the template mask generation module uses the SAM model to segment the hull region of the first frame of the deck area image in the deck area video to obtain the hull mask of the first frame of the deck area image.

[0034] This application uses the open-source SAM model to extract the template mask, and to intuitively analyze the generation effect of the template mask, a system is built as follows: Figure 3 The interactive user interface (UI) shown is as follows. Users only need to upload video of the deck area where waves are to be detected and counted, and the system will automatically extract the first frame of the video (displayed on the screen). Figure 3 (Top left corner). Then, the user clicks on a specified hull area (green indicates the forward region, red indicates the reverse region) to guide the SAM model in segmenting the target object. The SAM model outputs the hull mask for the first frame. The user can interact with the system multiple times, adjusting the click position until a satisfactory template mask is generated (displayed in the top left corner). Figure 3 (on the right side).

[0035] Next, it is necessary to obtain the hull mask for each frame of the deck area image after the first frame in the deck area video. Considering that hull mask segmentation needs to handle the dynamic changes of the hull in consecutive video frames (such as shaking, deck submersion caused by waves), the open-source video target segmentation model XMem (X-Memory) can effectively track the consistency of the hull in time-series frames. Based on the template mask of the first frame, it continuously tracks the position and shape changes of the hull in consecutive video frames, adapting to dynamic scene interference such as hull shaking, and avoiding segmentation failure caused by inter-frame changes. Moreover, wave detection relies on the "difference between the template mask and the real-time mask of the current frame". The XMem model can stably output the hull mask for each frame, ensuring the accuracy of the difference analysis. Based on this, the real-time mask generation module uses the XMem model to perform target segmentation on each frame of the deck area image after the first frame in the deck area video, obtaining the real-time mask of each frame of the deck area image. The segmentation target is the hull area that does not contain waves.

[0036] The task of video object segmentation is to predict the mask of the target in the current frame based on the template mask specified in the first frame and the image of the current frame. This method effectively generates the hull mask for all subsequent frames. Figure 4The image shows the segmentation results of frames 10, 200, and 280 in the deck area video. The results demonstrate that wave ingress significantly alters or disrupts the integrity and shape of the hull mask. This mask difference provides a clear criterion for detecting wave ingress. Therefore, by utilizing the difference between the real-time mask generated for each frame and the template mask of frames without wave ingress, the presence of wave ingress in the current frame can be intelligently determined, laying the foundation for subsequent statistical analysis.

[0037] The deck wave detection module extracts the contour features of the template mask and each real-time mask, as well as the semantic features of each frame of the deck area image. It calculates the difference in contour features between each real-time mask and the template mask, and combines this with the semantic features of each frame of the deck area image to determine the deck wave detection result for each frame. In one embodiment, the contour features include the hull contour. The deck wave detection module determines the deck wave detection result for any frame of the deck area image as follows: Because contour differences are more robust than other measurement methods (such as area) and less susceptible to misjudgments due to issues like hull vibration, the comparison between the template mask and the current real-time mask is achieved by extracting the hull contour lines and calculating their differences. These differences primarily reflect the actual geometric changes in the hull contour caused by wave impact, thus providing a reliable basis for coarse-grained wave impact judgment. This effectively distinguishes between wave-impacted and non-wave-impacted frames, as the contour changes in wave-impacted frames are typically significantly greater than those in non-wave-impacted frames.

[0038] Edge detection algorithms are used to extract the ship's hull outline from the template mask and the real-time mask of the deck area image, respectively, and the outline similarity between the template mask and the real-time mask is calculated. Figure 5 As shown, the green outline represents the hull outline of the template mask, and the red outline represents the hull outline of the current real-time mask. Figure 5 (a) is Figure 4 The hull outline in the 10th frame of the video showing the deck area, as shown by a real-time mask. Figure 5 (b) is Figure 4 The hull outline in the 200th frame of the video of the deck area, shown in a real-time mask.

[0039] The specific edge detection algorithm and contour similarity calculation method can adopt existing techniques. Considering the characteristic that Hu moments are insensitive to translation, rotation and scaling, this application calculates the Hu moments of the template mask and the real-time mask respectively, and uses the difference between the Hu moments of the template mask and the real-time mask as the contour similarity. The larger the difference in Hu moments, the greater the contour similarity and the greater the contour difference between the two masks. Figure 5 The contour similarity between the two masks in (a) is 0.003. Figure 5(b) The contour similarity of the two masks is 0.17, corresponding to the 10th and 200th frames of the deck area video. The wave phenomenon is more severe in the 200th frame of the deck area image, and there are more waves on the hull structure.

[0040] When the contour similarity is greater than the first threshold, the deck wave detection result of the deck area image is determined to be an on-deck wave; when the contour similarity is less than the second threshold, the deck wave detection result of the deck area image is determined to be a non-on-deck wave; when the contour similarity is greater than the second threshold and less than the first threshold, the semantic features of the deck area image are obtained, and the deck wave detection result of the deck area image is determined based on the semantic features combined with the real-time mask.

[0041] To achieve robust wave detection, two thresholds are set: a maximum threshold and a minimum threshold. The first threshold is the maximum threshold, and the second threshold is the minimum threshold. Frames exceeding the maximum threshold are directly identified as wave-indicating frames, while frames below the minimum threshold are identified as non-wave-indicating frames. However, for frames falling between the two thresholds, due to subtle contour differences that may be caused by jitter or other non-wave-indicating factors, further fine-grained judgment is required. The specific values ​​of the first and second thresholds can be set based on practical application and experience.

[0042] Coarse-grained judgment relies on the geometric differences of the hull mask and ignores the visual semantic information of the image. In fine-grained judgment, the detection results from the coarse-grained stage are supplemented to determine the presence of waves from a semantic perspective. The semantic features of the deck area image include wave semantic features. The Grounding DINO model is used to obtain the wave target boxes corresponding to the wave semantic features of the deck area image. The wave target box is the smallest bounding rectangle of the area where the waves are located in the deck area image. The Grounding DINO model is an open-source object detection framework that supports multimodal input and can detect corresponding target boxes in images based on any natural language description. Specifically, we input the current frame into the model and use "whitewave" as a text prompt for detection. The model outputs wave target boxes, such as... Figure 6 As shown.

[0043] Furthermore, determining the deck wave detection result in the deck area image based on semantic features and real-time masking includes: calculating the overlap area between the wave target box and the real-time mask; if the overlap area exceeds an area threshold, the deck wave detection result in the deck area image is determined to be an on-deck wave; otherwise, the deck wave detection result in the deck area image is determined to be a non-on-deck wave. The overlap area between the wave target box and the current real-time mask is used to determine whether a wave appears within the ship's hull area; the area threshold can be set based on historical data and experience.

[0044] By combining coarse-grained and fine-grained judgment mechanisms, a comprehensive analysis from global geometric characteristics to local visual semantics is achieved, enabling robust detection of wave phenomena in various complex scenarios.

[0045] A time series of wave detection results for the deck area video, from the first frame to the current frame, is generated using a deck wave detection model to determine whether the deck is wet or impacted by waves. In the time series of wave detection results, a time series of 0 corresponds to no wave detection, indicating that the deck is not wet, while a time series of 1 corresponds to wave detection, indicating that wave activity has occurred.

[0046] Step 3: Merge the time series of wave detection results on the deck, filter the merged time series, and count the number of waves hitting the ship's deck area.

[0047] Since wave impacts are usually a continuous process, directly counting the number of 1s may lead to misjudgments. Therefore, this application introduces an improved neighborhood merging strategy to merge adjacent markers. For example, the time series of deck wave impact detection results, "000000000000111111100000000", is simplified to 010, thus more concisely counting the number of wave impacts. In practical applications, the time series of deck wave impact detection results may contain erroneous judgments, such as "0000000000001101111110000000", where a single "0" appearing between two sets of "1" sequences is a misjudgment by the model. If these separated 1s are directly counted as two wave impact events, it will affect the accuracy of the statistics. To solve the problem of "false interruptions" or "fragmented wave impact events" caused by short-term fluctuations in the detection results, this scheme introduces an improved neighborhood merging and denoising strategy for binary time series signals (wave impact = 1, non-wave impact = 0).

[0048] In one embodiment, merging the time series of wave detection results on the deck includes: Extracting consecutive wave intervals from the time series data where the deck wave detection results are consecutive waves, i.e., scanning all consecutive intervals of 1 on the time axis, representing the duration of the wave rise process; based on the characteristic that the time interval between two physically adjacent waves is usually short, any two adjacent wave intervals with an interval not exceeding the threshold max_gap are merged into the same wave interval; for example, when the interval between two wave intervals is less than 1 second, this interval is likely caused by sensor noise or a brief water level drop, rather than two truly independent waves. The interval between two adjacent wave intervals is the number of frames between the first frame of the deck area image of the previous wave interval and the first frame of the deck area image of the subsequent wave interval.

[0049] After completing the interval merging, the duration filtering is further performed to obtain the number of waves in the ship deck area. This includes filtering out all wave intervals in the merged time series whose interval length is less than the interval threshold min_region_len to remove transient false signals caused by accidental disturbances, and counting the total number of remaining wave intervals to obtain the number of waves in the ship deck area. The interval length of the wave interval is the number of continuous frames from the first frame of the deck area image to the last frame of the deck area image in the wave interval.

[0050] Through this series of processing steps, smooth, continuous, and physically reasonable wave event markers can be obtained on the time series. In actual video, the observed wave duration and interval are affected by both the wave cycle and the frame rate (fps), so the threshold can be adaptively set by combining the frame rate and historical statistics.

[0051] In one embodiment, the threshold max_gap=fps is determined based on the frame rate fps of typical deck area video; wave segments with an interval of less than 1 second are regarded as the same event. In common real-time video, fps≈23, so max_gap=23 can be directly taken to merge pseudo-interruptions caused by brief fluctuations or changes in speed.

[0052] The minimum duration, `min_region_len`, is related to the inherent timescale of the wave impact. First, reliable wave impact intervals are selected from a typical deck area video, and the average duration of these intervals is calculated. This average duration reflects the typical length of a real event. Typical deck area videos are obtained through analysis of historical video data. Due to complex sea conditions or frame processing errors, the start or end frames of wave impacts may be missed, resulting in a shorter detected duration than the actual value. Therefore, the interval threshold is determined based on the average interval length `avg_region_len` of all wave impact intervals in the typical deck area video. To compensate for potential missed detections of the start and end frames of wave initiation by the algorithm, a threshold of 1 / 3 of the average value is set. This lowers the filtering threshold, ensuring that even if the detection duration of a real event is shortened to 1 / 3 of its original length due to missed detections, the threshold requirement is still met, preventing it from being filtered as a "brief false signal" and ensuring that genuine wave initiation events are preserved. This strategy provides more robust and physically plausible wave initiation event marking under different sea states and speeds.

[0053] The application scenario of the deck wave detection and statistical method of this application is further illustrated by an example. The deck wave detection and statistical method needs to be deployed in a deck wave detection system, such as... Figure 7As shown in the diagram, label 1 represents a single-point panoramic device, label 2 represents a single-point wide-angle camera, label 3 represents a gateway, label 4 represents an image stitching server, and label 5 represents a deck wave monitoring server. The single-point panoramic device is deployed on the roof of the cockpit and typically consists of 12 cameras. Its embedded hardware design uses a multi-layer PCB board, where single-channel camera image acquisition and panoramic image stitching algorithms are implemented. The main control chip is a domestically produced chip. After stitching the images from each camera into a 360° panoramic image, the embedded hardware transmits the video data to the gateway. Several single-point wide-angle cameras are positioned behind the visual blind spots of the single-point panoramic device caused by the upper structure to complete the panoramic image, and transmit the video data to the gateway. The gateway, deployed in the control room, receives high-speed data transmissions from the single-point panoramic device and the single-point wide-angle cameras, converts network protocols and uses a unified interface to aggregate the data into a single transmission line for the image stitching server. The image stitching server, deployed in the control room, receives high-speed video data from the gateway. Using built-in software, it stitches images from various single-point wide-angle cameras onto a 360° panoramic image sent by a single-point panoramic device, completing the deck surface video image. The resulting video stream is then sent in real-time to the deck wave monitoring server. The deck wave monitoring server incorporates software based on the deck wave detection and statistics method described in this application. It monitors each frame of the video stream, identifies the number and size of waves, provides uninterrupted deck wave monitoring services, and interacts with the user.

[0054] Since the deck wave detection and statistics method requires fixed cameras to capture video of the deck area during actual ship navigation as input, this application adopts a hybrid layout of cameras to reduce obstruction and interference from superstructures such as masts and antennas and to obtain clear, comprehensive deck video without blind spots. Specifically, a single-point panoramic device is installed on top of the bridge, and single-point wide-angle cameras are installed in blind spots, ensuring that the overlapping area of ​​the viewing angles of any two cameras is no less than 10%-15% of the image area. Then, image stitching technology is used to obtain a complete video of the ship's deck surface. The single-point panoramic device and the single-point wide-angle camera are connected to an image stitching server via a gateway, transmitting each frame of the acquired deck area image to the image stitching server for image stitching. Specific image stitching methods can refer to existing image stitching algorithms. The stitched deck area image is further transmitted to the deck wave monitoring server, which outputs the current deck wave situation in real time.

[0055] The specific installation locations of the single-point panoramic equipment and the single-point wide-angle camera are related to the ship's structure. Based on common ship geometry analysis, a 10,000-ton cargo ship (110m overall length, 17.5m beam, 7.6m depth) and a 37m self-propelled deck barge (37.2m overall length, 9.6m beam, 2.7m depth) were selected as demonstration installation schemes for large and small ships, respectively. Figure 8 , Figure 9As shown in the figure, number 1 is a single-point panoramic device, and number 2 is a single-point wide-angle camera.

[0056] The above descriptions are merely preferred embodiments of this application, and this application is not limited to the above embodiments. It is understood that other improvements and variations that can be directly derived or conceived by those skilled in the art without departing from the spirit and concept of this application should be considered to be included within the protection scope of this application.

Claims

1. A method for detecting and statistically analyzing waves on a ship's deck based on artificial intelligence, characterized in that, The method for detecting and statistically analyzing waves on ship decks includes: Acquire a video of the ship's deck area at the current moment, the video of the deck area including each frame of the deck area image of the ship at the current moment and within a predetermined continuous time period T before it; The deck area video is input into the deck wave detection model, and the time series of deck wave detection results is obtained by detecting each frame of the deck area image. The deck wave detection model is used to generate the hull mask of each frame of the deck area image in the deck area video, and determines the deck wave detection result of each frame of the deck area image based on the contour features of the hull mask of each frame of the deck area image and the semantic features of each frame of the deck area image. The deck wave detection result indicates the presence of waves in the deck area image. The hull mask of each frame of the deck area image covers the hull structure in the deck area image where there are no waves. The time series of wave detection results on the deck are merged, and the merged time series is filtered and the number of waves on the ship's deck area is statistically obtained.

2. The method for detecting and statistically analyzing waves on a ship's deck according to claim 1, characterized in that, The deck wave detection model includes a template mask generation module, a real-time mask generation module, and a deck wave detection module that are cascaded together. After the deck area video is input into the deck wave detection model, the template mask generation module generates the hull mask of the first frame of the deck area image in the deck area video as the template mask. The real-time mask generation module sequentially generates the hull mask of each subsequent frame of the deck area image as the real-time mask of each subsequent frame of the deck area image. The deck wave detection module extracts the contour features of the template mask and each real-time mask, as well as the semantic features of each frame of the deck area image, calculates the difference in contour features between each real-time mask and the template mask, and combines the semantic features of each frame of the deck area image to determine the deck wave detection result of each frame of the deck area image.

3. The method for detecting and statistically analyzing waves on a ship's deck according to claim 2, characterized in that, The contour features include the ship's hull contour, and the deck wave detection module determines the deck wave detection result for any frame of the deck area image, including: The ship's hull outline is extracted from the template mask and the real-time mask of the deck area image using an edge detection algorithm, and the outline similarity between the template mask and the real-time mask is calculated. When the contour similarity is greater than the first threshold, the deck wave detection result of the deck area image is determined to be an on-deck wave; when the contour similarity is less than the second threshold, the deck wave detection result of the deck area image is determined to be a non-on-deck wave; when the contour similarity is greater than the second threshold and less than the first threshold, the semantic features of the deck area image are obtained, and the deck wave detection result of the deck area image is determined based on the semantic features combined with the real-time mask.

4. The method for detecting and statistically analyzing waves on a ship's deck according to claim 3, characterized in that, The semantic features include wave semantic features. The Grounding DINO model is used to obtain the wave target boxes corresponding to the wave semantic features of the deck area image. The wave target boxes are the smallest bounding rectangles of the area where the waves are located in the deck area image. The deck wave detection results of the deck area image determined based on the semantic features and real-time mask include: Calculate the overlap area between the wave target bounding box and the real-time mask. When the overlap area exceeds the area threshold, determine that the wave detection result of the deck area image is a wave on the deck; otherwise, determine that the wave detection result of the deck area image is a non-wave on the deck.

5. The method for detecting and statistically analyzing waves on a ship's deck according to claim 1, characterized in that, The time series processing of wave detection results on the deck includes: Extract all wave intervals in the time series where the deck wave detection results are consecutive waves, and merge any two adjacent wave intervals with an interval not exceeding the threshold max_gap into the same wave interval; the interval between two adjacent wave intervals is the number of frames between the first frame of the deck area image of the previous wave interval and the first frame of the deck area image of the next wave interval.

6. The method for detecting and statistically analyzing waves on a ship's deck according to claim 5, characterized in that, The number of waves recorded on the ship's deck area includes: All wave-climbing intervals with an interval length less than the interval threshold min_region_len in the merged time series are filtered out, and the total number of wave-climbing intervals remaining is counted to obtain the number of wave-climbing intervals in the ship deck area; the interval length of the wave-climbing interval is the number of consecutive frames from the first frame of the deck area image to the last frame of the deck area image in the wave-climbing interval.

7. The method for detecting and statistically analyzing waves on a ship's deck according to claim 6, characterized in that, The threshold `max_gap` is determined based on the frame rate (fps) of the typical deck area video; the interval threshold is determined based on the average interval length (avg_region_len) of all wave-climbing intervals in the typical deck area video. .

8. The method for detecting and statistically analyzing waves on a ship's deck according to claim 2, characterized in that, The template mask generation module uses the SAM model to segment the hull region of the first frame of the deck region image in the deck region video to obtain the hull mask of the first frame of the deck region image.

9. The method for detecting and statistically analyzing waves on a ship's deck according to claim 2, characterized in that, The real-time mask generation module uses the XMem model to sequentially segment the deck area images in each frame after the first frame in the deck area video to obtain the real-time mask of each frame of the deck area image. The segmentation target is the hull area that does not contain waves.

Images