A method and system for monitoring and predicting the performance of a tug based on deep learning

Abstract

The application relates to the technical field of deep learning, and more specifically to a tugboat monitoring and predicting method and system based on deep learning, which comprises the following steps: step 1: a fixed camera and a towline tension gauge are arranged on a front deck and share a clock; a discrete sample sequence with time slice numbers is formed; step 2: a forward section string is formed by forward filtering and splicing from the earliest anchor point; backward splicing is symmetrically performed from the latest anchor point, a fixed section is formed, and a towing, closing, standby or evading state is labeled; a risk event sequence, a cable direction trajectory sequence and a tension interval sequence are generated; step 3: hierarchical alarms and steering suggestions are generated according to the risk event sequence, the cable direction trajectory sequence and the tension interval sequence according to preset rules. The application improves the safety and prediction accuracy of the tugboat operation, and significantly enhances the intelligence and reliability of the port operation.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of deep learning, and particularly relates to a tugboat monitoring and prediction method and system based on deep learning. BACKGROUND

[0002] In port operations, channel guidance and large ship berthing and unberthing processes, tugboats serve as key auxiliary forces, undertaking tasks such as traction, pushing, adjusting course and maintaining ship stability. The safety and reliability of tugboat operations are directly related to the berthing efficiency of large ships and the safety level of overall port operations. With the continuous increase in ship tonnage, the complexity of drag cable stress and the difficulty of tugboat operation increase significantly. How to monitor and predict the operating state of the tugboat in real time has become a key problem in the research of the shipping industry and port automation.

[0003] In the prior art, the monitoring of tugboat operations usually relies on the following methods:

[0004] Video monitoring: fixed cameras or shipboard cameras are used to capture the drag cable and the operating scene, and manual or computer vision methods are used to analyze the tension state and swing of the drag cable. This method can provide intuitive image information, but is greatly affected by light, weather and obstructions, such as night operations and rain and fog weather, which can distort video images and reduce recognition accuracy.

[0005] Sensor monitoring: common methods include using drag cable tension meters, gyroscopes, GPS and inertial navigation units to record drag cable stress, tugboat attitude and position information in real time. This method is accurate, but the cost is high, and complex data fusion of different sensors is required, which can introduce time delay and errors.

[0006] Automatic identification system (AIS) and radar: AIS or port-based radar is used to monitor the relative position and speed of the tugboat and the ship. This technology is suitable for overall traffic situation analysis, but it is difficult to provide effective support for real-time dynamics and small stress changes of the drag cable at the detailed level. SUMMARY

[0007] The main purpose of the present application is to provide a tugboat monitoring and prediction method and system based on deep learning, which generates anchor points by discretizing video and tension data, performs forward splicing and backward backfilling, and forms a shaped section under the constraints of propagation and bidirectional intersection, thereby realizing the unified output of risk event sequences, cable trajectory sequences and tension interval sequences. The present application not only maintains the stability and explainability of the prediction in complex environments, but also quickly adjusts the rules through a lightweight self-correction mechanism during the result publishing stage, ensuring that the system has adaptability and traceability in long-term operation. Thus, the present application not only improves the safety and prediction accuracy of tugboat operations, but also significantly enhances the intelligence and reliability of port operations.

[0008] To solve the above problems, the technical scheme of the present application is implemented as follows:

[0009] A towing vessel monitoring and prediction method based on deep learning, the method comprising the following steps:

[0010] Step 1: A fixed camera is arranged on the front deck and shares a clock with a towline tension gauge; the acquired video data and tension data are sliced into fixed-length slices, the main line segment and the main outlet point of the towline are extracted, and the deep learning towline segmenter is called back to extract the main line segment and the main outlet point when the extraction fails; the outlet direction, the outlet point area, the swing and the tension are discretized by levels to form a discrete sample sequence with time slice numbers;

[0011] Step 2: The earliest anchor point is used to forward filter and splice to form a forward section string; the latest anchor point is used to perform backward splicing symmetrically, and the side with longer coverage time or more anchor points is taken as the conflict resolution result; constraint propagation is performed on the merged sections, which includes geometric consistency constraint and dynamic continuity constraint; after constraint propagation, the anchor point slices are intersected and written back in both directions to form a shaped section and label its state as towing, closing, standby or avoiding; according to the above, a risk event sequence, a cable trajectory sequence and a tension interval sequence are generated;

[0012] Step 3: According to the risk event sequence, the cable trajectory sequence and the tension interval sequence, hierarchical alarms and maneuvering suggestions are generated according to preset rules.

[0013] Further, in step 1, the video data is divided into continuous time slices according to a fixed time length; the image processing procedure performed for each time slice includes denoising, edge extraction, line segment detection and extraction of the main line segment and the main outlet point of the towline near the outlet port.

[0014] Further, in step 1, when the line segment detection fails, the deep learning towline segmenter is called to generate a towline mask and fall back to the image processing procedure of extracting the main line segment and the main outlet point of the towline near the outlet port; according to the orientation of the main line segment relative to the vertical reference axis of the camera, the outlet direction is discretized into five levels: inner left, left, straight, right and inner right; according to the projection position of the main outlet point on the lower edge of the image, the outlet point area is discretized into left, center and right; the number of times of changing the outlet direction is counted in a window composed of the last several time slices to discretize the swing into small, medium and large; the tension gauge reading is discretized into four levels: low, medium, high and very high through a calibration segmentation table; the discrete level of the outlet direction, the outlet point area, the swing level, the tension level and the time slice number are recorded together to form a discrete sample sequence.

[0015] Further, in step 2, the anchor set is generated from the discrete sample sequence according to the following rules: a traction anchor is generated when the tension level does not decrease and at least once increases in three consecutive time slices; a standby anchor is generated when the swing level remains small in three consecutive time slices and the tension level is low or medium; a convergence anchor is generated when the out-line direction discrete level unidirectionally progresses to the same side in three consecutive time slices.

[0016] Further, in step 2, the process of forward stitching to form a forward section string includes: selecting candidate segments in the future direction from the earliest anchor point and stitching them into a forward section string; the candidate segments meet the following transition rules: the out-line direction discrete level changes at most one grade between adjacent time slices; the out-line point position changes in the order of left, center, and right; the tension level and the swing level change in the same direction in the same time slice; if multiple candidate segments meet the same number of items, the segment with high tension level is preferred; and the process of backward stitching symmetrically includes: selecting candidate segments in the past direction from the latest anchor point and forming a backward section string according to the same transition rules as the forward stitching; when the forward section string and the backward section string are inconsistent in the same time slice, conflict resolution is performed, and the side with longer coverage time is preferred; if the coverage time is the same, the side containing more traction anchors is preferred.

[0017] Further, in step 2, the temporary section after merging the forward section string and the backward section string is processed piece by piece to implement three types of constraints until convergence; the geometric consistency constraint includes: when the out-line point position is left, the out-line direction discrete level is limited to inner left, left, or positive; when the out-line point position is right, the out-line direction discrete level is limited to inner right, right, or positive; the dynamic continuity constraint includes: the tension level of any two adjacent time slices is changed by a single step; when a cross-level change is detected, the intermediate level is backfilled and the time slice is marked as a modified slice; the safety boundary constraint includes: in the case of large swing level and high or extremely high tension level, the out-line direction discrete level of the next time slice converges to the ship center normal direction; for records that have deviated from the ship center normal direction, the record is rolled back to the nearest legal level and the time slice is marked as an avoidance slice.

[0018] Further, in step 2, the process of writing back the anchor slice in both directions, forming the shaping section and assigning the section label includes: reversibly checking the results after constraint propagation on the time axis, if a time slice is marked as a correction slice or an avoidance slice and at the same time triggers the traction anchor condition, then write back the outflow direction discrete level and outflow point area of the slice with the traction anchor as the leading, so that the forward and backward paths are consistent at this slice; after completing the write back, the continuous time slices are merged into shaping sections, and each shaping section is assigned a section label selected from traction, convergence, standby and avoidance.

[0019] Further, in step 2, the three prediction sequences are output in the time order of the shaping sections; the generation method of the risk event sequence is: outputting a lateral risk when the swing level changes from small or medium to large in a short time and the tension level is high or extremely high, outputting a rebound risk when the tension level suddenly drops from high or extremely high to low or medium and the outflow direction discrete level is inward, and outputting an over-tension risk when the tension level remains extremely high for a plurality of continuous time slices; the generation method of the cable trajectory sequence is: forming a two-tuple of the outflow direction discrete level and the outflow point area of each time slice, and arranging these two-tuples in time order; the generation method of the tension interval sequence is: counting the frequency of each tension level in each shaping section, taking the tension level with the highest frequency as the tension interval label of the section, and arranging these labels in the time order of the shaping sections.

[0020] A deep learning-based tug monitoring and prediction system, the system comprising: a data acquisition unit configured to set a fixed camera and a towline tension meter on the front deck and share a clock; slice the acquired video data and tension data, extract the main line segment and the main outflow point of the towline, and call a deep learning towline segmenter to extract when failed; discretize the outflow direction, outflow point area, swing and tension to form a discrete sample sequence with time slice number; a sequence analysis unit configured to filter and splice forward sections by the earliest anchor point in a forward direction; perform backward splicing symmetrically with the latest anchor point, and take the side with longer coverage time or more traction anchors as the conflict resolution result; perform constraint propagation on the merged section, the constraint propagation including geometric consistency constraint and dynamic continuity constraint; write back the anchor slice in both directions after constraint propagation, form shaping sections and label them as traction, convergence, standby or avoidance; generate a risk event sequence, a cable trajectory sequence and a tension interval sequence accordingly; a result output unit configured to output the risk event sequence

[0021] The deep learning-based tugboat monitoring and prediction method and system has the following beneficial effects: the reversible cable direction splicing and constraint propagation algorithm is introduced in the tugboat monitoring and prediction process, so that the prediction result has self-consistency and interpretability in the logical structure, thereby overcoming the defects of the existing deep learning method "black box". In the monitoring link, the image and tension dual-source discretization method is used, which reduces the dependence on complex fusion of multi-source sensors, and greatly improves the feasibility and reproducibility of system deployment. In the prediction link, through the continuous processing chain of anchor point generation, forward splicing, backward filling and constraint propagation, the evolution of the tug cable direction, the wire outlet point position, the swing strength and the tension level can be kept progressive and continuous, and false risk prompts caused by single-point data mutation are avoided, thereby ensuring the stability and reliability of the prediction. At the same time, the invention combines the bidirectional intersection and shaping mechanism to unify the prediction results in different time dimensions, which can output consistent risk event sequences, cable direction trajectories and tension interval sequences, which not only ensures that the results can be used for alarm prompts, but also facilitates the generation of operation suggestions. More importantly, the invention designs a lightweight self-correction mechanism in the result publishing stage. When the actual operation and the prediction are inconsistent, only the gear change range or the back priority in the transfer rule needs to be adjusted to complete the iterative update. This adjustment method is non-invasive, does not change the overall algorithm logic, but can quickly compensate for prediction bias, ensuring the adaptability and self-learning ability of the system in long-term operation. Through the above improvements, the invention can realize high-precision, real-time, interpretable and traceable monitoring and prediction of the tugboat operation process in complex environments, significantly improving the safety of tugboat operation and the intelligent level of port scheduling. BRIEF DESCRIPTION OF DRAWINGS

[0022] Figure 1 A method flowchart of a deep learning-based tugboat monitoring and prediction method is provided for the embodiments of the invention.

[0023] Figure 2 A tug cable swing frequency analysis diagram is provided for the embodiments of the invention.

[0024] Figure 3 A multi-anchor point time sequence analysis diagram is provided for the embodiments of the invention. DETAILED DESCRIPTION

[0025] In order to enable personnel in the technical field to better understand the present application scheme, the technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, not all. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor should be within the scope of protection of the present application.

[0026] REFERENCEFigure 1 A deep learning-based tugboat monitoring and prediction method, the method comprising the following steps:

[0027] Step 1: Set a fixed camera and a towline tension gauge on the foredeck and share a clock; slice the obtained video data and tension data to a fixed length, extract the towline main line segment and the main outlet point, and call the deep learning towline segmenter for back extraction when it fails; discretize the outlet direction, outlet point area, swing and tension by level to form a discrete sample sequence with time slice number;

[0028] The fixed camera is installed on the foredeck of the tugboat near the upper part of the towline outlet, with the lens facing the bow and the field of view covering the towline outlet and the deck area nearby. The installation height of the fixed camera is as low as possible to reduce the obstruction of the deck rail. The camera position is fixed and does not change. The towline tension gauge is installed on the force transmission path of the towline outlet, which can continuously output the instantaneous reading of the towline force. The towline tension gauge uses the same system time source as the fixed camera. The fixed camera and the towline tension gauge are connected to the same controller. The controller only provides unified system time and is responsible for writing the arriving data to the time-stamped buffer, without complex calculation. This can ensure that the picture and reading at the same time belong to the same time slice, fundamentally avoiding the judgment error caused by time drift.

[0029] The controller periodically generates a sampling beat to align the camera frame and the tension reading to the unified beat. The continuous time axis is divided into equal-length time slices. In this embodiment, the length of each time slice is set to 0.2 seconds. The reason for choosing 0.2 seconds is that the change speed of the towline guiding action is usually lower than 5 times per second, and this interval can capture the direction change while filtering the instantaneous jitter. Each time slice is assigned a unique time slice number. The fixed camera triggers frame capture at the start time of the time slice, and the towline tension gauge reads a value at the same time. If an individual time slice is missing an image or reading, the time slice is marked as missing, and the subsequent process uses the results of adjacent time slices for compensation.

[0030] The brightness equalization adopts a block histogram equalization method to enhance the local contrast for shadows and backlight. This can compress the brightness variation of the streamer into a more stable range, avoiding the subsequent edge detection being misled by large dark areas. The denoising process first performs time-domain averaging to reduce the random flicker caused by wave light and small water droplets, and then performs median filtering to remove salt spray spots. Time-domain averaging preferentially suppresses random noise, and median filtering preferentially preserves structural edges. The order is fixed to ensure that the edges are not excessively smoothed after the contrast is enhanced. The region of interest is defined by the following process: based on a one-time teaching image, a polygonal region enveloping the streamer outlet in the fixed camera frame is drawn, and saved as a static mask. Since the fixed camera position does not change, this region does not need to be adjusted in subsequent runs. Defining the region of interest can significantly reduce the search range of the straight line detection and reduce the interference of background structures on streamer recognition.

[0031] The process of extracting the main line segment and the main outlet point specifically includes:

[0032] The preferred geometric method includes:

[0033] (1) Edge extraction: edge extraction based on gradient threshold is performed within the region of interest. This method produces a continuous set of edge points at high-contrast edges, making the streamer stand out in the background.

[0034] (2) Straight line detection: straight line detection based on parameter space mapping is performed on the edge point set to obtain several candidate line segments. This method finds straight line structures by counting the concentration of collinear pixels, and can distinguish the long strip structure of the streamer from the short edges of the background.

[0035] (3) Line segment splicing: when the streamer is slightly curved, a single candidate line segment cannot cover the overall trend of the streamer. In this case, the endpoints of adjacent and nearly pointing candidate line segments are spliced to form a polyline. The basis for splicing is the proximity of the endpoint distance and the continuity of the direction, which can maintain geometric continuity while avoiding the incorrect connection of unrelated short line segments.

[0036] (4) Main line segment determination: among all candidate line segments and polylines, the one that passes through the inside of the region of interest and has the longest length in the lower half of the frame is selected as the main line segment. The reason for selecting the line segment near the camera side and longer is that the streamer projection near the camera has a larger width, clearer texture, and lower probability of being obscured by other structures, which is statistically closer to the real streamer.

[0037] Fallback segmentation method: when the geometric method does not obtain a main line segment that meets the conditions in a certain time slice, a deep learning streamer segmenter is called to perform pixel-level segmentation on the region of interest, outputting a streamer mask image, including:

[0038] (1) Binaryzation and thinning: the mask image is binaryzated and thinned to obtain a binary image, which is used as the input of the geometric method. (2) Region growing: the binary image is input into the region growing algorithm to obtain a region of interest.Figure Two After the value operation, a thinning operation is performed to obtain a skeleton map with single-pixel width. The thinned skeleton can extract the centerline of the streamer from the wideband region, making the subsequent connectivity analysis more stable.

[0039] (2) Connected branch analysis: All connected branches are extracted from the skeleton map, and the length and local width of each branch in the lower half of the image are calculated. The branch that meets the following conditions is selected as the main line segment: the branch passes through the inside of the region of interest; the local width of the original mask corresponding to the branch is larger than that of the same type of branch. Selecting a branch with a larger local width has two advantages: the projection of the streamer in the near field is wider, so it is less likely to be replaced by background fine lines under noise conditions; at the same time, a branch with a larger width has stronger continuity before thinning, and the probability of skeleton breakage is lower.

[0040] The process of determining the main exit point includes: taking the intersection of the main line segment and the lower edge of the image as the main exit point. If there is no intersection, the point closest to the lower edge of the image on the main line segment is selected as the main exit point. Fixing the installation position of the camera relative to the exit port makes the streamer most likely to enter from the lower edge of the image. Using the lower edge as a positioning reference can obtain a more stable starting point and reduce misjudgment caused by background structures.

[0041] The deep learning streamer segmenter is based on the publicly available semantic segmentation paradigm, and selects a full convolutional structure combining an encoder and a decoder. The encoding end uses a pre-trained residual feature extraction network on public data to obtain a more robust representation of low light and complex textures. The decoding end adopts the general approach of upsampling and multi-scale context fusion, which enhances the response to different fine and coarse scale streamer textures through pyramid feature aggregation. Finally, a binary classification output channel is used to distinguish between "streamer pixels" and "non-streamer pixels". The reason for using such a structure is that the streamer in the deck environment may appear as a thin, long, and high-contrast line, or as an intermittent fragment due to reflection and splashes. Multi-scale fusion can cover both thick and thin lines in the same forward inference.

[0042] In terms of training data, video frames covering daytime, nighttime, backlight, rain and fog, water splash shielding, lens water droplets, and deck reflection are continuously collected from fixed cameras, and frames are extracted at time intervals to avoid near neighbor high similarity and ensure scene diversity. Then, pixel-level polygon annotation is performed using a public annotation tool, only the physical towline body is annotated, shadows, reflections and ground marking lines are not annotated, and when encountering shielding areas, the visible boundary is closed, and when encountering serious reflection leading to unclear boundary, the uncertain area is recorded in the annotation description for weak supervision weight processing during training. To improve generalization ability, the training set, validation set and test set are divided according to scene and weather, avoiding images of the same period or the same berthing task appearing in training and validation at the same time, and ensuring evaluation independence. Data augmentation is performed on the training set, including brightness and contrast disturbance, color temperature shift, random cropping to the region of interest, scale scaling, horizontal flipping, motion blur, synthetic rain and fog, and highlight shielding patch, as well as a small probability of random erasing, to approximate the low illumination, shaking, reflection and water splash in actual operation; all augmentations maintain the geometric consistency of the towline annotation to avoid label drift.

[0043] In the training process, the standard process of the deep learning framework is adopted, the batch image and the corresponding mask are loaded, the forward inference, loss calculation and reverse update are performed; the loss is composed of pixel-level cross entropy and region overlap degree class loss to alleviate the class imbalance problem caused by "slim target occupying a small proportion in the picture", and the weight of the pixels near the towline boundary is increased to promote the model to learn clearer segmentation transition at the edge; the ready-made adaptive type is selected for the optimizer, the learning rate is segmented decayed or cosine type slow descent with training rounds, and mixed precision is enabled in training to improve throughput; the segmentation intersection over union and pixel accuracy are evaluated on the validation set every few rounds, if it does not improve for several times in a row, it is stopped early, and the weight with the best performance on the validation set is used as the final model. To facilitate engineering deployment, the trained network is exported to the general open neural network exchange format or the static graph format that can be loaded by the mainstream inference engine, and then the forward inference is accelerated on the edge side graphics processing unit or general processor; if there is a requirement for response time, conventional means such as integer quantization or weight quantization and layer fusion can be used to reduce the delay without significantly losing accuracy.

[0044] The execution process during reasoning is as follows: first, the input frame is cropped in the region of interest, color normalized and scaled, and sent to the segmenter to obtain a probability map of each pixel; the probability map is binarized into a streamer mask by a fixed threshold, and then small connected domain removal, hole filling and opening and closing operations are performed in turn to make the mask continuous and remove isolated noise points; then the mask is thinned to obtain a single-pixel-wide skeleton, the connected component analysis is performed, the length, continuity and local width of each skeleton branch near the lower edge of the image are calculated, and the branch that meets the conditions of "crossing the region of interest", "the local width near the camera side is larger" and "the skeleton breaking times are fewer" is selected as the main branch; the intersection of the main line segment and the lower edge of the image is recorded as the main outflow point, if there is no intersection, the nearest foot point of the main line segment to the lower edge is taken as the main outflow point; at the same time, the segmentation confidence is generated according to the average value of the probability map in the main line segment and its neighborhood, the connected length of the main branch and the mask consistency, and the low confidence does not immediately rewrite the current time slice result, but uses the main line segment and the main outflow point of the last time slice and marks it as to be confirmed in the record to avoid the direction jitter caused by short-time noise being amplified.

[0045] Step 1 uses geometric straight line detection to preferentially determine the main line segment and the main outflow point, when there are not enough straight line candidates in a certain time slice or the straight line detection is judged as untrusted in the time continuity check, the deep learning streamer segmenter is called to execute the above reasoning process, and the obtained main line segment and main outflow point are written into the state record table as the fallback result; if the straight line detection and the segmentation result are available at the same time but there is significant inconsistency, the main line segment of the segmentation result is preferentially used for direction dispersion and outflow point position judgment, and the difference between the two is retained in the form of a marker for subsequent anchor point generation and conflict resolution; in order to ensure continuous robustness, two types of difficult samples are automatically collected during system operation: one type is "geometric method fails but segmentation succeeds" time slice, and the other type is "segmentation confidence is continuously low or inconsistent with manual review" time slice; these samples are sent into the offline retraining process regularly, following the same labeling specification and data augmentation strategy as the initial training, and the deep learning streamer segmenter is fine-tuned and updated; the updated weights are put online in gray mode, the segmentation quality in the verification window and the influence on the downstream anchor points, forward stitching, backward backfilling and constraint propagation are observed, and if negative fluctuations occur, the system is rolled back to the previous version.

[0046] If there are multiple main line segment candidates satisfying the condition in the same time slice, disambiguation is performed according to the following order: prefer the candidate that passes through the geometric center of the region of interest; if there are still multiple candidates, prefer the candidate that has a longer length in the lower half of the picture; if there are still multiple candidates, prefer the candidate that has a smaller change in direction compared to the main line segment of the previous time slice. Such order arrangement can take into account both the geometric reliability of the current frame and the temporal continuity across frames. When there is no main line segment in the previous time slice, ignore the cross-frame constraint and select only according to the geometric reliability of the current time slice. This can avoid forcing outdated information to be propagated to the current frame when there are consecutive missing frames.

[0047] The longitudinal reference axis of the fixed camera is defined as the direction pointing from bottom to top of the bow in the picture. The out-line direction is discretized into five levels: inner left, left, straight, right, inner right, according to the left-right offset of the main line segment's orientation in the image relative to the longitudinal reference axis. The discretization boundary setting method is: divide the possible direction range into five equal intervals, corresponding to the above five levels. The equal interval division approach can maintain consistent resolution under different ship types and different installation angles, so that the subsequent rule judgment does not depend on specific numerical values. When the two end points of the main line segment in the picture have a very small height difference and the direction is not clear, the out-line direction discrete level is set to straight. This can avoid misjudging slight shaking as left-right offset.

[0048] The process of generating the out-line point zone position includes: dividing the picture width into three segments, corresponding to the left, middle, and right three zone positions. The position of the main out-line point in the horizontal direction is mapped to the zone position interval of one-third of the width, that is, the out-line point zone position. The reason for using three equal divisions instead of more segments is that the horizontal position of the cable out-line port relative to the picture is basically stable after installation, and three equal divisions can reduce the zone position shaking caused by slight camera parallax changes while maintaining resolution.

[0049] The process of generating the swing level includes: establishing a sliding window with a length of nine frames, and counting the number of changes in the discrete level of the out-line direction within the window. A change refers to a different discrete level in adjacent time slices. The counting result is discretized into three grades: zero or one times is recorded as small, two or three times is recorded as medium, and four or more times is recorded as large. Nine frames are selected as the window length because this length can cover about one to two typical cable swing periods, and at the same time has obvious smoothing effect on short-time noise. The three-grade division allows the subsequent rules to distinguish between different strengths of swing without relying on specific angle values.

[0050] The process of tension level generation includes: after the fixed camera installation is completed, on-site calibration is performed, static zero point readings, light dragging readings, regular operation readings and high load readings of the streamer tension meter are collected, and four threshold segments corresponding to four levels: low, medium, high and extremely high are established. In operation, the instantaneous reading in each time slice is put into a time window with a length of five frames, the median of the window is taken as the representative value of the time slice, and the threshold segment is mapped to the tension level. Using the median can resist the influence of a single abnormal reading on the level judgment, so that the level reflects the continuous trend rather than the occasional peak. When a certain time slice is missing the tension reading, the tension level of the last time slice is used, and the time slice is marked as backfill in the state record table. The subsequent steps can selectively reduce the weight of the backfill slice in the rule decision.

[0051] The process of state record table generation includes: at the end of each time slice, a record is generated and written into the following fields: time slice number, outgoing line direction discrete level, outgoing point area position, swing level, tension level. Each field is an enumeration item of the above discrete results, and does not contain continuous numerical values. The advantage of discrete recording is that the subsequent rules only need to compare enumeration items without numerical calculation, and the processing process is more stable and reproducible. For example, the state record of a certain time slice can be: time slice number is two hundred and thirteen, outgoing line direction discrete level is left, outgoing point area position is medium, swing level is small, and tension level is medium. The above format can be directly used for storage and transmission.

[0052] If the outgoing point area position is left and the outgoing line direction discrete level is inner right, it means that the main line segment of the current frame is likely to be disturbed by the background edge. At this time, the second ranked line segment in the candidate set is re-determined; if it is still inconsistent, the direction of the main line segment of the last time slice is used and the time slice is marked as to be confirmed. When the tension level is extremely high and the length of the main line segment is abnormally short in the lower half of the picture, it is extremely likely that water spray or fog has blocked the streamer. At this time, the outgoing line direction discrete level and the outgoing point area position of the last time slice are kept unchanged, and the time slice is marked as blocked. This processing can avoid misinterpreting short-term blocking as a sudden change in the direction of the streamer. When a plurality of consecutive time slices are marked as to be confirmed or blocked, the priority of the backtracking segmentation method is raised, so that the segmentation result is given priority to the geometric method until a stable main line segment is obtained.

[0053] In the video preprocessing procedure, a balance step of bright part suppression and dark part enhancement is added to limit the brightness of strong light reflection area while amplifying the dark texture of the streamer. In this way, the sensitivity of edge detection to the streamer texture can be maintained. For fixed position bright spots appearing in continuous multiple frames, a temporal consistency mask is used to suppress the edge response of the position. Fixed bright spots are usually from the mirror water droplets, while the effective edge of the streamer shows a moving feature in time. By using this difference, the interference can be effectively removed. When the end points of the main line segment move between adjacent time slices by a distance more than a certain proportion of the picture width, the threshold of straight line detection is reduced to allow weaker edges to be retained, while the time domain average intensity of the time window is increased to stabilize the direction determination.

[0054] Step 2: forward piece-by-piece screening and splicing are performed from the earliest anchor point, a backward splicing is symmetrically performed from the latest anchor point, and the side with longer coverage time or more anchor points is taken as the conflict resolution result; constraint propagation is performed on the merged segments, including geometric consistency constraints and dynamic continuity constraints; after the constraint propagation, the anchor point pieces are bidirectionally intersected and written back to form the shaped segments and label their states as traction, convergence, standby or avoidance; according to this, a risk event sequence, a cable direction trajectory sequence and a tension interval sequence are generated;

[0055] In specific implementation, for each row containing the time slice number, the wire-out direction discrete level, the wire-out point area, the swing level and the tension level in the discrete sample sequence, a sliding window with a fixed length of 3 is used for traversal, and whether the generation conditions of the traction anchor point, the standby anchor point and the convergence anchor point are met is judged window by window. All judgments are completed by enumeration item comparison, without involving continuous numerical operation. To avoid repeated triggering, once the anchor point is confirmed, the suppression window processing is performed on the time slices covered by the anchor point.

[0056] The discrete sample sequence is arranged in ascending order of time slice number. A window is formed with 3 consecutive time slices, and the first slice, the second slice and the last slice participate in the judgment in turn. Each time an anchor point is generated, the three time slices in the window are immediately suppressed, and the next traversal starts from the first time slice after the suppressed interval. By using a window with a length of 3, single-frame jitter and instantaneous false detection can be filtered, while the response speed to the change of the streamer operation is maintained.

[0057] When the tension level in the window does not show a decrease in the order of the first to the last piece, and at least one increase occurs, it is determined to be a pulling anchor candidate. The constraint that there is no decrease can exclude short-time noise that is "loose first and tight later", and the constraint of increase can capture the trend of sustained force. The anchor point falls on the second piece of the window. The time falls on the middle piece, on the one hand to avoid misjudging the first piece that has not yet formed a trend as an anchor point, and on the other hand to reduce the lag caused by taking the last piece that has entered the stable stage as an anchor point. When there is a large jump in the line direction dispersion level in the window, a pulling anchor point can still be generated, but a "direction unstable" label is recorded in the additional information of the anchor point for subsequent conflict resolution and sorting. The advantage of this is that the pulling behavior is directly based on force changes and should not be masked by lateral swings, but it still retains a reminder of lateral instability.

[0058] When the swing level in the window is small in all three time pieces, and the tension level of the three time pieces is low or medium, it is determined to be a standby anchor candidate. The fact that the swing level is small indicates that the streamer direction is basically stable, and the fact that the tension level is maintained at low or medium indicates that there is no significant force, both of which correspond to the static state of the deck operation. The anchor point falls on the last piece of the window. Selecting the last piece can ensure that the static state has lasted for the minimum duration, reducing the probability of short-term static being mistaken for standby. When a single time piece is missing in the window and the two pieces before and after it meet the conditions, the missing piece is considered to meet the conditions to continue generating anchor points. This can maintain the continuity of the determination in the case of camera obstruction or tension reading loss.

[0059] When the line direction dispersion level in the window is unidirectionally progressive to the same side in the three time pieces, it is determined to be a closing anchor candidate. Unidirectional progression includes one of the following ordered patterns: positive to left to inner left, left to inner left to inner left, positive to right to inner right, right to inner right. Using fixed ordered patterns instead of general "increase" or "decrease" can clearly distinguish actual closing actions from left-right swings. The anchor point falls on the second piece of the window. Selecting the second piece can accurately position the moment when the trend has appeared at the beginning of the sustained closing, rather than before the trend is formed or after it has been completed. When the line point area in the window moves from the middle to the direction of the closing side, a "area evidence" label is recorded in the additional information of the anchor point. The synchronous movement of the area usually accompanies the closing action, and recording this information helps subsequent priority determination and interpretation display.

[0060] If there are multiple anchor candidates in the same window, select them in the order of traction anchor first, approach anchor first, approach anchor first, standby anchor. Force trend can most directly reflect the change of work stage, and placing it in the highest priority can reduce the risk of missing key actions. When there are multiple candidates in the same priority, decide in the following order: prefer approach anchor with "zone evidence" or traction anchor with "direction unstable" label; if there are still multiple, select the one with longer interval between window coverage time and existing anchor to avoid excessive density of anchors. Time slices outside the suppressed window are not affected, and new windows start from the most recent unsuppressed time slice until the end of the sequence.

[0061] Each anchor record contains the following: anchor type, landing time slice number, window first slice time slice number, window last slice time slice number, enumeration item sequence for judgment, additional information label. Write anchors in ascending order of time slice number into the anchor set, and establish a pointer from the anchor to the discrete sample sequence. The purpose of retaining the pointer is that when subsequent review is needed, the corresponding three time slices can be directly reviewed and the enumeration item combination at that time can be restored, avoiding reliance on external interpretation.

[0062] The state record table is arranged in ascending order of time slice number, and each row contains the time slice number, the discrete level of the exit direction, the exit point zone, the swing level, and the tension level. The anchor set contains the anchor type and the landing time slice number.

[0063] The process of forward stitching specifically includes: taking the anchor point with the smallest time slice number from the anchor point set as the starting point. Write the row corresponding to the anchor point landing time slice number to the forward section string, and then stitch in the increasing direction of the time slice number. At each step of stitching, take the next time slice as a candidate segment based on the end time slice of the forward section string. If the time slice is marked as missing or to be confirmed in the state record table, continue to search within the range of the next two time slices to form a set of up to three candidate segments. This processing can cross individual missing and maintain the continuity of the section. Compare the candidate segment with the previously stitched time slice one by one, and only when the following three conditions are met at the same time, the candidate segment is considered to be stitched: the difference between the two is not more than one grade. The adjacent relationship is in turn left and left, left and right, right and right, and right and inner right. The meaning of limiting the difference to one grade is to constrain the gradient of the direction change, so that the trend of the section conforms to the common process of continuously guiding the streamer, rather than jumping across multiple direction grades in a single frame. The outlet point location restriction: it cannot directly jump from left to right or from right to left in a frame. It must transition through the intermediate location. This restriction constrains the change of the outlet position in the camera field of view to continuous movement, avoiding misjudging the temporary relocation caused by local occlusion or reflection as overall migration across the field of view. In the same candidate segment, the tension level cannot rise while the amplitude level falls, or the tension level cannot fall while the amplitude level rises. Such opposite combinations are less common in the mechanical behavior of the streamer, and are mostly due to short-term measurement jitter. Shielding such combinations can make the section closer to the normal state of "easier to produce swing when force is applied, and easier to converge when force is collected".

[0064] When there are multiple candidate segments that meet the above three restrictions at the same time, the candidate segment with higher tension level is selected to write to the forward section string. There are two benefits to this preference for high tension: first, high tension usually corresponds to substantial pulling or pushing action, and stitching along consecutive points of high tension can better capture the main line of the operation; second, points of high tension are less affected by incidental noise, and the stability of the section is better. If the tension levels of multiple candidate segments are the same, the candidate segment with the closest time distance is selected to reduce the uncertainty caused by the stitching span. If there is no candidate segment that meets the three restrictions within the range of the preferred time slice and the next two time slices, the forward stitching is ended at the current anchor point. The subsequent stitching will be restarted from the next anchor point. This termination strategy can naturally divide discontinuous abnormal intervals and avoid forcing unreliable states into the section.

[0065] The backward backfilling process specifically includes: taking the anchor point with the largest time slice number from the anchor point set as the starting point. Writing the row corresponding to the anchor point landing time slice number to the backward section string, and then backfilling in the decreasing direction of the time slice number. At each backfilling step, taking the previous time slice as the candidate slice based on the end time slice of the backward section string; if the time slice is missing or to be confirmed, continue to search within the previous two time slices to form a set of up to three candidate slices. The three restrictions are the same as the forward splicing. The same restrictions ensure that the sections in both directions remain consistent in behavior constraints, facilitating subsequent alignment. When multiple candidate slices simultaneously satisfy the three restrictions, the one with a higher tension level is preferentially selected; if the tension levels are the same, the one with the closest time distance is selected. If there is no candidate slice that satisfies the three restrictions within the preferential time slice and the previous two time slices, the backward backfilling ends at the current anchor point.

[0066] The forward and backward alignment and conflict resolution process specifically includes: mapping the forward section string and the backward section string to the same time axis. Find the overlapping time slice range as the range that needs to be resolved. The non-overlapping part is directly retained.

[0067] In the overlapping range, compare the four enumerated fields: outgoing direction discrete level, outgoing point area, swing level, and tension level. As long as any field is different, it is considered that the time slice state is inconsistent. For a time slice with inconsistency, calculate the continuous coverage length of the time slice in the forward section string and the continuous coverage length in the backward section string, respectively. Continuous coverage length refers to the number of time slices from the respective starting anchor point to the time slice, along the respective direction without interruption. The side with longer coverage length is preferentially retained. Longer coverage usually means that the path has crossed more adjacent small changes, indicating that the side has maintained a self-consistent evolution trajectory for a longer time, and has higher reliability.

[0068] When the coverage lengths are the same and there is still inconsistency, count the number of traction anchors contained between the starting point and the time slice on both sides, respectively. The side with more traction anchors is preferentially retained. Traction anchors are directly derived from the continuous rise of tension level, and have high value for identifying operation stages. The side with more traction anchors is usually more consistent with the real stress process. After making a choice, write the four enumerated fields of the retained side back to the merged result, and mark the corresponding time slice of the other side as covered. After completing all the resolution in the overlapping range, merge the forward section string and the backward section string into a time-continuous merged section sequence, which is used for subsequent constraint propagation and section finalization.

[0069] The out-line direction discrete level limit is changed within adjacent gears, which can limit the evolution of the streamer direction to a continuous and interpretable path. The force and guidance of the streamer change gradually in physics, and forcibly allowing a jump across multiple gears will mix accidental noise into the path, leading to unstable subsequent section labels. Prioritizing high tension when there are parallel candidates can make the section extend along a more "forceful" trajectory. Fragments with high tension are not easily disturbed by visual factors such as water surface reflection and local occlusion, and thus have more support for subsequent closing, pulling and avoiding decisions.

[0070] The coverage length represents the continuity of evidence from the anchor point. The stronger the continuity, the less the influence of accidental fluctuations and short-term missing. Conflict resolution with coverage length as the first priority can obtain a merged result close to the real operation trajectory without relying on complex numerical models. The pulling anchor point is triggered by the continuous non-decrease and increase of the tension level, and has stable operation semantics. Taking it as the deciding factor when the coverage lengths are equal can select a path that is more consistent with the actual pulling process among multiple feasible paths, thereby improving the interpretability and review value of the merged section.

[0071] The temporary section is composed of records arranged in ascending order of time slice number, and each record contains time slice number, out-line direction discrete level, out-line point location, swing level, tension level, and marked items generated during propagation. According to the order of geometric consistency constraint, dynamic continuity constraint and safety boundary constraint, the temporary section is traversed from the earliest time slice to the latest time slice. If any field is modified in a round of traversal, the next round of traversal is continued; if no modification occurs in the whole round of traversal, it is determined to be converged. To prevent repeated back and forth on abnormal data, a maximum of 10 rounds of traversal are performed; if it still does not converge after 10 rounds, the 10th round result is taken as the output and a prompt mark of "propagation not fully converged" is written in the section header.

[0072] The geometric consistency constraint (pairing correction of the out-line point area and the out-line direction discrete level) specifically includes: regarding the out-line direction discrete level as inside left, left, positive, right, and inside right in sequence from left to right. For each time slice, the allowed out-line direction discrete level set is limited according to the current out-line point area, and the nearest backtracking is performed for illegal cases: when the out-line point area is left, the allowed set is inside left, left, and positive; if the current is right or inside right, it is directly changed to positive, and a “geometric backtracking” mark is added to the time slice. When the out-line point area is right, the allowed set is inside right, right, and positive; if the current is left or inside left, it is directly changed to positive, and a “geometric backtracking” mark is added to the time slice. When the out-line point area is middle, the allowed set is positive; if the current is inside left, left, right, or inside right, it is changed to positive, and a “geometric backtracking” mark is added to the time slice. The nearest backtracking described above directly compresses the cross-side pointing to the center axis direction. The pointing close to the center axis direction is more stable in the presence of occlusion and reflection, and can reduce the cross-side misjudgment caused by transient visual deviation.

[0073] The power continuity constraint (the tension level of adjacent time slices cannot be stepped up) specifically includes: for each pair of adjacent time slices, it is checked whether the tension level jumps up from the previous slice to the current slice, and a single-step backfill correction is performed on the current slice, while adding a “correction slice” mark: when the previous slice is low and the current slice is high or extremely high, the current slice is changed to medium. When the previous slice is medium and the current slice is extremely high, the current slice is changed to high. When the previous slice is extremely high and the current slice is medium or low, the current slice is changed to high. When the previous slice is high and the current slice is low, the current slice is changed to medium. The single-step backfill compresses the sudden jump that does not meet the force gradient to the adjacent level, which helps to match the continuous force of the real traction process and avoid treating the accidental peak as a stage change.

[0074] The safety boundary constraint (prohibiting moving away from the ship body normal side when the high force has a large swing) specifically includes: when the swing level of a certain time slice is large and the tension level is high or extremely high, the direction of the next time slice after the time slice is corrected, while adding a “avoidance slice” mark. To determine “moving away from the ship body normal side”, the specific judgment and backtracking are given according to the out-line point area of the current time slice: when the out-line point area is left, the next time slice is changed to positive if it is right or inside right; if it is already inside left, left, or positive, it remains unchanged. When the out-line point area is right, the next time slice is changed to positive if it is left or inside left; if it is already inside right, right, or positive, it remains unchanged. When the out-line point area is middle, the next time slice is changed to positive if it is inside left, left, right, or inside right; if it is already positive, it remains unchanged. Under the combination of high force and large swing, the pointing of the next time slice is forced to return to the center axis side, which can reduce the risk of further deviation to the outside, causing the tow cable to sweep the side or hang on the component, and make the trajectory return to the safer in-band first.

[0075] The process of two-way intersection and finalization (reversible check, write-back consistency, section label generation) includes: in the propagation of the converged result, check from front to back piece by piece. If a time slice is marked as a correction slice or an avoidance slice, and the 3-slice window centered on the slice meets the traction anchor point determination condition (the tension level does not decrease and at least once increases within the window), the write-back consistency process is triggered. This setting makes the force trend have higher dominance in the unstable data segment. For a slice that triggers write-back, determine the new out-line direction discrete level and out-line point area in the following order, and cover the original value of the slice, while adding a "traction write-back" label: out-line direction discrete level write-back: prefer to use the out-line direction discrete level of the previous time slice, provided that the level is within the allowed set of the out-line point area of this slice. If not, use the out-line direction discrete level of the next time slice, provided that the level is within the allowed set of the out-line point area of this slice. If neither the previous nor the next meets the condition, set the out-line direction discrete level of this slice to positive. The above order first ensures consistency with adjacent evolution; when adjacent information is not available or not legal, the central axis direction can obtain a more robust compromise.

[0076] Count the majority of the out-line point areas in the 3-slice window centered on the slice; if there is a majority, use the majority as the write-back value. If there is no majority, determine the area according to the out-line direction discrete level after write-back: set to left when it is inner left or left, set to right when it is inner right or right, and set to center when it is positive. The majority can prefer to retain the stable spatial position within the short time window; when the spatial position distribution is scattered, the area is determined according to the current pointing side, which can make the area and the pointing consistent on the same side, reducing the transverse self-contradiction.

[0077] After completing all write-backs, traverse from the earliest time slice backward, and merge adjacent time slices into the same section when one of the following conditions is met: the four enumerated fields are exactly the same. Only the out-line direction discrete level changes by one adjacent gear, and the other three fields are consistent. When none of the above conditions is met, start a new section from the current time slice. After the traversal is completed, a number of finalized sections are obtained. For each finalized section, determine a unique section label in the following priority order: traction: the section contains at least one traction anchor point, or the section has the most occurrences of tension level at high or extremely high and the most occurrences of swing level at medium or large. Approach: the section has the most occurrences of ordered progression of out-line direction discrete level to the inside, and the number of occurrences of tension level at medium or high is not less than the number of occurrences of out-line direction alternating. Standby: the section has the most occurrences of swing level at small, and the most occurrences of tension level at low or medium. Avoidance: the section contains a time slice marked as an avoidance slice, and none of the above three conditions is triggered. The purpose of using this priority order is to cover the force-dominated phase first, then the direction-dominated approach process, followed by the identification of static stable state, and finally the residual unsafe rollback behavior is classified as avoidance, ensuring that the labels are mutually exclusive and easy to review.

[0078] After the shaping section is generated, the first and last time slices of each segment are compared with the first and last time slices of its adjacent segments. If a breakpoint appears that is legal within the segment but illegal at the adjacent place between segments (for example, the left segment tail is left and positive, the right segment head is right and positive, and there is no transition in between), a transition time slice is inserted at the breakpoint, the out-line direction discrete level of the transition time slice is set to positive, the out-line point position is set to middle, the swing level and tension level are kept as the values of the left segment tail, and a "transition insertion" label is added. This approach pulls the unattainable cross-side jump into an interpretable transition form, which is convenient for subsequent review and display. Based on this, the constraint propagation presses the segments that do not conform to the geometric and dynamic rules back into the stable band; the bidirectional intersection and shaping unify the selection of the forward and backward directions while preserving the traction trend, forming a clear boundary and explicit label shaped segment, providing a determined and traceable input for the subsequent joint prediction generation.

[0079] The shaped segments are arranged in chronological order, each shaped segment is composed of several time slices, and each time slice contains the out-line direction discrete level, the out-line point position, the swing level, and the tension level. All the shaped segments are concatenated into a single time axis in ascending order of the start time slice number. The time axis is scanned time slice by time slice, and a fixed-length window is used for local statistics and change detection when necessary. The window is only used to suppress transient jitter and does not change the original discrete value.

[0080] The process of generating the risk event sequence includes: using a window with a length of 5 for all risk judgments, and the window covers the current time slice and the two time slices before and after it. The length of 5 is selected because this length has a significant inhibitory effect on single-frame noise while maintaining sensitivity to motion within about 1 second. To avoid repeated triggering at adjacent time slices, once a similar risk is output, the inhibition of the similar risk is enabled for the next 5 time slices. When there is a trend of swing level from small or medium to large in the window, and the tension level is high or extremely high in the current time slice or the window, a lateral risk is output at the current time slice. The reason for requiring the swing level to be small or medium first and then rising to large is to confirm that a transition from stable to strong swing has occurred; at the same time, a high or extremely high tension level is required because strong swing is more likely to cause a sweep or a hard collision when the force is large. The judgment falls on the current time slice, which can prompt immediately when an unfavorable combination occurs, reducing the lag.

[0081] When the tension level of the previous time slice is high and the current time slice is low, or the previous time slice is very high and the current time slice is medium or low, and the wire-out direction dispersion level of the current time slice is inner left or inner right, the current time slice outputs a rebound risk. The sudden drop of stress from high to low or from very high to medium and below usually means that the pulling force is quickly released; at this time, the towline points to the inner side closer to the ship structure, and the sweep caused by the rebound is more likely to touch the sensitive parts, so this combination is judged as a rebound risk. When the tension level remains very high for 5 consecutive time slices, the middle time slice of the consecutive interval outputs an over-tension risk. A threshold of 5 consecutive time slices is used to exclude short-term peaks and identify the situation of continuous over-limit. Positioning the event in the middle time slice makes it easy to accurately capture the representative position of the continuous phase when reviewing. For each output risk event, record the event type, the positioning time slice number, the window coverage used for judgment, the enumeration sequence of the swing level and the tension level in the window, and the values of the wire-out direction dispersion level and the wire-out point position at the positioning moment. Complete records facilitate subsequent review and interpretation.

[0082] The process of generating the cable direction trajectory sequence includes: reading the wire-out direction dispersion level and the wire-out point position from the earliest time slice to the latest time slice on a single time axis, forming a two-tuple in time sequence and writing it into the cable direction trajectory index in turn. The two-tuple directly reflects the joint change of "direction and position", saving intermediate coordinate conversion and having higher reproducibility. When two adjacent positioning segments have a cross-side change from left to right or from right to left at the boundary without a transition time slice in between, a transition two-tuple is inserted first, with the direction being positive and the position being medium, and then the subsequent two-tuples are written. Inserting a transition value can restore the unattainable cross-side jump to an interpretable coherent transition, facilitating subsequent visualization and downstream rule reference. When the two-tuples in the continuous 3 time slices repeatedly switch between two adjacent directions while the wire-out point position remains unchanged, replace the direction of the middle time slice with the side that appears more frequently. This way, the direction jitter caused by weak texture or water splashes can be suppressed without changing the true position advance and retreat.

[0083] The process of generating the tension interval sequence includes: for each profiling section, counting the occurrence times of each tension level in the section, and taking the tension level with the most occurrence times as the tension interval label of the section. By using the item with the most occurrence times, the long-term stress state of the section can be expressed with a single label, which facilitates quick understanding of the operation and maintenance personnel. When two or more tension levels have the same occurrence times and are the most, the tension level with higher risk is selected as the label. For example, high and medium are taken together, and high is taken; extreme high and high are taken together, and extreme high is taken. In this way, when there is uncertainty, the safety margin can be tilted to the conservative direction, reducing the risk of missing reports. If the first and last time slices of the profiling section are labeled as correction slices, and their tension levels are inconsistent with most values in the section, they are not counted in the statistics. This avoids the bias of single frame backfilling on the whole section label, making the label closer to the stable state in the section.

[0084] The risk events are arranged in ascending order of time slice number to form a risk event sequence; all the two-tuples are arranged in time sequence to form a cable direction trajectory sequence; and the tension interval labels of each profiling section are arranged in the order of the profiling sections to form a tension interval sequence. The three sequences share the same time baseline, and can be compared piece by piece during playback. A light check is performed before output: when a time slice is labeled as an avoidance slice and the cable direction trajectory index shows two-tuples continuously away from the center axis, and there is no lateral risk event near the time slice, a lateral risk event is supplemented and the source is labeled as consistency check. In this way, the missing reports caused by the suppression window can be made up, and the mutual verification of risk prompt and trajectory performance can be maintained. The three sequences are written into structured records respectively, and a pointer to the original time slice is saved for each risk event and each profiling section. When it needs to be reviewed, the engineering personnel can directly locate to the discrete field combination at that time, and quickly check the trigger basis.

[0085] Step 3: generating graded alarms and manipulation suggestions according to the risk event sequence, the cable direction trajectory sequence and the tension interval sequence according to the preset rules.

[0086] Specifically, the generation of the hierarchical alarm specifically includes: setting three levels of alarms: prompt, warning, and emergency. The three levels are divided according to the type, persistence and superposition of the risk event, including: (1) Lateral risk: single trigger is defined as warning; 2 times or more in a window of length 5, upgrade to emergency. (2) Rebound risk: single trigger is defined as warning; when it appears immediately after the lateral risk in a window of length 5, it is upgraded to emergency. (3) Overstretch risk: once triggered, it is directly defined as emergency. By using the above hierarchical approach, "occasional and observable" and "persistent and must intervene" can be distinguished, unnecessary high-level prompts can be reduced, and stronger prompts can be given to truly dangerous combinations. Scan the risk event sequence from the front to the back along a single time axis. For the determined alarm, open a suppression window of length 5, and do not repeat the same type and level of alarm within the window. The suppression window can effectively avoid the same event being prompted multiple times in a short period of time, so that the on-duty personnel focus on the disposal action rather than repeatedly confirming. Each alarm contains the following fields: time slice number, alarm level, risk event type, trigger window range, trigger basis summary, and pointer mapping. The pointer mapping points to the original video segment and the state record table row, facilitating immediate playback and verification.

[0087] The generation process of the manipulation suggestion specifically includes: for different risk events, generating suggestions according to the executable constraints of adjacent time slices: (1) Lateral risk: in the next 3 time slices, set the target out-of-line direction dispersion level to positive, limit the out-of-line point area to the middle, and prohibit direct crossing from the middle to the left or right. This can direct the tow rope to the middle axis band and reduce the relative lateral sweeping of the ship body or the bulkhead. (2) Rebound risk: in the next 3 time slices, prohibit the out-of-line direction dispersion level from being continuously pushed inward left or inward right, and prefer to return it to positive; at the same time, suggest reducing the target value of the tension level in a continuous manner rather than suddenly in a single frame. Gradual rollback can reduce the sweep back caused by elastic release. (3) Overstretch risk: in the next 5 time slices, keep the out-of-line direction dispersion level from deviating from positive, limit the out-of-line point area to the middle, and require the tension level not to continue to be extremely high. Limiting the direction and area to the middle axis band can provide a safety margin for load reduction. Each suggestion is accompanied by an applicable time range and a corresponding binary target for execution checking against the cable trajectory sequence. When there are multiple alarms in the same time slice, take the suggestion corresponding to the emergency level as the main one, and other suggestions as supplementary prompts. This can give clear dominant actions in complex scenarios and avoid contradictions.

[0088] As Figure 2As shown, the tow rope swing frequency analysis diagram reveals the swing response characteristics of the tow rope under different frequency excitations. The abscissa represents the frequency, in hertz (Hz), ranging from 0 to 1.0 Hz; the ordinate represents the swing response, in degrees (°), ranging from 0 to 70°. According to the frequency response curve, it can be observed that there is a significant resonance peak near 0.3 Hz, at which the swing response reaches a maximum value of about 70°, marked as "maximum response". The main resonance peak occurs at 0.2 Hz, with a swing response of about 50°, which is the first main resonance frequency of the tow rope system. There is a sub-harmonic response at a higher frequency of 0.6 Hz, with a swing response of about 30°. Based on the swing response characteristics, the present application divides the swing level into three intervals: the small swing zone corresponds to a frequency less than 0.2 Hz, in which the swing response is relatively small and stable; the medium swing zone corresponds to a frequency of 0.2-0.4 Hz, which covers the main resonance frequency range; the large swing zone corresponds to a frequency greater than 0.4 Hz, in which there is a sub-harmonic response, but the overall swing response tends to decay. Through in-depth analysis of the swing frequency characteristics, the present application can accurately identify the dynamic state of the tow rope. When the swing frequency is detected to fall in the medium swing zone or the large swing zone, the system determines that the tow rope is in an unstable state, and the corresponding risk warning mechanism needs to be started. This analysis method provides important frequency domain characteristic parameters for subsequent constraint propagation and risk prediction, significantly improving the identification accuracy of the system for abnormal states of the tow rope.

[0089] As Figure 3As shown, the multi-anchor time series analysis diagram shows the identification, distribution and processing flow of different types of anchors in time series by the application. The horizontal coordinate represents time, with a unit of seconds (s) and a range of 0 to 50 s; and the vertical coordinate represents, from top to bottom, the pulling anchor, the approaching anchor, the standby anchor, the evading anchor and the section state. On the time axis, the system identifies four pulling anchors (T1-T4) located at about 5 s, 10 s, 25 s and 40 s, which meet the condition that the tension level does not decrease and at least once increases in the continuous three time slices. Three approaching anchors (A1-A3) are located at about 7 s, 20 s and 35 s, corresponding to the case that the out-line direction dispersion level is unidirectionally progressive to the same side in the continuous three time slices. Four standby anchors (S1-S4) are distributed at about 8 s, 12 s, 32 s and 42 s, representing the stable state that the swing level remains small and the tension level is low or medium in the continuous three time slices. Two evading anchors (E1-E2) appear at about 15 s and 28 s, which are triggered by the safe boundary constraint. According to the time series processing rule, the system takes the earliest anchor T1 as the starting point to perform forward section splicing, forming a forward section string from 5 s to 15 s; and takes the latest anchor T4 as the starting point to perform backward section splicing, forming a backward section string from 40 s to 20 s. In the 15 s-20 s time period, the forward and backward section strings overlap to form a conflict resolution zone. The system adopts the principle of longer coverage time or more pulling anchors to resolve the conflict, and finally generates a continuous shaped section. The time series analysis method realizes the full-time tracking and accurate modeling of the streamer running state through the collaborative identification of multiple types of anchors and the bidirectional splicing strategy, and provides a reliable time sequence basis for the subsequent risk event sequence generation and prediction alarm.

[0090] The above-described and above-embodied examples are only used to illustrate the technical solutions of the present application, but not to limit the same; although the present application has been described in detail with reference to the foregoing examples, those skilled in the art should understand that the technical solutions recorded in the foregoing examples can be modified, or some technical features can be replaced by equivalent ones; and such modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A deep learning-based tugboat monitoring and prediction method, characterized in that, The method includes the following steps: Step 1: Set up a fixed camera and tow cable tension meter on the foredeck and share a clock; slice the acquired video data and tension data into fixed length slices, extract the main tow cable segment and main exit point, and call the deep learning tow cable segmenter to backtrack the extraction if it fails; discretize the exit direction, exit point location, swing amplitude and tension at different levels to form a discrete sample sequence with time slice number. Step 2: First, sequentially filter and splice segments from the earliest anchor point to form a forward segment string; then, symmetrically splice backward segments from the latest anchor point, using the side with longer coverage time or more traction anchor points as the conflict resolution result; perform constraint propagation on the merged segments, including geometric consistency constraints and dynamic continuity constraints; after constraint propagation, bidirectionally converge and write back anchor point segments to form a defined segment and label it with traction, convergence, standby, or avoidance status; based on this, generate a risk event sequence, cable trajectory sequence, and tension interval sequence. Step 3: Generate graded alarms and control suggestions according to preset rules based on the risk event sequence, cable trajectory sequence, and tension interval sequence; In step 1, when line segment detection fails, the deep learning tow cable segmenter is invoked to generate a tow cable mask and the process reverts to the image processing flow that extracts the main line segment and main exit point of the tow cable near the exit point. Based on the orientation of the main line segment relative to the camera's longitudinal reference axis, the exit direction is discretized into five levels: inner left, left, center, right, and inner right. Based on the projection position of the main exit point on the lower edge of the image, the exit point location is discretized into left, center, and right. The number of changes in the exit direction is counted within a window composed of the most recent time slices, and the swing amplitude is discretized into small, medium, and large. The tension meter reading is discretized into four levels: low, medium, high, and extremely high using a calibration segmentation table. The exit direction discretization level, exit point location, swing amplitude level, tension level, and time slice number are recorded together to form a discrete sample sequence.

2. The deep learning-based tugboat monitoring and prediction method as described in claim 1, characterized in that, In step 1, the video data is divided into continuous time slices with a fixed duration. The image processing flow performed for each time slice includes: noise reduction, edge extraction, line segment detection, and extraction of the main line segment and main exit point of the tow cable near the exit point.

3. The deep learning-based tugboat monitoring and prediction method as described in claim 2, characterized in that, In step 2, an anchor set is generated from the discrete sample sequence according to the following rules: a traction anchor is generated when the tension level does not decrease and increases at least once within three consecutive time slices; a standby anchor is generated when the swing amplitude level remains small and the tension level is low or medium within three consecutive time slices; and a convergence anchor is generated when the discrete level of the outgoing direction progresses unidirectionally to the same side within three consecutive time slices.

4. The deep learning-based tugboat monitoring and prediction method as described in claim 3, characterized in that, Step 2, the process of forward segment selection and splicing to form a forward segment string from the earliest anchor point, includes: selecting candidate segments one by one in the future direction from the earliest anchor point and splicing them into a forward segment string; the candidate segments simultaneously satisfy the following transfer rules: the dispersion level of the exit direction changes by at most one level between adjacent time slices; the exit point location changes sequentially in the adjacent order of left, center, and right; the tension level and the swing amplitude level change in the same direction within the same time slice; if multiple candidate segments satisfy the same number of entries, the segment with the higher tension level is selected first; and the process of symmetrically performing backward splicing from the latest anchor point includes: forming a backward segment string from the latest anchor point in the past direction according to the same transfer rules as forward splicing; when the forward segment string and the backward segment string have inconsistent states in the same time slice, conflict resolution is performed, and the side with a longer coverage time is retained first; if the coverage time is the same, the side containing more traction anchor points is retained first.

5. The deep learning-based tugboat monitoring and prediction method as described in claim 4, characterized in that, In step 2, the temporary segments resulting from the merging of the forward and backward segment strings are subjected to three types of constraints until convergence. These constraints include: geometric consistency constraints (limiting the discrete level of the exit direction to inner left, left, or positive when the exit point is on the left, and inner right, right, or positive when the exit point is on the right); dynamic continuity constraints (limiting the tension level change between any two adjacent time slices to a single-level step; when a cross-level change is detected, the intermediate level is used to backfill, and the time slice is marked as a correction slice); and safety boundary constraints (converging the discrete level of the exit direction of the next time slice towards the ship's center normal when the swing amplitude is large and the tension level is high or extremely high); and for records that have deviated from the ship's center normal, reverting to the nearest legal level and marking the time slice as an avoidance slice.

6. The deep learning-based tugboat monitoring and prediction method as described in claim 5, characterized in that, In step 2, the process of bidirectional convergence and back-writing anchor points to form a defined segment and assigning segment labels includes: performing a reversible check on the results after constraint propagation on the time axis; if a time slice is marked as a correction slice or an avoidance slice and simultaneously triggers the traction anchor point condition, then the outgoing direction discrete level and outgoing point location of the slice are back-written with the traction anchor point as the main factor, so that the forward and backward paths are consistent at the slice; after completing the back-writing, the continuous time slices are converged into defined segments, and each defined segment is assigned a segment label selected from traction, convergence, standby, and avoidance.

7. The deep learning-based tugboat monitoring and prediction method as described in claim 6, characterized in that, In step 2, three prediction sequences are output according to the time sequence of the shaping section. The risk event sequence is generated as follows: when the swing amplitude changes from small or medium to large in a short period of time and the tension level is high or extremely high, lateral risk is output; when the tension level suddenly drops from high or extremely high to low or medium and the outgoing direction dispersion level is inward, rebound risk is output; when the tension level remains extremely high for multiple consecutive time slots, over-tension risk is output. The cable trajectory sequence is generated as follows: the outgoing direction dispersion level and the outgoing point location of each time slot are combined into a binary pair, and these binary pairs are arranged in time sequence. The tension interval sequence is generated as follows: the frequency of occurrence of each tension level in each shaping section is counted, the tension level with the highest frequency is taken as the tension interval label of that section, and these labels are arranged in time sequence of the shaping section.

8. A system for implementing the deep learning-based tugboat monitoring and prediction method according to any one of claims 1 to 7, characterized in that, The system includes: a data acquisition unit for setting up fixed cameras and towline tension meters on the foredeck and sharing a clock; slicing the acquired video and tension data into fixed-length slices, extracting the main towline segment and main exit point, and calling a deep learning towline segmenter to backtrack extraction in case of failure; discretizing the exit direction, exit point location, swing amplitude, and tension at different levels to form a discrete sample sequence with time slice numbers; a sequence analysis unit for filtering and splicing forward segments from the earliest anchor point to form a forward segment string; performing backward splicing symmetrically from the latest anchor point, and using the side with longer coverage time or more traction anchor points as the conflict resolution result; performing constraint propagation on the merged segments, including geometric consistency constraints and dynamic continuity constraints; writing back the anchor point slices bidirectionally after constraint propagation to form a defined segment and marking it with traction, approach, standby, or avoidance status; generating a risk event sequence, cable trajectory sequence, and tension interval sequence accordingly; and a result output unit for generating graded alarms and control suggestions based on the risk event sequence, cable trajectory sequence, and tension interval sequence according to preset rules.

Images