A computer vision-based ship traffic situation awareness method

Abstract

This invention discloses a computer vision-based method for ship traffic situational awareness, comprising: collecting ship traffic data and preprocessing it to obtain a video frame sequence; performing ship target detection and outputting a set of ship detection results; constructing an improved ByteTrack model and outputting a set of trajectory segments; performing a Poisson-Dobernoli hybrid filter to obtain a set of ship tracks; performing coordinate mapping and track element calculation, calculating speed, heading, and position; performing risk event determination and outputting risk event results; and performing graded early warning and interpretable processing to generate early warning information. This invention, through the collaborative processing of the improved ByteTrack model and the Poisson-Dobernoli hybrid filter, achieves graded early warning and interpretable output of risk events for ship targets under complex sea surface imaging conditions.

Description

Technical Field

[0001] This invention relates to the field of intelligent transportation technology, and in particular to a method for ship traffic situational awareness based on computer vision. Background Technology

[0002] Current ship traffic monitoring and situational awareness technologies primarily rely on shore-based radar, Automatic Identification System (AIS), electronic charts, and manual monitoring to acquire ship position and navigation status information, enabling traffic situation display, navigation organization, and risk warning. With the development of refined port and waterway supervision, high-density navigation management in nearshore waters, and intelligent shipping, the limitations of relying solely on radar or AIS are becoming increasingly apparent, particularly regarding target obstruction, signal loss, information delays, and insufficient coverage of non-AIS vessels. This has spurred the introduction of video perception technology based on shipborne or shore-based cameras into ship monitoring and safety assistance scenarios. Video perception provides richer appearance and scene information, offering advantages such as coverage of close-range targets, supplementing AIS blind spots, and visual evidence collection of abnormal behavior. However, maritime scenarios are often plagued by complex imaging factors, including fog, backlighting, sea surface reflection, wave swells, reduced contrast due to rain and fog, and low pixel ratios for small, distant targets. These factors cause target visibility and background interference to change significantly over time, leading to large fluctuations in the confidence level of ship target detection results, with both false alarms and missed detections, further impacting the stability of tracking and track output.

[0003] Existing vision-based multi-target tracking and situational analysis solutions typically employ frame-by-frame target detection and data association to generate trajectories. Based on these trajectories, speed and heading navigation elements are calculated, and regional rules governing channel boundaries and restricted areas are used to determine and issue warnings for intrusion, deviation, reverse navigation, and abnormal anchoring events. However, current solutions have significant limitations in complex maritime scenarios. The frame-by-frame processing is highly sensitive to short-term missed detections, false detections, global image jitter, and localized glare noise, easily leading to trajectory fragmentation, the generation of numerous short trajectories, and identity switching issues during target convergence. This makes it difficult to form stable, continuous trajectory segments and tracks suitable for situational calculations. Most solutions lack unified probabilistic modeling for target birth, disappearance, clutter, and data association uncertainties, making it difficult to quantify track existence and quality. This results in discontinuous risk event assessments over time, inconsistent tiered warning thresholds, and a lack of supporting evidence chains for regulatory action. Consequently, these solutions fail to meet the engineering application requirements for stable, continuous, and interpretable ship traffic situational awareness in complex sea conditions.

[0004] Therefore, how to provide a computer vision-based method for ship traffic situational awareness is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0005] One objective of this invention is to propose a computer vision-based method for ship traffic situation awareness. This invention comprehensively utilizes video preprocessing, ship target detection, multi-target tracking using an improved ByteTrack model, trajectory estimation with a Poisson-Dobernoli hybrid filter, and coordinate mapping and rule-based judgment techniques. This forms a complete process from ship traffic data collection by onboard cameras, through preprocessing, detection, trajectory segment generation, trajectory filtering, trajectory element calculation, traffic situation element extraction, risk event determination, to tiered early warning and interpretable output. The improved ByteTrack model drives trajectory generation through time windows, and position and body... This invention achieves stable and continuous tracking under complex sea surface imaging conditions through segmented updates, trajectory shape filtering and delay confirmation, and trajectory state refinement. A Poisson-Dobernouri hybrid filter uses trajectory segment-level representation for prediction and updating, and maintains the output based on a refined trajectory lifecycle structure, resulting in a set of ship tracks containing existence probabilities. Compared with existing technologies, this invention can generate reliable tracks under conditions of fog, backlight, sea surface reflection, wave turbulence, obstruction, and distant small targets. It also enables graded early warning of risk events and interpretable chain-of-evidence output for waterways and restricted areas, exhibiting advantages such as strong robustness, good continuity, and high engineering feasibility.

[0006] A computer vision-based method for ship traffic situation awareness according to an embodiment of the present invention includes: Collect ship traffic data output from shipborne camera equipment, preprocess the ship traffic data, and obtain video frame sequences; Perform ship target detection frame by frame on the video frame sequence and output the ship detection result set for each frame. The ship detection result set includes the detection box position parameters and the corresponding confidence scores. An improved ByteTrack model is constructed. Based on time window-driven trajectory generation, the set of ship detection results is cached to generate a set of trajectory segments. A position and identity separation update mechanism is introduced to perform updates. Shape features are calculated through trajectory shape filtering and delayed confirmation. Life cycle state assignment and state transition management are carried out by trajectory state refinement. The set of trajectory segments is output. An improved Poisson-Bernoulli hybrid filter is implemented, and the trajectory segment set is predicted and updated using a trajectory segment-level representation. The set of ship tracks is then maintained and output based on a refined trajectory lifecycle structure. The coordinate mapping and track element calculation of the set of ship tracks are performed. The tracks in pixel coordinates are mapped to the water reference coordinate system, and the speed, heading and position sequence of each track are calculated. Based on track elements, waterway areas and restricted areas data, traffic situation elements are extracted, risk events are determined, and risk event results are output. The system performs tiered early warnings and interpretable processing on the results of risk events, generating early warning information including risk level, track fragments, and keyframe indexes.

[0007] Optionally, the ship traffic data includes video frame sequences, timestamp sequences, camera extrinsic parameters, and waterway data.

[0008] Optionally, obtaining the video frame sequence includes: The raw video stream output by the shipborne camera is acquired, decoded according to the frame rate to obtain the raw frame sequence, and a corresponding acquisition time identifier is generated for each frame in the raw frame sequence to form a timestamp sequence. The raw frame sequence and the timestamp sequence are associated and stored. Acquire the camera extrinsic parameters of the shipborne camera equipment and store them in association with the original frame sequence; acquire channel data and convert the channel data to the water reference coordinate system corresponding to the camera extrinsic parameters. The original frame sequence is preprocessed to obtain a video frame sequence. The preprocessing includes image denoising, brightness and contrast adjustment, dehazing, and image stabilization. The image stabilization includes extracting feature points from adjacent frames, calculating feature point matching pairs, estimating global motion transformation parameters based on the matching pairs, and performing geometric transformation on the current frame to obtain a stabilized frame.

[0009] Optionally, the set of ship detection results for each output frame includes: The size of each frame in the video frame sequence is normalized, and the normalized image is input into the ship target detection unit to obtain a set of candidate detection boxes. Each candidate detection box in the set of candidate detection boxes contains bounding box parameters and confidence. The bounding box parameters are represented by a quadruple, which consists of the x-coordinate of the top left corner of the detection box, the y-coordinate of the top left corner of the detection box, the width of the detection box, and the height of the detection box. Non-maximum suppression is performed on the candidate detection box set to obtain the ship detection result set for each frame. In the non-maximum suppression process, the intersection-union ratio is used as the overlap metric. The intersection-union ratio is defined as the ratio of the area of ​​the intersection region of two detection boxes to the area of ​​the union region. When the intersection-union ratio is greater than a preset threshold, the detection boxes with low confidence are deleted. The set of ship detection results for each frame is output and recorded. Each detection result includes bounding box parameters, confidence level, frame number and acquisition time identifier. The set of ship detection results is composed of frames according to the frame number.

[0010] Optionally, the set of output trajectory segments includes: An improved ByteTrack model was constructed, including a time window-driven trajectory generation module, a location and identity separation and update module, a trajectory shape filtering and delay confirmation module, and a trajectory state refinement module; The time window-driven trajectory generation module caches the ship detection result set and generates trajectory segments. The ship detection result set is written into the time window cache in sequence according to the frame number. When the time window is updated, the ship detection result set in the time window is read and candidate association pairs are generated. Based on the candidate association pairs, the detection boxes of adjacent frames in the time window are associated to form a trajectory segment set. The location and identity separation update module updates the trajectory segment set, establishing a location state set and an identity set. The location state set records the bounding box parameters of the trajectory segment in each frame and the position and velocity state derived from the bounding box parameters. The identity set records the identity and identity continuity information of the trajectory segment. During each time window update, the location state set is updated first, and then the identity set is updated. The trajectory shape filtering and delayed confirmation module performs shape feature calculation and confirmation processing on the trajectory segment set. For each trajectory segment, the trajectory shape feature is calculated. When the trajectory shape feature meets the confirmation condition, the trajectory segment is marked as a confirmed trajectory segment. When the trajectory shape feature does not meet the confirmation condition and does not meet the rejection condition, the trajectory segment is marked as a delayed confirmation trajectory segment and retained until the next time window to continue updating. When the trajectory shape feature meets the rejection condition, the corresponding trajectory segment is deleted. The trajectory state refinement module performs lifecycle state assignment and state transition management on the trajectory segment set, divides the trajectory lifecycle state, and assigns state values ​​and performs state transitions for each trajectory segment based on the detection box association results and trajectory shape features within the time window, and outputs the trajectory segment set.

[0011] Optionally, the obtained set of ship tracks includes: Based on the trajectory segment set, construct the Poisson process component and the multi-Bernoulli mixed component, initialize the state variables and existence probability of each component, and establish a filtering iteration sequence that is executed according to the time window update cycle; The trajectory segment-level representation is used to express the observation of the trajectory segment set. The trajectory segment is used as the filtering observation unit and an observation state vector is generated for each trajectory segment. At each time window update, the state of the multi-Bernoulli mixture component is predicted and the state is updated based on the observation state vector to obtain the updated component set. Data association and component generation processing are performed on the updated component set. Candidate association pairs with each component are generated for each trajectory segment and an association result set is formed. New multi-Bernoulli components are generated for unassociated trajectory segments and multi-Bernoulli mixture components are added. The existence probability of components not updated by trajectory segments is updated while maintaining the state estimation sequence. The maintenance and output of the life cycle state structure of the multi-Bernoulli mixed components are refined. The life cycle state of the track is divided into birth candidate state, active state, decay state, dormant state and termination state. Based on the set of associated results and the change of existence probability, the state is assigned and the state transition is managed for each component, and the set of ship tracks is output.

[0012] Optionally, the coordinate mapping and track element calculation of the ship track set includes: Determine the coordinate mapping relationship between the pixel coordinate system and the water reference coordinate system. The coordinate mapping relationship is determined by the camera extrinsic parameters and camera intrinsic parameters. Based on the water surface plane assumption, establish the transformation from pixel points to plane points of the water reference coordinate system. For each track in the set of ship tracks, the midpoint of the lower edge of the corresponding bounding box is selected as the pixel observation point within each time window, and the coordinate transformation is performed on the pixel observation point to obtain the position point sequence in the water reference coordinate system, and a timestamp is associated with each position point in the position point sequence; The velocity sequence and heading sequence are calculated based on the position point sequence and timestamp. The velocity value of adjacent moments in the velocity sequence is calculated by the difference between the Euclidean distance of adjacent position points and the adjacent timestamp. The heading value of adjacent moments in the heading sequence is calculated by the angle between the line connecting adjacent position points and the longitudinal axis of the water reference coordinate system. The position sequence, velocity sequence and heading sequence are output as track elements.

[0013] Optionally, the output risk event results include: Read the data of the waterway area and the restricted area and establish regional units in the water reference coordinate system. The regional units include the waterway boundary polygon, the waterway centerline and the restricted area polygon. Traffic situation elements are extracted based on flight path elements. These elements include the number of ships, ship density, heading distribution statistics, and speed distribution statistics in the monitoring area. Risk events are determined and output based on track elements and regional units. Risk event determination includes restricted area intrusion events, channel deviation events, and reverse navigation events. Risk event results include event type, time period, and associated track identifier.

[0014] Optionally, the early warning information for generating risk levels, track fragments, and keyframe indexes includes: Receive risk event results and generate event logs. Event logs include event type, associated track identifier, event time period, event location sequence, and keyframe sequence number set. The risk level is calculated for the event record. The risk level is divided into three levels: Level 1, Level 2 and Level 3. The risk level is determined by the event intensity parameter and the classification threshold parameter. The event intensity parameter includes the depth of the intrusion into the restricted area, the distance of the deviation from the waterway and the duration of the reverse navigation. Generate and output early warning information. The early warning information includes risk level, event type, associated track identifier, event occurrence time period, track segment and key frame index. The track segment is the position sequence and heading sequence within the event occurrence time period. The key frame index is the set of frame numbers in the key frame number set that fall within the event occurrence time period.

[0015] The beneficial effects of this invention are: This invention, based on video preprocessing and ship target detection, combines an improved ByteTrack model with a Poisson-Dobernoli hybrid filter to achieve synergistic enhancement of multi-target ship tracking and trajectory estimation. Compared with existing visual tracking schemes that rely on frame-by-frame correlation and are susceptible to fluctuations in detection confidence, this invention utilizes time windows to drive trajectory generation, separates and updates position and identity, filters trajectory shape and delays confirmation, and refines trajectory states. This enables ship trajectory segments to maintain higher continuity and stability under conditions such as fog, backlight, sea surface reflection, wave swell, occlusion, and distant small targets. Furthermore, it uses a Poisson-Dobernoli hybrid filter with trajectory segment-level representation to predict and update trajectory segments. Combined with a refined trajectory lifecycle structure, it outputs a set of ship trajectories containing existence probabilities, thereby reducing trajectory breaks and identity switching caused by false alarms and missed detections, and improving the reliability and usability of trajectory output.

[0016] This invention further maps flight paths from pixel coordinates to a water reference coordinate system, calculates flight path elements such as speed, heading, and position sequence, and extracts traffic situation elements based on data from waterway areas and restricted areas. It also identifies risks of restricted area incursions, waterway deviations, and reverse navigation, forming a tiered early warning and interpretable output process. Compared to existing situational awareness systems that only provide warning conclusions and lack traceable evidence, this invention outputs early warning information including risk levels, flight path segments, and keyframe indexes, enabling spatiotemporal location of risk events and presentation of evidence chains, facilitating regulatory handling. Simultaneously, through a unified flight path element calculation and rule-based judgment process, it improves the consistency of situational awareness results and the feasibility of engineering implementation. Attached Figure Description

[0017] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings: Figure 1 This is a flowchart of a computer vision-based ship traffic situation perception method proposed in this invention. Figure 2 This is a structural block diagram of an improved ByteTrack model for a computer vision-based ship traffic situational awareness method proposed in this invention. Figure 3This is a functional diagram of the Poisson-Dobernal hybrid filter for a computer vision-based ship traffic situation perception method proposed in this invention. Detailed Implementation

[0018] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0019] refer to Figure 1 , Figure 2 and Figure 3 A computer vision-based method for ship traffic situational awareness includes: Collect ship traffic data output from shipborne camera equipment, preprocess the ship traffic data, and obtain video frame sequences; Perform ship target detection frame by frame on the video frame sequence and output the ship detection result set for each frame. The ship detection result set includes the detection box position parameters and the corresponding confidence scores. An improved ByteTrack model is constructed. Based on time window-driven trajectory generation, the set of ship detection results is cached to generate a set of trajectory segments. A position and identity separation update mechanism is introduced to perform updates. Shape features are calculated through trajectory shape filtering and delayed confirmation. Life cycle state assignment and state transition management are carried out by trajectory state refinement. The set of trajectory segments is output. An improved Poisson-Bernoulli hybrid filter is implemented, and the trajectory segment set is predicted and updated using a trajectory segment-level representation. The set of ship tracks is then maintained and output based on a refined trajectory lifecycle structure. The coordinate mapping and track element calculation of the set of ship tracks are performed. The tracks in pixel coordinates are mapped to the water reference coordinate system, and the speed, heading and position sequence of each track are calculated. Based on track elements, waterway areas and restricted areas data, traffic situation elements are extracted, risk events are determined, and risk event results are output. The system performs tiered early warnings and interpretable processing on the results of risk events, generating early warning information including risk level, track fragments, and keyframe indexes.

[0020] In this embodiment, the ship traffic data includes video frame sequences, timestamp sequences, camera extrinsic parameters, and waterway data.

[0021] In this embodiment, obtaining the video frame sequence includes: The raw video stream output by the shipborne camera is acquired, decoded according to the frame rate to obtain the raw frame sequence, and a corresponding acquisition time identifier is generated for each frame in the raw frame sequence to form a timestamp sequence. The raw frame sequence and the timestamp sequence are associated and stored. Acquire the extrinsic parameters of the shipborne camera equipment and store them in association with the original frame sequence; acquire channel data and convert the channel data to the water reference coordinate system corresponding to the camera extrinsic parameters. Specifically, the conversion of the channel data to the water reference coordinate system corresponding to the camera extrinsic parameters involves: When converting channel data to the water reference coordinate system corresponding to the camera extrinsic parameters, the origin, coordinate axis direction, and unit of the water reference coordinate system are determined to be consistent with the world coordinate system defined by the camera extrinsic parameters. The spatial representation of each channel element in the channel data is read, including the channel centerline point set, the channel boundary point set, and the vertex sequence of the restricted area polygon. If the channel data is in the geographic coordinate system, the latitude and longitude are converted into planar coordinates. Then, the planar coordinates are converted to the water reference coordinate system based on the translation of the reference point and the rotation of the azimuth angle. When the channel data is already in local planar coordinates, the coordinate unification is completed directly based on the translation of the origin and the rotation of the coordinate axes. Finally, the coordinate set of the channel centerline, channel boundary, and restricted area polygon in the water reference coordinate system is obtained and stored in association with the camera extrinsic parameters. Preprocessing is performed on the original frame sequence to obtain a video frame sequence. Preprocessing includes image denoising, brightness and contrast adjustment, dehazing, and image stabilization. Image stabilization includes extracting feature points from adjacent frames, calculating feature point matching pairs, estimating global motion transformation parameters based on the matching pairs, and performing a geometric transformation on the current frame to obtain a stabilized frame. The steps of extracting feature points from adjacent frames, calculating feature point matching pairs, and estimating global motion transformation parameters based on the matching pairs are as follows: grayscale conversion and scale normalization are performed in the previous frame and the current frame, respectively. In each frame, corner point and key point detection operators are used to extract feature points and generate corresponding feature descriptors for each feature point. Then, based on the descriptor similarity, initial matching of feature points is performed between the two frames to obtain a set of feature point matching pairs. The matching pair set is then subjected to consistency screening to remove obvious mismatches. On the retained matching pairs, a parameter fitting method is used to estimate the global motion transformation parameters. The global motion transformation parameters are selected as two-dimensional affine transformation parameters. The estimation results are determined by the coordinate correspondence of the matching pairs in the two frames. The process of performing geometric transformation on the current frame to obtain a stable frame specifically involves: converting global motion transformation parameters into a transformation matrix for image resampling, performing reverse mapping calculation on the pixel coordinates of the current frame to align the transformed pixel positions with the coordinate system of the previous frame, using interpolation resampling on the non-integer coordinates obtained from the reverse mapping to generate the transformed image, and cropping or filling the edge hole regions generated after the transformation. Finally, a stable frame that is globally aligned with the previous frame is output and the stabilized video frame sequence is formed according to the frame number.

[0022] In this embodiment, the set of ship detection results for each frame output includes: Each frame in the video frame sequence is normalized in size, and the normalized image is input into the ship target detection unit to obtain a candidate detection box set. Each candidate detection box in the candidate detection box set includes bounding box parameters and confidence scores. The bounding box parameters are represented by a quadruple, which consists of the x-coordinate of the top-left corner of the detection box, the y-coordinate of the top-left corner of the detection box, the width of the detection box, and the height of the detection box. Specifically, the candidate detection box set is obtained as follows: The size-normalized single-frame image is input into the ship target detection unit. During the image feature extraction stage, the ship target detection unit generates multi-layer feature representations through convolution and visual transformation structures. During the candidate generation stage, the feature representations are used for target prediction and bounding box regression. The target prediction outputs the ship presence confidence corresponding to each candidate position, and the bounding box regression outputs the bounding box parameters corresponding to the candidate positions. The bounding box parameters are represented in the form of a quadruple of the upper left corner of the detection box, the upper left corner of the detection box, the width of the detection box, and the height of the detection box. The bounding box parameters and corresponding confidence of all candidate positions in the same frame are summarized to form the candidate detection box set of the current frame. Non-maximum suppression processing is performed on the candidate detection box set to obtain the ship detection result set for each frame. In the non-maximum suppression processing, the intersection-union ratio is used as the overlap metric. The intersection-union ratio is defined as the ratio of the area of ​​the intersection region of two detection boxes to the area of ​​the union region. When the intersection-union ratio is greater than a preset threshold, the detection boxes with low confidence are deleted. The preset threshold is specifically set to 0.5. The set of ship detection results for each frame is output and recorded. Each detection result includes bounding box parameters, confidence level, frame number and acquisition time identifier. The set of ship detection results is composed of frames according to the frame number.

[0023] In this embodiment, the set of output trajectory segments includes: An improved ByteTrack model is constructed, including a time-window driven trajectory generation module, a location and identity separation and update module, a trajectory shape filtering and delay confirmation module, and a trajectory state refinement module, wherein: When constructing the improved ByteTrack model, a time window caching structure and window-level processing flow are set up on the original framework based on detection box association to generate trajectory. This allows the ship detection result set to enter the time window according to the frame number and complete the detection box association within the window to generate trajectory segments. In the trajectory management structure, the trajectory information is split into two types of data structures: position status and identity. The position status is used to record the bounding box parameters and the displacement and velocity information derived from them, while the identity is used to record the trajectory number and continuity information. The position status and identity are updated separately in the update process. A trajectory shape feature calculation unit is added to the trajectory segment confirmation structure to calculate the trajectory segment length, displacement sequence, heading change sequence and scale change sequence, and to perform confirmation, delayed confirmation or deletion of the trajectory segment. The trajectory state set is refined in the life cycle management structure and state transition rules are defined. The trajectory state is refined into initialization, pending confirmation, confirmed, occlusion retention, recurrence, decay and deletion. After each time window update, the trajectory segment is assigned a state value and the state transition is managed, and the trajectory segment set is output. The time-window driven trajectory generation module caches and generates trajectory segments from the ship detection result set. The ship detection result set is written sequentially into the time window cache according to frame number. During each time window update, the ship detection result set within the time window is read and candidate association pairs are generated. Based on these candidate association pairs, adjacent frame detection boxes within the time window are associated to form a trajectory segment set, where: The generation of candidate association pairs specifically involves: dividing the ship detection result set of each frame within the time window into a high-confidence detection set and a low-confidence detection set with a confidence level of 0.5; generating a trajectory candidate set at the predicted position of the tracking trajectory in the previous frame or the previous time step; using the trajectory candidate set and the high-confidence detection set of the current frame as the first association input; calculating the matching cost between each trajectory and each detection box and forming a cost matrix; the matching cost is determined by the intersection-union ratio of the detection boxes; simultaneously setting a threshold to eliminate trajectory-detection combinations that do not meet the matching cost condition; after the first association, constructing the second association input with the unmatched trajectory and the low-confidence detection set; calculating the cost matrix and performing threshold filtering in the same way; the trajectory-detection combinations retained by the first association and the second association together constitute the candidate association pair set; the threshold elimination of matching cost is to retain only trajectory-detection combinations with an intersection-union ratio of not less than 0.30. The process of associating and forming a trajectory segment set involves the following steps: Within each time window update, matching is performed sequentially from earliest to latest frame number. The cost matrix of the candidate association pair set is solved using an assignment algorithm to obtain the matching result. For successfully matched trajectories and detection boxes, the current frame detection box is added to the corresponding trajectory's detection box sequence, and the trajectory's frame number sequence is updated. For unmatched but retained trajectories, a missing marker is recorded in the current frame, and the trajectory is retained. For detection boxes not matched by any trajectory, a new trajectory is created and assigned a new identity. At the end of the time window, the continuous detection box sequence formed by each trajectory within the time window is output as a trajectory segment. The trajectory segment set is composed of the trajectory segment's identity, frame number set, and bounding box parameter sequence. The assignment algorithm uses the cost matrix of the candidate association pair as input and employs the Hungarian algorithm to obtain a one-to-one minimum total cost matching result between the trajectory and the detection box. The location and identity separation update module updates the trajectory segment set, establishing a location state set and an identity set. The location state set records the bounding box parameters of the trajectory segment in each frame and the position and velocity state derived from the bounding box parameters. The identity set records the identity and identity continuity information of the trajectory segment. During each time window update, the location state set is updated first, and then the identity set is updated. The trajectory shape filtering and delayed confirmation module performs shape feature calculation and confirmation processing on the trajectory segment set. For each trajectory segment, it calculates the trajectory shape feature. When the trajectory shape feature meets the confirmation criteria, the trajectory segment is marked as a confirmed trajectory segment. When the trajectory shape feature does not meet the confirmation criteria but does not meet the removal criteria, the trajectory segment is marked as a delayed confirmation trajectory segment and retained until the next time window for continued updating. When the trajectory shape feature meets the removal criteria, the corresponding trajectory segment is deleted. Specifically: The calculated trajectory shape features specifically include: trajectory shape features include trajectory segment length, adjacent frame displacement sequence, velocity sequence, heading change sequence, and scale change sequence. The trajectory segment length is the number of valid frames within the trajectory segment. The adjacent frame displacement sequence is obtained from the coordinate difference of the center point of the adjacent frame bounding box. The velocity sequence is obtained from the difference between the adjacent frame displacement and the adjacent timestamp. The heading change sequence is obtained by calculating the angle between the adjacent frame displacement vector and the vertical axis of the water reference coordinate system and then taking the difference between adjacent frames. The scale change sequence is obtained from the area difference or aspect ratio difference of the adjacent frame bounding box. The maximum value, mean, and variance of the displacement sequence, velocity sequence, heading change sequence, and scale change sequence are calculated to form a trajectory shape feature set. The confirmation and rejection conditions are as follows: The confirmation condition is set as follows: the trajectory segment length is not less than five frames, the maximum value of the displacement sequence of adjacent frames is not greater than 0.6, the maximum absolute value of the heading change sequence is not greater than 30 degrees, and the maximum absolute value of the scale change sequence is between 0.6 and 1.4. The rejection condition is set as follows: the trajectory segment length is less than two frames, or there is an outlier greater than 1.2 in the displacement sequence of adjacent frames, or there is an outlier greater than 90 degrees in the heading change sequence, or there is an outlier less than 0.3 or greater than 3 in the scale change sequence. The trajectory state refinement module performs lifecycle state assignment and state transition management on the trajectory segment set. It divides the trajectory lifecycle states and assigns state values ​​and performs state transitions for each trajectory segment based on the detection box association results within the time window and trajectory shape features, outputting a trajectory segment set, where: The process of dividing the trajectory lifecycle state is as follows: the trajectory segment lifecycle state is divided into initialization state, pending confirmation state, confirmed state, occlusion retention state, recurrence state, decay state, and deletion state. The initialization state is a newly created trajectory segment, the pending confirmation state is a trajectory segment with delayed confirmation, the confirmed state is a trajectory segment that meets the confirmation conditions, the occlusion retention state is a trajectory segment that is not matched in the current time window but is still retained, the recurrence state is a trajectory segment that is rematched after occlusion, the decay state is a trajectory segment that has been continuously unmatched and reaches the decay criterion, and the deletion state is a trajectory segment that meets the deletion conditions and is removed. The state assignment and state transition are as follows: after each time window update, the state assignment and transition are performed based on the detection box association results and trajectory shape features. Newly created trajectory segments are assigned to the initialization state. When the initialization state and the pending confirmation state trajectory segments meet the confirmation conditions, they are transferred to the confirmed state. When the rejection conditions are met, they are transferred to the deletion state. When neither the confirmation nor rejection conditions are met, they are maintained or transferred to the pending confirmation state. When a confirmed state trajectory segment does not match in the current time window and does not exceed the number of retention windows, it is transferred to the occlusion retention state. When it is re-matched, it is transferred to the reproduction state and then transferred to the confirmed state in the next update. When the number of consecutive non-matches exceeds the number of decay windows, it is transferred to the decay state. When the number of consecutive non-matches exceeds the number of deletion windows, it is transferred to the deletion state.

[0024] In this embodiment, obtaining the set of ship tracks includes: Based on the trajectory segment set, a Poisson process component and a multi-Bernoulli mixture component are constructed. The state variables and existence probabilities of each component are initialized, and a filtering iteration sequence is established to be executed according to the time window update cycle, wherein: The initialization of the state variables and existence probability of each component is specifically as follows: the observation state vector of each trajectory segment in the trajectory segment set is used as the initial state estimate. The observation state vector consists of the x-coordinate of the center point of the bounding box, the y-coordinate of the center point of the bounding box, the width of the bounding box, and the height of the bounding box. For the multi-Bernoulli component generated by the trajectory segment that is not associated with any existing component, the existence probability is initialized to 0.8, and the state covariance is initialized to a diagonal matrix. The variance of the center point coordinate is 25, and the variance of the width and height is 16. For the existing multi-Bernoulli component, the state estimate and existence probability updated in the previous time window are kept as the initial values. For the Poisson process component, the spatial intensity is initialized to a constant intensity that is uniformly distributed in the monitoring area, and the range of action is limited to the area covered by the waterway data. The establishment of the filtering iteration sequence executed according to the time window update cycle is specifically as follows: the time window number of the trajectory segment set is used as the filtering time step number, the update interval of adjacent time steps is determined according to the time window sliding step size, the state estimate of the previous time step is propagated to the current time step in the prediction stage, the trajectory segment set of the current time step is used to complete the component update in the update stage, and the updated component set is stored as the input of the next time step, forming a filtering iteration sequence that is recursively pushed by the time window; A trajectory segment-level representation is used to express the observation of the trajectory segment set. The trajectory segment is used as a filtering observation unit, and an observation state vector is generated for each trajectory segment. At each time window update, the state of the multi-Bernoulli mixture components is predicted, and the state is updated based on the observation state vector, resulting in the updated component set, where: The generation of the observation state vector is specifically as follows: for each trajectory segment, read the bounding box quadruple parameters of each frame within the time window, calculate the x-coordinate of the center point of each frame as the x-coordinate of the upper left corner plus half the width, and the y-coordinate of the center point as the y-coordinate of the upper left corner plus half the height, and take the average value of the x-coordinate of the center point, the y-coordinate of the center point, the width and the height within the time window to obtain the observation state vector of the trajectory segment. The state update process specifically involves: calculating the residual between the observed state vector and the predicted state of the candidate association and updating the state estimate and state covariance. The residual is the difference between the observed state vector and the predicted state in the observation space. For the successfully matched components, the state updated based on the residual is used as the state estimate for the current time window and the existence probability is updated synchronously. Finally, the updated component set is obtained by summarizing the results. Data association and component generation are performed on the updated component set. For each trajectory segment, candidate association pairs with each component are generated, forming an association result set. New multi-Bernoulli components are generated for unassociated trajectory segments, and multi-Bernoulli mixture components are added. For components not updated by trajectory segments, the existence probability is updated while maintaining the state estimation sequence, where: The process of generating candidate association pairs with each component for each trajectory segment and forming an association result set is as follows: First, calculate the mean of the predicted state and the predicted covariance of each multi-Bernoulli component for the current time window, and generate an observation state vector for each trajectory segment. Calculate the matching cost for any trajectory segment and any component. The matching cost is the squared Mahalanobis distance between the observed state vector and the mean of the predicted state. The threshold of a Mahalanobis distance not greater than 9.2 is used to filter and obtain a set of candidate association pairs. Construct a cost matrix from the set of candidate association pairs and use the Hungarian algorithm to solve for one-to-one matching to obtain the association result set of trajectory segments and components. The association result set consists of successfully matched pairs, a set of unmatched trajectory segments, and a set of unmatched components. The process of generating new Bernoulli components and adding them to unassociated trajectory segments specifically involves: creating new components for each unmatched trajectory segment, using the observation state vector of the trajectory segment as the initial state mean of the new component, and adding the Bernoulli mixture component after assigning a track identifier and a time window sequence number start value to the new component. The process of updating the existence probability of components not updated by the trajectory segment and maintaining the state estimation sequence is as follows: retain the predicted state mean and predicted covariance as the state estimate of the time window for the unmatched components, and add the current time window state estimate to the state estimation sequence of the components while recording the unupdated markers. Update the existence probability of the components according to a fixed decay factor, with the decay factor set to 0.7. The maintenance and output of the trajectory lifecycle state structure for multi-Bernoulli mixed components are refined. The trajectory lifecycle state is divided into birth candidate state, active state, decay state, dormant state, and termination state. Based on the association result set and the change in existence probability, state assignment and state transition management are performed on each component, and the ship trajectory set is output. The trajectory lifecycle state division is as follows: the trajectory lifecycle state corresponding to each multi-Bernoulli component is divided into birth candidate state, active state, dormant state, decay state, and termination state. The birth candidate state is a component newly created by an unmatched trajectory segment and has not yet reached the stable output condition. The active state is a component whose continuous time window is updated by the trajectory segment and whose existence probability is not less than 0.8. The dormant state is a component whose current time window has not been updated by the trajectory segment but whose existence probability is not less than 0.6. The decay state is a component that has not been updated continuously and whose existence probability is less than 0.6 but not less than 0.3. The termination state is a component whose existence probability is less than 0.3.

[0025] In this embodiment, the coordinate mapping and track element calculation of the ship track set includes: The coordinate mapping relationship between the pixel coordinate system and the water reference coordinate system is determined. This mapping relationship is determined by camera extrinsic and intrinsic parameters. Based on the assumption of a water surface plane, a transformation from pixel points to points on the water reference coordinate system plane is established. Specifically, this transformation involves: The camera matrix is ​​constructed from the camera intrinsic parameters, and the rotation matrix and translation vector from the water reference coordinate system to the camera coordinate system are constructed from the camera extrinsic parameters. Under the assumption of a water surface plane, the water surface in the water reference coordinate system is defined as a plane equation, a plane normal vector, and a plane determined by a point on the plane. The imaging ray in the camera coordinate system is obtained by back-projecting the pixel. The intersection of the imaging ray and the water surface plane is obtained to obtain the coordinates of the intersection point. The coordinates of the intersection point are output as the plane point in the water reference coordinate system. For each track in the set of ship tracks, the midpoint of the lower edge of the corresponding bounding box is selected as the pixel observation point within each time window, and the coordinate transformation is performed on the pixel observation point to obtain the position point sequence in the water reference coordinate system, and a timestamp is associated with each position point in the position point sequence; The velocity sequence and heading sequence are calculated based on the position point sequence and timestamp. The velocity value of adjacent moments in the velocity sequence is calculated by the difference between the Euclidean distance of adjacent position points and the adjacent timestamp. The heading value of adjacent moments in the heading sequence is calculated by the angle between the line connecting adjacent position points and the longitudinal axis of the water reference coordinate system. The position sequence, velocity sequence and heading sequence are output as track elements.

[0026] In this embodiment, the output of risk event results includes: Read the data of the waterway area and the restricted area and establish regional units in the water reference coordinate system. The regional units include the waterway boundary polygon, the waterway centerline and the restricted area polygon. Traffic situation elements are extracted based on flight path elements. These elements include the number of ships, ship density, heading distribution statistics, and speed distribution statistics in the monitoring area. Based on track elements and regional units, risk event determination is performed and risk event results are output. Risk event determination includes prohibited area intrusion events, channel deviation events, and reverse navigation events. Risk event results include event type, time period, and associated track identifier. Specifically, performing risk event determination and outputting risk event results involves: For each track, the position sequence and heading sequence are traversed by timestamp. When the position point falls into the no-navigation zone polygon, a no-navigation zone intrusion event is determined, and the time period is determined by the first and last intrusion times. When the vertical distance from the position point to the channel centerline is greater than the deviation threshold and the duration exceeds the duration threshold, a channel deviation event is determined, and the time period is determined. When the angle between the heading and the channel's permissible direction is greater than the reverse navigation threshold and the duration exceeds the duration threshold, a reverse navigation event is determined, and the time period is determined. The event type, occurrence time, and associated track identifier of each event are summarized and output as the risk event result. The deviation threshold is set to 50 meters, the duration threshold is set to 30 seconds, and the reverse navigation threshold is set to an angle between the heading and the channel's permissible direction of not less than 120 degrees.

[0027] In this embodiment, the early warning information for generating risk levels, track segments, and keyframe indexes includes: Receive risk event results and generate event logs. Event logs include event type, associated track identifier, event time period, event location sequence, and keyframe sequence number set. Risk levels are calculated for event logs, and are divided into three levels: Level 1, Level 2, and Level 3. The risk level is determined using event intensity parameters and classification threshold parameters. Event intensity parameters include the depth of intrusion into the restricted area, the distance of deviation from the navigation channel, and the duration of reverse navigation. The calculation of risk level for event records is as follows: extract the corresponding event intensity parameter from the event record according to the event type; when the event type is a no-navigation zone intrusion event, calculate the intrusion depth, which is the maximum value of the minimum distance from the location point to the boundary of the no-navigation zone within the event time period; when the event type is a channel deviation event, calculate the deviation distance, which is the maximum value of the vertical distance from the location point to the centerline of the channel within the event time period; when the event type is a reverse navigation event, calculate the reverse navigation duration, which is the duration of the event time period. The tiered threshold parameters are specifically defined as follows: the tiered threshold parameters are a set of three levels of thresholds set according to the event type, including the no-navigation zone intrusion depth threshold, the channel deviation distance threshold, and the reverse navigation duration threshold. The first-level threshold for the no-navigation zone intrusion depth is 100 meters, the second-level threshold is 50 meters, and the third-level threshold is 20 meters. The first-level threshold for the channel deviation distance is 150 meters, the second-level threshold is 80 meters, and the third-level threshold is 50 meters. The first-level threshold for the reverse navigation duration is 120 seconds, the second-level threshold is 60 seconds, and the third-level threshold is 30 seconds. Generate and output early warning information. The early warning information includes risk level, event type, associated track identifier, event occurrence time period, track segment and key frame index. The track segment is the position sequence and heading sequence within the event occurrence time period. The key frame index is the set of frame numbers in the key frame number set that fall within the event occurrence time period.

[0028] Example 1: To verify the feasibility of this invention in practice, it was applied to a monitoring scenario of the main channel and adjacent anchorage area of ​​a coastal port. Shipborne cameras were installed on patrol vessels to continuously capture video of the port entry and exit channels, the edge of the anchorage, and the port basin entrance. The video resolution was 1920×1080, the frame rate was 25 frames per second, and the total acquisition time was approximately 26 hours. During the acquisition process, low visibility fog, morning backlight, strong afternoon sea surface reflection, platform shaking caused by waves, and obstruction from passing vessels occurred. Small, distant targets often appeared smaller than 30×30 pixels in the image, leading to fluctuations in detection confidence, short-term missed detections, and concurrent false alarms due to reflections. Traditional frame-by-frame correlation easily resulted in trajectory fragmentation, identity switching, and false alarms, making it difficult to form reliable tracks for situational awareness calculations and stably output event evidence.

[0029] In the scene, the original video is decoded into a frame sequence and denoised, brightness and contrast adjusted, dehazed, and image stabilized. Ship detection bounding boxes and confidence scores are then output frame by frame. In the improved ByteTrack model, detection results are cached using time windows. Within the time window, detection bounding boxes are associated to generate trajectory segments, and position status updates and identity updates are performed separately. Trajectory shape features such as trajectory segment length, adjacent frame displacement, heading changes, and scale changes are calculated for each trajectory segment. Trajectory segments that meet the confirmation criteria are entered into the confirmed state, while those that do not meet the confirmation criteria but not the removal criteria are entered into the exclusion state. Delayed confirmation and cross-window accumulation are combined with refined trajectory lifecycle states to manage occlusion retention, recurrence, and decay. Then, trajectory segments are used as observation units for Poisson-Bernoulli hybrid filtering, and prediction, updating, association, and component generation are completed according to time window periods. The output of ship tracks containing existence probabilities is maintained with a refined track lifecycle structure. Finally, the tracks are mapped to the water reference coordinate system, and the speed, heading, and position sequence are calculated. Intrusion, deviation, and reverse navigation are determined by combining channel boundaries and restricted area polygons, and early warning information containing risk level, track segments, and keyframe indexes is output.

[0030] Using manual verification as a reference, approximately 2,400 valid vessel navigation segments were labeled, covering various states including port entry, departure, anchoring, encounters, and overtaking. Even in challenging periods with low visibility and backlighting, it consistently outputs continuous tracks, with single-target continuous tracking exceeding 40 minutes. Track recovery after obstruction mostly occurred within two time windows. Addressing false alarms caused by sea surface reflection, the number of short-term false targets entering the track filtering and event determination chain was significantly reduced after track shape filtering and delay confirmation. In terms of waterway monitoring, records of event type, occurrence time, associated track identifiers, track segments, and keyframe indexes were generated for events such as restricted area intrusion, channel deviation, and reverse navigation. On-site verification personnel can quickly locate evidence frames and verify track segments based on the keyframe index, thus meeting the application requirements for stable tracks and interpretable alarms under complex sea conditions.

[0031] Table 1 Comparison of Ship Situation Awareness Methods in Complex Sea States

[0032] As shown in Table 1, in terms of detection performance, the detection rate of the method of this invention is 91.8%, significantly higher than the original ByteTrack's 82.4%, and also higher than RT-DETR+ByteTrack's 85.6% and the improved ByteTrack's 88.9%. Simultaneously, the false alarm rate is reduced to 5.2%, a significant decrease compared to the original ByteTrack's 9.6%, and better than BoT-SORT's 8.8% and StrongSORT's 8.4%. The data indicates that even when factors such as haze, backlighting, and reflection cause fluctuations in detection confidence and an increase in false alarms, the method of this invention can effectively reduce false alarms while improving the detection rate, thus reducing the number of noisy targets entering the tracking and event determination chain.

[0033] In terms of tracking and track quality, the track continuity rate of the method of this invention reaches 89.6%, an improvement of 15.4 percentage points compared to the original ByteTrack, and higher than the 85.3% of the improved ByteTrack alone. It is also significantly better than the 77.5% of BoT-SORT and the 76.8% of StrongSORT. The ID switching frequency is 2.1 times per hour, lower than the 5.8 times per hour of the original ByteTrack, and better than the 3.2 times per hour of the improved ByteTrack alone. In terms of positioning accuracy, the average positioning error of this invention is 12.9 meters, which is better than the 19.4 meters of the original ByteTrack and the 17.6 meters of RT-DETR+ByteTrack. It can be seen that this invention can output more continuous, more stable, and more accurate tracks under complex sea surface imaging and occlusion conditions, providing a more reliable data foundation for situational awareness calculation and risk assessment.

[0034] From the perspective of business alarms and engineering real-time performance, the alarm accuracy rate of this invention is 90.3%, higher than the original ByteTrack's 79.1%, and also higher than RT-DETR+ByteTrack's 82.4% and the improved ByteTrack's 87.2%. This indicates that, based on the improved track quality, the risk event judgment based on airway and restricted area rules is closer to the results of manual review. In terms of real-time performance, this invention processes frames at 32 FPS, meeting the online processing requirements of 25 FPS video streams, with an end-to-end latency of 155 milliseconds, providing real-time warning output capability. Compared with StrongSORT's 28 FPS and 160 milliseconds, and BoT-SORT's 31 FPS and 145 milliseconds, this invention significantly improves the overall performance of detection, tracking, and alarming while maintaining deployable real-time capabilities.

[0035] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A computer vision-based method for ship traffic situational awareness, characterized in that, include: Collect ship traffic data output from shipborne camera equipment, preprocess the ship traffic data, and obtain video frame sequences; Perform ship target detection frame by frame on the video frame sequence and output the ship detection result set for each frame. The ship detection result set includes the detection box position parameters and the corresponding confidence scores. An improved ByteTrack model is constructed. Based on time window-driven trajectory generation, the set of ship detection results is cached to generate a set of trajectory segments. A position and identity separation update mechanism is introduced to perform updates. Shape features are calculated through trajectory shape filtering and delayed confirmation. Life cycle state assignment and state transition management are carried out by trajectory state refinement. The set of trajectory segments is output. A Poisson-Bernoulli hybrid filter is performed, and the trajectory segment set is predicted and updated using a trajectory segment-level representation. The set of ship tracks is then maintained and output based on a refined trajectory lifecycle structure. The coordinate mapping and track element calculation of the set of ship tracks are performed. The tracks in pixel coordinates are mapped to the water reference coordinate system, and the speed, heading and position sequence of each track are calculated. Based on track elements, waterway areas and restricted areas data, traffic situation elements are extracted, risk events are determined, and risk event results are output. The system performs tiered early warnings and interpretable processing on the results of risk events, generating early warning information including risk level, track fragments, and keyframe indexes.

2. The method for ship traffic situational awareness based on computer vision according to claim 1, characterized in that, The ship traffic data includes video frame sequences, timestamp sequences, camera extrinsic parameters, and waterway data.

3. The method for ship traffic situation awareness based on computer vision according to claim 1, characterized in that, The process of obtaining the video frame sequence includes: The raw video stream output by the shipborne camera is acquired, decoded according to the frame rate to obtain the raw frame sequence, and a corresponding acquisition time identifier is generated for each frame in the raw frame sequence to form a timestamp sequence. The raw frame sequence and the timestamp sequence are associated and stored. Acquire the camera extrinsic parameters of the shipborne camera equipment and store them in association with the original frame sequence; acquire channel data and convert the channel data to the water reference coordinate system corresponding to the camera extrinsic parameters. The original frame sequence is preprocessed to obtain a video frame sequence. The preprocessing includes image denoising, brightness and contrast adjustment, dehazing, and image stabilization. The image stabilization includes extracting feature points from adjacent frames, calculating feature point matching pairs, estimating global motion transformation parameters based on the matching pairs, and performing geometric transformation on the current frame to obtain a stabilized frame.

4. The method for ship traffic situation awareness based on computer vision according to claim 1, characterized in that, The set of ship detection results for each output frame includes: The size of each frame in the video frame sequence is normalized, and the normalized image is input into the ship target detection unit to obtain a set of candidate detection boxes. Each candidate detection box in the set of candidate detection boxes contains bounding box parameters and confidence. The bounding box parameters are represented by a quadruple, which consists of the x-coordinate of the top left corner of the detection box, the y-coordinate of the top left corner of the detection box, the width of the detection box, and the height of the detection box. Non-maximum suppression is performed on the candidate detection box set to obtain the ship detection result set for each frame. In the non-maximum suppression process, the intersection-union ratio is used as the overlap metric. The intersection-union ratio is defined as the ratio of the area of ​​the intersection region of two detection boxes to the area of ​​the union region. When the intersection-union ratio is greater than a preset threshold, the detection boxes with low confidence are deleted. The set of ship detection results for each frame is output and recorded. Each detection result includes bounding box parameters, confidence level, frame number and acquisition time identifier. The set of ship detection results is composed of frames according to the frame number.

5. The method for ship traffic situational awareness based on computer vision according to claim 1, characterized in that, The set of output trajectory segments includes: An improved ByteTrack model was constructed, including a time window-driven trajectory generation module, a location and identity separation and update module, a trajectory shape filtering and delay confirmation module, and a trajectory state refinement module; The time window-driven trajectory generation module caches the ship detection result set and generates trajectory segments. The ship detection result set is written into the time window cache in sequence according to the frame number. When the time window is updated, the ship detection result set in the time window is read and candidate association pairs are generated. Based on the candidate association pairs, the detection boxes of adjacent frames in the time window are associated to form a trajectory segment set. The location and identity separation update module updates the trajectory segment set, establishing a location state set and an identity set. The location state set records the bounding box parameters of the trajectory segment in each frame and the position and velocity state derived from the bounding box parameters. The identity set records the identity and identity continuity information of the trajectory segment. During each time window update, the location state set is updated first, and then the identity set is updated. The trajectory shape filtering and delayed confirmation module performs shape feature calculation and confirmation processing on the trajectory segment set. For each trajectory segment, the trajectory shape feature is calculated. When the trajectory shape feature meets the confirmation condition, the trajectory segment is marked as a confirmed trajectory segment. When the trajectory shape feature does not meet the confirmation condition and does not meet the rejection condition, the trajectory segment is marked as a delayed confirmation trajectory segment and retained until the next time window to continue updating. When the trajectory shape feature meets the rejection condition, the corresponding trajectory segment is deleted. The trajectory state refinement module performs lifecycle state assignment and state transition management on the trajectory segment set, divides the trajectory lifecycle state, and assigns state values ​​and performs state transitions for each trajectory segment based on the detection box association results and trajectory shape features within the time window, and outputs the trajectory segment set.

6. The method for ship traffic situational awareness based on computer vision according to claim 1, characterized in that, The obtained set of ship tracks includes: Based on the trajectory segment set, construct the Poisson process component and the multi-Bernoulli mixed component, initialize the state variables and existence probability of each component, and establish a filtering iteration sequence that is executed according to the time window update cycle; The trajectory segment-level representation is used to express the observation of the trajectory segment set. The trajectory segment is used as the filtering observation unit and an observation state vector is generated for each trajectory segment. At each time window update, the state of the multi-Bernoulli mixture component is predicted and the state is updated based on the observation state vector to obtain the updated component set. Data association and component generation processing are performed on the updated component set. Candidate association pairs with each component are generated for each trajectory segment and an association result set is formed. New multi-Bernoulli components are generated for unassociated trajectory segments and multi-Bernoulli mixture components are added. The existence probability of components not updated by trajectory segments is updated while maintaining the state estimation sequence. The maintenance and output of the life cycle state structure of the multi-Bernoulli mixed components are refined. The life cycle state of the track is divided into birth candidate state, active state, decay state, dormant state and termination state. Based on the set of associated results and the change of existence probability, the state is assigned and the state transition is managed for each component, and the set of ship tracks is output.

7. The method for ship traffic situational awareness based on computer vision according to claim 1, characterized in that, The coordinate mapping and track element calculation of the ship track set includes: Determine the coordinate mapping relationship between the pixel coordinate system and the water reference coordinate system. The coordinate mapping relationship is determined by the camera extrinsic parameters and camera intrinsic parameters. Based on the water surface plane assumption, establish the transformation from pixel points to plane points of the water reference coordinate system. For each track in the set of ship tracks, the midpoint of the lower edge of the corresponding bounding box is selected as the pixel observation point within each time window, and the coordinate transformation is performed on the pixel observation point to obtain the position point sequence in the water reference coordinate system, and a timestamp is associated with each position point in the position point sequence; The velocity sequence and heading sequence are calculated based on the position point sequence and timestamp. The velocity value of adjacent moments in the velocity sequence is calculated by the difference between the Euclidean distance of adjacent position points and the adjacent timestamp. The heading value of adjacent moments in the heading sequence is calculated by the angle between the line connecting adjacent position points and the longitudinal axis of the water reference coordinate system. The position sequence, velocity sequence and heading sequence are output as track elements.

8. The method for ship traffic situational awareness based on computer vision according to claim 1, characterized in that, The output of risk event results includes: Read the data of the waterway area and the restricted area and establish regional units in the water reference coordinate system. The regional units include the waterway boundary polygon, the waterway centerline and the restricted area polygon. Traffic situation elements are extracted based on flight path elements. These elements include the number of ships, ship density, heading distribution statistics, and speed distribution statistics in the monitoring area. Risk events are determined and output based on track elements and regional units. Risk event determination includes restricted area intrusion events, channel deviation events, and reverse navigation events. Risk event results include event type, time period, and associated track identifier.

9. The method for ship traffic situational awareness based on computer vision according to claim 1, characterized in that, The early warning information that generates risk levels, track segments, and keyframe indexes includes: Receive risk event results and generate event logs. Event logs include event type, associated track identifier, event time period, event location sequence, and keyframe sequence number set. The risk level is calculated for the event record. The risk level is divided into three levels: Level 1, Level 2 and Level 3. The risk level is determined by the event intensity parameter and the classification threshold parameter. The event intensity parameter includes the depth of the intrusion into the restricted area, the distance of the deviation from the waterway and the duration of the reverse navigation. Generate and output early warning information. The early warning information includes risk level, event type, associated track identifier, event occurrence time period, track segment and key frame index. The track segment is the position sequence and heading sequence within the event occurrence time period. The key frame index is the set of frame numbers in the key frame number set that fall within the event occurrence time period.

Images