A port ship information management system and method based on image processing

Abstract

This invention belongs to the field of port vessel management technology, specifically relating to a port vessel information management system and method based on image processing. This invention generates vessel movement trajectories by real-time acquisition of port video streams and target detection and tracking. By combining this with movement constraint conditions associated with identity information, it achieves real-time compliance identification of vessel movement trajectories, effectively addressing the data coverage blind spots and real-time deficiencies in traditional supervision. Furthermore, through behavioral pattern analysis of the vessel movement trajectory dataset, it can identify and output abnormal behavior patterns, sending early warning signals of course deviations to the corresponding vessels. This improves the comprehensiveness and accuracy of port vessel supervision, providing a more efficient management model for safe port operations.

Description

Technical Field

[0001] This invention belongs to the field of port and ocean vessel management technology, specifically relating to a port and ocean vessel information management system and method based on image processing. Background Technology

[0002] As the process of port intelligence accelerates, the traditional manual supervision model is no longer able to cope with the increasing density of ships and the complex navigation environment. With the continuous development of information technology, the supervision of ocean-going vessels in ports is also transforming from manual supervision to intelligent supervision. This not only ensures a dual improvement in supervision efficiency and accuracy, but also shows significant advantages in key aspects such as risk warning, behavior tracing and emergency response, providing a more solid guarantee for the safe operation of ports.

[0003] While existing technologies employ ship monitoring, they are susceptible to signal obstruction, equipment malfunctions, or human intervention, resulting in data coverage blind spots and insufficient real-time performance. This is particularly problematic in complex port environments, where it is difficult to assess the compliance and potential risks of ocean-going vessels based on their operational dynamics. Consequently, regulatory delays and frequent misjudgments inevitably occur, failing to meet the current demand for intelligent port ocean-going vessel management. Therefore, this solution provides an image processing-based method for port ocean-going vessel information management to address these issues. Summary of the Invention

[0004] The purpose of this invention is to provide a port vessel information management system and method based on image processing, which can perform real-time dynamic monitoring of port vessels and improve the comprehensiveness and real-time nature of the monitoring.

[0005] The specific technical solution adopted by this invention is as follows: A port vessel information management method based on image processing includes: Load and display the port site map, and mark the location coordinates and identification of all target ocean-going vessels on the map in real time; Real-time acquisition of on-site video streams at the port, and target detection and tracking based on the on-site video streams to generate the movement trajectory of each ocean-going vessel; Identify and associate the target ocean vessel's identity information and the corresponding motion restriction range, and then determine the motion constraints of the target ocean vessel based on the motion restriction range; Based on motion constraints, the system identifies in real time whether the motion trajectory of each vessel is compliant and triggers alarm information when violations are detected. When normal behavior is detected, the system continuously collects vessel motion trajectory data and summarizes it into a vessel motion trajectory dataset. Based on the ocean vessel motion trajectory dataset, behavioral pattern analysis is performed on the ocean vessel motion trajectory to identify and output abnormal behavior patterns of ocean vessels, and a route deviation warning signal is sent to the corresponding ocean vessel under the abnormal behavior pattern.

[0006] In a preferred embodiment, the step of loading and displaying the port site map and marking the location coordinates and identification marks of all target ocean-going vessels on the map in real time includes: Acquire map data of the port area and load a port site map including a basic geographic information layer, a dynamic labeling layer for ocean-going vessels, and an alarm area layer; The real-time location coordinates of the ocean-going vessel are mapped to the coordinate system of the port's on-site map, and the corresponding identification icon and real-time movement direction of the ocean-going vessel are overlaid and displayed. Static anchor point coordinates are preset on the basic geographic information layer, and position mapping is achieved through the affine transformation matrix between the geographic coordinate system and the image pixel coordinates.

[0007] In a preferred embodiment, the step of generating the motion trajectory of each ocean vessel by performing target detection and tracking based on the live video stream includes: The on-site video stream is decoded to extract continuous video frames, and multi-scale ocean vessel target detection is performed in the continuous video frames to identify the bow and stern coordinates of the target ocean vessel. The direction of travel of the target ocean vessel is determined based on the bow and stern coordinates. Combined with the length-to-beam ratio constraint and the prior knowledge of the continuity of the heading angle, the center point of the target ocean vessel in the current frame is determined. Cross-frame target tracking is performed by calculating the Euclidean distance and heading angle deviation of the target ocean vessel between adjacent frames, generating a motion trajectory data sequence containing timestamps, position coordinates, and heading angles.

[0008] In a preferred embodiment, after the motion trajectory data sequence is output, the motion trajectory data sequence is subjected to trajectory data cleaning and outlier filtering. The specific steps are as follows: The motion trajectory data sequence is segmented and smoothed by using median filtering of position coordinates and heading angle within a sliding time window to reduce trajectory noise caused by video frame jitter or target detection errors. Based on the continuity constraint of ocean vessel motion, the displacement distance and heading changes between adjacent trajectory points are detected; If the displacement distance exceeds the upper limit of displacement or the change in heading exceeds the threshold of angle change, it is determined to be an abnormal trajectory point and interpolation correction is performed. The interpolated motion trajectory data sequence is mapped to the port geographic coordinate system to form a standardized trajectory dataset in the geographic coordinate system.

[0009] In a preferred embodiment, the step of determining the motion constraint conditions of the target ocean vessel based on the motion restriction range includes: Based on the vessel's identity information, retrieve the movement restriction range bound to the identity information. The movement restriction range includes at least the static geographical constraint range and the dynamic behavioral constraint range. The scope of static geographic constraints is defined based on port maps and includes the channel areas that ocean-going vessels are permitted to enter, the designated berth areas, and the static restricted areas that are prohibited from entering. The scope of dynamic behavioral constraints is generated based on the identity attributes of the ocean-going vessel and port rules, including the safe distance to be maintained with adjacent ocean-going vessels, the speed limit range within the waterway, and the time limit for staying in the port area; Within the motion restriction range, retrieve the static geographical constraint range and dynamic behavioral constraint range corresponding to the target ocean vessel, and combine them into motion constraint conditions.

[0010] In a preferred embodiment, the step of identifying whether the motion trajectories of each vessel are compliant in real time based on motion constraints includes: Acquire standardized trajectory data of currently monitored ocean-going vessels and determine their spatial inclusion relationship with static geographical constraints to identify whether the target ocean-going vessels have engaged in boundary-crossing behavior; If it exists, an out-of-bounds alarm message will be triggered; If it does not exist, the compliance comparison with the scope of dynamic behavior constraints will continue, specifically including: Determine whether the real-time speed is within the speed limit range. If it is not within the speed limit range, it is considered a speed violation; otherwise, it is considered normal driving. Based on the real-time position coordinates of each vessel, the real-time distance between the current vessel and the nearest adjacent vessel is calculated. If the real-time distance is less than the safe distance, it is determined to be a violation of the safe distance; otherwise, it is determined to be normal navigation. Within the berth area, the continuous dwell time of the current ocean vessel is calculated cumulatively. If the dwell time exceeds the limit, it is judged as a violation of berthing time limit; otherwise, it is judged as normal sailing.

[0011] In a preferred embodiment, the step of performing behavioral pattern analysis on the ship's motion trajectory based on the ship's motion trajectory dataset, and identifying and outputting abnormal behavior patterns of the ship includes: Obtain the historical trajectory data set of the target ocean vessel after it enters the port area from the ocean vessel motion trajectory dataset; Based on the historical trajectory data set, extract the motion feature vector of each historical trajectory; Unsupervised clustering of motion feature vectors yields multiple clusters of similar behavior patterns, where the center vector of each behavior pattern cluster is used to characterize the corresponding reference motion feature vector. The real-time motion feature vector corresponding to the latest real-time motion trajectory of the target ocean vessel entering the port is mapped to each behavior pattern cluster, and the pattern deviation between the real-time motion feature vector and each reference motion feature vector is calculated. Compare the pattern deviation with a preset deviation threshold; If the deviation exceeds the deviation threshold, it indicates that the real-time movement trajectory of the target ocean vessel has deviated. The confidence level of the deviation is then checked. If the confidence level is passed, the deviation is determined to be an abnormal behavior pattern, and an early warning signal is triggered simultaneously. If the deviation of the pattern does not exceed the deviation threshold, it indicates that the real-time movement trajectory of the target ocean vessel conforms to the normal driving mode, and the target ocean vessel is considered to be driving and docking normally in the port.

[0012] In a preferred embodiment, the step of performing a confidence check on the deviation behavior includes: The duration of the target ocean vessel's deviation behavior within the port area is obtained, and the duration of the deviation is compared with a preset confidence threshold for deviation duration. If the duration of the deviation exceeds the confidence threshold for the deviation duration, it will be directly judged as abnormal behavior. If the duration of continuous deviation does not exceed the confidence threshold for deviation duration, the frequency of deviation of the target ocean vessel after entering the port area is counted, and the deviation frequency is compared with the preset frequency confidence threshold. When the frequency of deviation exceeds the frequency confidence threshold, it is directly judged as abnormal behavior; When the deviation frequency does not exceed the frequency confidence threshold, the trajectory trend prediction mechanism is executed for the target ocean vessel; When the trajectory trend prediction mechanism is executed, it starts with the real-time motion feature vector and extracts the trajectory segment before the deviation occurs to construct the prediction function; Predict the sequence of predicted trajectory points of the target ocean-going vessel within the demand forecasting time window based on the prediction function; The predicted trajectory point sequence is cross-validated with the motion constraints. If the validation result does not meet the motion constraints, the deviation behavior is determined to be abnormal behavior; otherwise, it is recorded as occasional deviation behavior.

[0013] The present invention also provides a port and ocean vessel information management system based on image processing, which uses the above-mentioned port and ocean vessel information management method based on image processing, including: The initialization module is used to load and display the port site map, and to mark the location coordinates and identification of all target ocean-going vessels on the map in real time; The trajectory output module is used to collect real-time video streams from the port and perform target detection and tracking based on the video streams to generate the motion trajectory of each ocean-going vessel. The constraint determination module is used to identify and associate the target ocean vessel's identity information and the motion restriction range corresponding to the identity information, and then determine the motion constraint conditions of the target ocean vessel based on the motion restriction range. The trajectory judgment module is used to identify whether the motion trajectory of each ship is compliant in real time based on motion constraints, and to trigger alarm information when a violation is identified. When a normal behavior is identified, the module continuously collects ship motion trajectory data and summarizes it into a ship motion trajectory dataset. The behavior analysis module is used to analyze the behavior patterns of ocean-going vessels based on the ocean-going vessel trajectory dataset, identify and output abnormal behavior patterns of ocean-going vessels, and send route deviation warning signals to the corresponding ocean-going vessels under abnormal behavior patterns.

[0014] And, an electronic device, the electronic device comprising: At least one processor; and a memory communicatively connected to the at least one processor; The memory stores a computer program that can be executed by the at least one processor, which enables the at least one processor to perform the above-described image processing-based port vessel information management method.

[0015] The technical effects achieved by this invention are as follows: This invention generates vessel movement trajectories by real-time acquisition of port video streams and target detection and tracking. By combining movement constraints with identity information, it achieves real-time compliance identification of vessel movement trajectories, effectively addressing the data coverage blind spots and real-time deficiencies in traditional supervision. Furthermore, through behavioral pattern analysis of the vessel movement trajectory dataset, it can identify and output abnormal behavior patterns, sending early warning signals of course deviations to the corresponding vessels. This enhances the comprehensiveness and accuracy of port vessel supervision, providing a more efficient management model for safe port operations. Attached Figure Description

[0016] Figure 1 This is a schematic diagram of the method flow of the present invention; Figure 2 This is a schematic diagram of the system modules of the present invention; Figure 3 This is a schematic diagram of the electronic device structure of the present invention. Detailed Implementation

[0017] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0018] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0019] Secondly, the term "an embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in a preferred embodiment" appearing in different places throughout this specification does not necessarily refer to the same embodiment, nor is it a single or selective embodiment that mutually excludes other embodiments.

[0020] Please see Figure 1 As shown, the present invention provides a port vessel information management method based on image processing, comprising: S1. Load and display the port site map, and mark the location coordinates and identification of all target ocean-going vessels on the map in real time; In step S1, with the development of maritime trade, ports, as key hubs of sea-land transportation, are facing increasing complexity in vessel scheduling and management. To ensure the orderly operation and berthing of vessels within the port, monitoring and behavioral management of these vessels is essential. In this embodiment, a port site map is first loaded, including information such as the port's channel distribution, berth locations, and restricted navigation areas. Simultaneously, the location and identification information of all vessels within the port area are collected to clarify the dynamics of each vessel. The steps of loading and displaying the port site map and marking the location coordinates and identification of all target vessels on the map in real time include: Acquire map data of the port area and load a port site map including a basic geographic information layer, a dynamic labeling layer for ocean-going vessels, and an alarm area layer; The real-time location coordinates of the ocean-going vessel are mapped to the coordinate system of the port's on-site map, and the corresponding identification icon and real-time movement direction of the ocean-going vessel are overlaid and displayed. Static anchor point coordinates are preset on the basic geographic information layer, and position mapping is achieved through the affine transformation matrix between the geographic coordinate system and the image pixel coordinates. Specifically, when loading the port site map, it is first necessary to acquire the map data of the port area and then process the map data into layers to construct a port site map containing a basic geographic information layer, a dynamic labeling layer for ocean-going vessels, and an alarm area layer. The basic geographic information layer, as the bottom layer, is used to present the port's topography and waterway distribution, providing a basic reference for subsequent location positioning. The dynamic labeling layer for ocean-going vessels is used to overlay and display the location information of each target ocean-going vessel in real time, including identification icons mapped from the real-time location coordinates of the vessels to the map coordinate system, such as the vessel's name and call sign. It also uses specific symbols to indicate the real-time movement direction of the vessels, allowing managers to intuitively grasp the dynamics of each vessel. The specific symbol forms are determined based on the actual situation. The system is configured according to actual needs. The alarm area layer is pre-marked on the map, such as no-navigation zones or speed-restricted zones. When a ship approaches or enters these areas, it can trigger a linkage alert with the ship's position in the dynamic marking layer. In addition, multiple static anchor point coordinates are preset on the basic geographic information layer. Static anchor point coordinates are usually selected from landmark objects in the port that have fixed geographic coordinates and are easy to identify. Specifically, they can be berth endpoints or guide signs. By using the actual geographic coordinate system coordinates of the static anchor points and their pixel coordinates on the map image, an affine transformation matrix between the geographic coordinate system and the image pixel coordinates can be established. This facilitates the conversion of the ship's coordinates into pixel coordinates on the map image, thereby completing the positioning and marking of the ship on the port's on-site map.

[0021] S2. Real-time acquisition of port video streams, and target detection and tracking based on the video streams to generate the movement trajectory of each ocean-going vessel; In step S2, when monitoring ships within the port, real-time video streams of the port are collected to capture the movement trajectories of the ships within the port area. The steps of target detection and tracking based on the live video stream to generate the movement trajectory of each ship include: The on-site video stream is decoded to extract continuous video frames, and multi-scale ocean vessel target detection is performed in the continuous video frames to identify the bow and stern coordinates of the target ocean vessel. The direction of travel of the target ocean vessel is determined based on the bow and stern coordinates. Combined with the length-to-beam ratio constraint and the prior knowledge of the continuity of the heading angle, the center point of the target ocean vessel in the current frame is determined. Cross-frame target tracking is performed by calculating the Euclidean distance and heading angle deviation of the target ocean vessel between adjacent frames, generating a motion trajectory data sequence containing timestamps, position coordinates, and heading angles. Specifically, after acquiring the live video stream, it is first decoded and converted into continuous video frames to provide basic data for subsequent target detection. Within these continuous video frames, a multi-scale ship target detection algorithm is used to identify the bow and stern coordinates of each target ship. Determining these coordinates reveals the ship's physical location and provides direct evidence for judging its direction of travel. Based on the identified bow and stern coordinates, the ship's direction of travel can be determined. Furthermore, considering the inherent aspect ratio constraints of the ship (different types of ships have aspect ratios within a certain range) and the prior knowledge of the course angle continuity (the course angle of a ship will not change drastically in a short period), the center point of the target ship in the current frame can be determined. This facilitates subsequent trajectory tracking of the target ship. After obtaining the center point of the current frame, cross-frame target tracking is performed by calculating the Euclidean distance between adjacent frames and the course angle deviation (the change in course angle between adjacent frames), ensuring continuous tracking of the same ship. Through continuous tracking, a sequence of motion trajectory data containing timestamps, location coordinates, and heading angles can be generated, which can then be used to determine the compliance of subsequent target ocean-going vessels.

[0022] Secondly, after the motion trajectory data sequence is output, trajectory data cleaning and outlier filtering are performed on the motion trajectory data sequence. The specific steps are as follows: The motion trajectory data sequence is segmented and smoothed by using median filtering of position coordinates and heading angle within a sliding time window to reduce trajectory noise caused by video frame jitter or target detection errors. Based on the continuity constraint of ocean vessel motion, the displacement distance and heading changes between adjacent trajectory points are detected; If the displacement distance exceeds the upper limit of displacement or the change in heading exceeds the threshold of angle change, it is determined to be an abnormal trajectory point and interpolation correction is performed. The interpolated and corrected motion trajectory data sequence is mapped to the port geographic coordinate system to form a standardized trajectory dataset in the geographic coordinate system. Specifically, after the motion trajectory data sequence is output, it is first segmented and smoothed using median filtering of position coordinates and heading angles within a sliding time window. This effectively reduces trajectory noise caused by occasional errors due to video frame jitter. Then, based on the continuity constraint of the ship's motion (the displacement and heading changes of a ship over a short period are usually gradual and do not occur abruptly), the displacement distance and heading change between adjacent trajectory points are detected. Specifically, the Euclidean distance between two adjacent trajectory points is calculated as the displacement distance, and the absolute difference in heading angles is calculated as the heading change. If the displacement distance between adjacent trajectory points exceeds a preset threshold... If the upper limit of displacement (i.e., the distance the ship moves exceeds the maximum displacement threshold corresponding to its reasonable sailing speed) or the change in heading exceeds the preset angle change threshold, the trajectory point is determined to be an abnormal trajectory point. For trajectory points determined to be abnormal, a linear interpolation method is used for correction. Specifically, the reasonable position and heading angle of the abnormal point at the time of the abnormal point are estimated based on the coordinates and timestamps of the normal trajectory points before and after the abnormal point. Then, the cleaned and corrected motion trajectory data sequence is mapped to the port geographic coordinate system through the affine transformation matrix between the previously established geographic coordinate system and the image pixel coordinates, thereby forming a standardized trajectory dataset in the geographic coordinate system.

[0023] S3. Identify and associate the target ocean vessel's identity information and the corresponding motion restriction range, and then determine the motion constraint conditions of the target ocean vessel based on the motion restriction range; In step S3, after the target vessel enters the port area, its identity information is simultaneously identified and cross-verified using pre-stored vessel registration information in the port management database to ensure the accuracy of identity recognition. After identity information identification and confirmation, the target vessel's corresponding movement restriction range is automatically associated with its identity information to ensure its safe navigation and berthing within the port area. The step of determining the movement constraints of the target vessel based on the movement restriction range includes: Based on the vessel's identity information, retrieve the movement restriction range bound to the identity information. The movement restriction range includes at least the static geographical constraint range and the dynamic behavioral constraint range. The scope of static geographic constraints is defined based on port maps and includes the channel areas that ocean-going vessels are permitted to enter, the designated berth areas, and the static restricted areas that are prohibited from entering. The scope of dynamic behavioral constraints is generated based on the identity attributes of the ocean-going vessel and port rules, including the safe distance to be maintained with adjacent ocean-going vessels, the speed limit range within the waterway, and the time limit for staying in the port area; Within the motion restriction range, retrieve the static geographical constraint range and dynamic behavioral constraint range corresponding to the target ocean vessel, and combine them into motion constraint conditions; Specifically, when determining the motion constraints of a target ocean-going vessel, the first step is to retrieve its associated motion restriction range from the port management database based on the vessel's identity information. This motion restriction range needs to be pre-defined according to the port's overall planning and safety regulations. It includes static geographical constraints and dynamic behavioral constraints. The static geographical constraints are a set of areas defined by geographic coordinates based on the port's on-site map, specifying the specific channel areas that the vessel is permitted to enter within the port. These channel areas typically have clearly defined width, depth, and navigation direction restrictions. Other constraints include designated berth areas, each with corresponding vessel type and tonnage requirements, where the vessel must berth and operate. Finally, there are static restricted areas, such as those within the port area. The static geographical constraints and dynamic behavioral constraints are based on the specific identity attributes of the vessel and the real-time operation rules of the port, and are dynamically generated behavioral norms. These include the minimum safe distance that a vessel must maintain with adjacent vessels during navigation to prevent collisions, speed limits in different channel sections or specific areas, and the time limit for a vessel to stay in specific areas of the port. If the time limit is exceeded, the vessel must apply to the port management department for a delayed berthing or immediate departure to ensure the port's turnover efficiency. After obtaining the static geographical constraints and dynamic behavioral constraints corresponding to the target vessel, they are integrated into motion constraints for the target vessel, which are then used as the basis for judging whether the vessel's motion trajectory is compliant.

[0024] S4. Based on motion constraints, identify in real time whether the motion trajectory of each ship is compliant, and trigger alarm information when a violation is identified. When a normal behavior is identified, continuously collect ship motion trajectory data and summarize it into a ship motion trajectory dataset. In step S4, after the motion constraints of the target vessel are determined, the real-time motion trajectory of the vessel is continuously monitored and its compliance is judged based on these constraints. This ensures that the vessel's navigation and berthing behavior within the port complies with safety regulations. When a violation is detected, an alarm message is sent to the target vessel and the port dispatch center is notified simultaneously. The time, location, and specific type of violation are automatically recorded. Conversely, under normal circumstances, the motion trajectory data of the target vessel is continuously recorded to facilitate subsequent analysis of its behavior patterns. The step of identifying the compliance of each vessel's motion trajectory in real time based on the motion constraints includes: Acquire standardized trajectory data of currently monitored ocean-going vessels and determine their spatial inclusion relationship with static geographical constraints to identify whether the target ocean-going vessels have engaged in boundary-crossing behavior; If it exists, an out-of-bounds alarm message will be triggered; If it does not exist, the compliance comparison with the scope of dynamic behavior constraints will continue, specifically including: Determine whether the real-time speed is within the speed limit range. If it is not within the speed limit range, it is considered a speed violation; otherwise, it is considered normal driving. Based on the real-time position coordinates of each vessel, the real-time distance between the current vessel and the nearest adjacent vessel is calculated. If the real-time distance is less than the safe distance, it is determined to be a violation of the safe distance; otherwise, it is determined to be normal navigation. Within the berth area, the continuous dwell time of the current ocean vessel is calculated cumulatively. If the dwell time exceeds the limit, it is judged as a violation of berthing time limit; otherwise, it is judged as normal sailing. Specifically, when identifying whether the movement trajectory of a target vessel is compliant, the system first acquires standardized trajectory data after the vessel enters the port monitoring area. Then, it determines the spatial inclusion relationship between the position coordinates in this standardized trajectory data and the static geographical constraints in the movement constraints. This determines whether the target vessel is within a permitted channel area or designated berth area. If it is in a static restricted area, it is directly identified as a boundary violation, and a boundary violation alarm is immediately triggered. This alarm will indicate the identity of the violating vessel, its current position coordinates, and the type of boundary violation area. If no boundary violation has occurred, a compliance comparison is further performed with the dynamic behavior constraints. First, the target vessel's speed is compared with the speed limit range in the dynamic behavior constraints. If the real-time speed exceeds the upper limit or falls below the lower limit, it is determined to be a speed violation. Second, based on the real-time position coordinates of each vessel, a Euclidean algorithm is used to determine compliance. The distance formula calculates the distance between the current ocean-going vessel and all other ocean-going vessels in the port, and filters out the nearest neighboring ocean-going vessel. If the real-time distance is less than the preset safe distance threshold, it is judged as a safety distance violation. For target ocean-going vessels that have entered the berth area, their continuous stay time will be counted simultaneously. If the continuous stay time exceeds the stay time limit specified in the dynamic behavior constraint range, and no delay berthing permission has been received from the port management department, it is judged as a berthing overtime violation. When any violation occurs, an alarm message containing the violation type, time of occurrence, specific location and related ocean-going vessel information will be generated immediately. The alarm signal will be sent to the bridge of the ocean-going vessel through the port dispatch platform and simultaneously pushed to the port monitoring platform so that the management personnel can intervene in time. For ocean-going vessels judged to be sailing normally, their standardized trajectory data will be continuously added to the ocean-going vessel movement trajectory dataset to enrich the sample library used for port ocean-going vessel management.

[0025] S5. Analyze the behavior patterns of ocean-going vessels based on the ocean-going vessel trajectory dataset, identify and output abnormal behavior patterns of ocean-going vessels, and send route deviation warning signals to the corresponding ocean-going vessels under abnormal behavior patterns. In step S5, when the target vessel is sailing normally within the port area, its standardized trajectory data will be used to perform corresponding behavioral pattern analysis to ensure that the target vessel continues to conform to the established navigation logic and historical behavior baseline. The steps of analyzing the vessel's trajectory based on the vessel's motion trajectory dataset to identify and output abnormal behavior patterns include: Obtain the historical trajectory data set of the target ocean vessel after it enters the port area from the ocean vessel motion trajectory dataset; Based on the historical trajectory data set, extract the motion feature vector of each historical trajectory; Unsupervised clustering of motion feature vectors yields multiple clusters of similar behavior patterns, where the center vector of each behavior pattern cluster is used to characterize the corresponding reference motion feature vector. The real-time motion feature vector corresponding to the latest real-time motion trajectory of the target ocean vessel entering the port is mapped to each behavior pattern cluster, and the pattern deviation between the real-time motion feature vector and each reference motion feature vector is calculated. Compare the pattern deviation with a preset deviation threshold; If the deviation exceeds the deviation threshold, it indicates that the real-time movement trajectory of the target ocean vessel has deviated. The confidence level of the deviation is then checked. If the confidence level is passed, the deviation is determined to be an abnormal behavior pattern, and an early warning signal is triggered simultaneously. If the deviation of the pattern does not exceed the deviation threshold, it indicates that the real-time movement trajectory of the target ocean vessel conforms to the normal driving mode, and the target ocean vessel is considered to be driving and docking normally in the port. Specifically, when identifying the behavior patterns of a target ocean-going vessel within a port area, it is first necessary to retrieve historical trajectory data from the vessel's past entries into the port that were deemed normal navigation, ensuring the comprehensiveness of the behavior pattern analysis. Then, based on the acquired historical trajectory data, multi-dimensional motion feature vectors are extracted for each historical trajectory, including average speed, speed variance, and rate of change of heading angle, etc. This allows for the characterization of the target vessel's navigation habits and behavior characteristics from multiple dimensions. Next, unsupervised clustering is used to perform cluster analysis on the motion feature vectors of all historical trajectories, grouping trajectories with similar motion characteristics into one category, thus obtaining multiple clusters of similar behavior patterns. The central vector of each behavior pattern cluster is extracted as a reference motion feature vector representing the corresponding behavior pattern within that cluster. These reference motion feature vectors constitute the normal behavior baseline of the target ocean-going vessel. When the target ocean-going vessel re-enters the port, its real-time motion feature vector is extracted, maintaining the same dimension as the historical feature vectors. The real-time motion feature vectors are then mapped to the previously obtained behavior pattern clusters, and cosine is calculated. Similarity is used to quantify the pattern deviation between the real-time motion feature vector and each reference motion feature vector. Then, the pattern deviation is compared with a preset deviation threshold, which is usually set according to the deviation distribution of historical data and the safety level requirements of port management. If the pattern deviation between the real-time motion feature vector and all reference motion feature vectors exceeds the preset deviation threshold, it is judged that the real-time motion trajectory of the corresponding target ship tends to deviate from the normal behavior baseline. Of course, to avoid the occurrence of misjudgment, a confidence check is also performed on the deviation behavior to ensure the rationality of the judgment result. After the check passes and it is confirmed that the deviation is not caused by external force majeure factors, the deviation behavior is judged as an abnormal behavior pattern. At this time, a route deviation warning signal is immediately sent to the corresponding ship, and the abnormal behavior pattern and related trajectory data are reported to the port management platform so that the management personnel can conduct further analysis and intervention. Conversely, if the pattern deviation does not exceed the deviation threshold, it indicates that the real-time motion trajectory of the target ship is consistent with the historical normal behavior pattern. The corresponding ship is regarded as sailing and docking normally, and its motion trajectory data continues to be collected to update the historical behavior baseline.

[0026] In addition, the steps for performing confidence checks on deviation behavior include: The duration of the target ocean vessel's deviation behavior within the port area is obtained, and the duration of the deviation is compared with a preset confidence threshold for deviation duration. If the duration of the deviation exceeds the confidence threshold for the deviation duration, it will be directly judged as abnormal behavior. If the duration of continuous deviation does not exceed the confidence threshold for deviation duration, the frequency of deviation of the target ocean vessel after entering the port area is counted, and the deviation frequency is compared with the preset frequency confidence threshold. When the frequency of deviation exceeds the frequency confidence threshold, it is directly judged as abnormal behavior; When the deviation frequency does not exceed the frequency confidence threshold, the trajectory trend prediction mechanism is executed for the target ocean vessel; When the trajectory trend prediction mechanism is executed, it starts with the real-time motion feature vector and extracts the trajectory segment before the deviation occurs to construct the prediction function; Predict the sequence of predicted trajectory points of the target ocean-going vessel within the demand forecasting time window based on the prediction function; The predicted trajectory point sequence is cross-validated with the motion constraints. If the validation result does not meet the motion constraints, the deviation behavior is determined to be abnormal behavior; otherwise, it is recorded as occasional deviation behavior.

[0027] In this implementation, when verifying the confidence level of the target vessel's deviation behavior, the system first calculates the duration of the deviation from the first occurrence of the deviation to the current time. This duration is then compared with a preset deviation duration confidence threshold. If the duration exceeds the threshold, it indicates that the target vessel is not experiencing a momentary deviation due to a brief error or external interference, but rather exhibiting a continuous abnormal sailing state, which is directly classified as abnormal behavior. If the duration does not exceed the threshold, the system further calculates the total frequency of various deviation behaviors that have occurred since the target vessel entered the port area, from the moment it entered the monitoring range until the current time. Even if the duration of each deviation is short, it indicates a certain abnormal tendency in the behavior, and is therefore directly judged as abnormal behavior. When the frequency of deviation does not exceed the frequency confidence threshold, the trajectory trend prediction mechanism is activated. When executing the trajectory trend prediction mechanism, the real-time motion feature vector at the current moment is used as the starting point, and trajectory segments from a period of time before the deviation occurred (the specific time length is set as needed) are extracted as historical data samples. A prediction function is constructed based on the historical data samples. The prediction function can reflect the movement trend of the target vessel, and thus can predict the position coordinate sequence of the target vessel within the demand prediction time window, that is, the predicted trajectory point sequence. The expression of the prediction function is: In the formula, Indicates the current speed. This represents the mean acceleration. Indicates the duration of demand forecasting. and This indicates a periodic turning pattern. Represents the Gaussian noise term. For dynamic disturbance coefficients, , and The environmental disturbance weights are calibrated using historical trajectory regression. The predicted trajectory point sequence is then cross-validated against the static geographical constraints and dynamic behavioral constraints in the motion constraints. This checks whether the predicted trajectory points will enter static restricted areas, whether the predicted speed will exceed the speed limit range, and whether the predicted positions of other vessels within the prediction period will be less than the safe distance. If the validation results show that the predicted trajectory point sequence will violate the motion constraints, then the current deviation is determined to be not accidental but an abnormal behavior pattern that will lead to violations. Conversely, if the predicted trajectory meets the motion constraints within the future time window, this deviation is recorded as an accidental deviation and does not trigger an alert. However, the deviation event is stored as historical data in the vessel motion trajectory dataset for subsequent updates to the behavior pattern cluster and reference motion feature vector.

[0028] Please see Figure 2 A port vessel information management system based on image processing, using the aforementioned port vessel information management method based on image processing, includes: The initialization module is used to load and display the port site map, and to mark the location coordinates and identification of all target ocean-going vessels on the map in real time; The trajectory output module is used to collect real-time video streams from the port and perform target detection and tracking based on the video streams to generate the motion trajectory of each ocean-going vessel. The constraint determination module is used to identify and associate the target ocean vessel's identity information and the motion restriction range corresponding to the identity information, and then determine the motion constraint conditions of the target ocean vessel based on the motion restriction range. The trajectory judgment module is used to identify whether the motion trajectory of each ship is compliant in real time based on motion constraints, and to trigger alarm information when a violation is identified. When a normal behavior is identified, the module continuously collects ship motion trajectory data and summarizes it into a ship motion trajectory dataset. The behavior analysis module is used to analyze the behavior patterns of ocean-going vessels based on the ocean-going vessel trajectory dataset, identify and output abnormal behavior patterns of ocean-going vessels, and send route deviation warning signals to the corresponding ocean-going vessels under abnormal behavior patterns.

[0029] The execution process of the above management system is consistent with the process of the aforementioned method, so it will not be repeated here.

[0030] Please see Figure 3 An electronic device, comprising: At least one processor; and memory that is communicatively connected to at least one processor; The memory stores a computer program that can be executed by at least one processor, which enables the at least one processor to perform the aforementioned image processing-based port and ocean vessel information management method.

[0031] The above description is merely a preferred embodiment of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention. Structures, devices, and operating methods not specifically described or explained in this invention are implemented according to conventional methods in the art unless otherwise specified or limited.

Claims

1. A port ship information management method based on image processing, characterized by, include: Load and display the port site map, and mark the location coordinates and identification of all target ocean-going vessels on the map in real time; Real-time acquisition of on-site video streams at the port, and target detection and tracking based on the on-site video streams to generate the movement trajectory of each ocean-going vessel; Identify and associate the target ocean vessel's identity information and the corresponding motion restriction range, and then determine the motion constraints of the target ocean vessel based on the motion restriction range; Based on motion constraints, the system identifies in real time whether the motion trajectory of each vessel is compliant and triggers alarm information when violations are detected. When normal behavior is detected, the system continuously collects vessel motion trajectory data and summarizes it into a vessel motion trajectory dataset. Based on the ocean vessel motion trajectory dataset, behavioral pattern analysis is performed on the ocean vessel motion trajectory to identify and output abnormal behavior patterns of ocean vessels, and a route deviation warning signal is sent to the corresponding ocean vessel under the abnormal behavior pattern.

2. The port vessel information management method based on image processing according to claim 1, characterized in that, The steps of loading and displaying the port site map, and marking the location coordinates and identification of all target ocean-going vessels on the map in real time, include: Acquire map data of the port area and load a port site map including a basic geographic information layer, a dynamic labeling layer for ocean-going vessels, and an alarm area layer; The real-time location coordinates of the ocean-going vessel are mapped to the coordinate system of the port's on-site map, and the corresponding identification icon and real-time movement direction of the ocean-going vessel are overlaid and displayed. Static anchor point coordinates are preset on the basic geographic information layer, and position mapping is achieved through the affine transformation matrix between the geographic coordinate system and the image pixel coordinates.

3. The port vessel information management method based on image processing according to claim 1, characterized in that, The steps for target detection and tracking based on live video streams to generate the motion trajectory of each ocean-going vessel include: The on-site video stream is decoded to extract continuous video frames, and multi-scale ocean vessel target detection is performed in the continuous video frames to identify the bow and stern coordinates of the target ocean vessel. The direction of travel of the target ocean vessel is determined based on the bow and stern coordinates. Combined with the length-to-beam ratio constraint and the prior knowledge of the continuity of the heading angle, the center point of the target ocean vessel in the current frame is determined. Cross-frame target tracking is performed by calculating the Euclidean distance and heading angle deviation of the target ocean vessel between adjacent frames, generating a motion trajectory data sequence containing timestamps, position coordinates, and heading angles.

4. The port vessel information management method based on image processing according to claim 3, characterized in that, After the motion trajectory data sequence is output, trajectory data cleaning and outlier filtering are performed on the motion trajectory data sequence. The specific steps are as follows: The motion trajectory data sequence is segmented and smoothed by using median filtering of position coordinates and heading angle within a sliding time window to reduce trajectory noise caused by video frame jitter or target detection errors. Based on the continuity constraint of ocean vessel motion, the displacement distance and heading changes between adjacent trajectory points are detected; If the displacement distance exceeds the upper limit of displacement or the change in heading exceeds the threshold of angle change, it is determined to be an abnormal trajectory point and interpolation correction is performed. The interpolated motion trajectory data sequence is mapped to the port geographic coordinate system to form a standardized trajectory dataset in the geographic coordinate system.

5. The port vessel information management method based on image processing according to claim 1, characterized in that, The step of determining the motion constraints of the target ocean vessel based on the motion restriction range includes: Based on the vessel's identity information, retrieve the movement restriction range bound to the identity information. The movement restriction range includes at least the static geographical constraint range and the dynamic behavioral constraint range. The scope of static geographic constraints is defined based on port maps and includes the channel areas that ocean-going vessels are permitted to enter, the designated berth areas, and the static restricted areas that are prohibited from entering. The scope of dynamic behavioral constraints is generated based on the identity attributes of the ocean-going vessel and port rules, including the safe distance to be maintained with adjacent ocean-going vessels, the speed limit range within the waterway, and the time limit for staying in the port area; Within the motion restriction range, retrieve the static geographical constraint range and dynamic behavioral constraint range corresponding to the target ocean vessel, and combine them into motion constraint conditions.

6. The port vessel information management method based on image processing according to claim 1, characterized in that, The steps for identifying whether the motion trajectories of each ocean vessel comply with motion constraints in real time include: Acquire standardized trajectory data of currently monitored ocean-going vessels and determine their spatial inclusion relationship with static geographical constraints to identify whether the target ocean-going vessels have engaged in boundary-crossing behavior; If it exists, an out-of-bounds alarm message will be triggered; If it does not exist, the compliance comparison with the scope of dynamic behavior constraints will continue, specifically including: Determine whether the real-time speed is within the speed limit range. If it is not within the speed limit range, it is considered a speed violation; otherwise, it is considered normal driving. Based on the real-time position coordinates of each vessel, the real-time distance between the current vessel and the nearest adjacent vessel is calculated. If the real-time distance is less than the safe distance, it is determined to be a violation of the safe distance; otherwise, it is determined to be normal navigation. Within the berth area, the continuous dwell time of the current ocean vessel is calculated cumulatively. If the dwell time exceeds the limit, it is judged as a violation of berthing time limit; otherwise, it is judged as normal sailing.

7. The port vessel information management method based on image processing according to claim 1, characterized in that, The steps of analyzing the behavior patterns of ocean-going vessels based on the ocean-going vessel trajectory dataset, identifying and outputting abnormal behavior patterns of ocean-going vessels, include: Obtain the historical trajectory data set of the target ocean vessel after it enters the port area from the ocean vessel motion trajectory dataset; Based on the historical trajectory data set, extract the motion feature vector of each historical trajectory; Unsupervised clustering of motion feature vectors yields multiple clusters of similar behavior patterns, where the center vector of each behavior pattern cluster is used to characterize the corresponding reference motion feature vector. The real-time motion feature vector corresponding to the latest real-time motion trajectory of the target ocean vessel entering the port is mapped to each behavior pattern cluster, and the pattern deviation between the real-time motion feature vector and each reference motion feature vector is calculated. Compare the pattern deviation with a preset deviation threshold; If the deviation exceeds the deviation threshold, it indicates that the real-time movement trajectory of the target ocean vessel has deviated. The confidence level of the deviation is then checked. If the confidence level is passed, the deviation is determined to be an abnormal behavior pattern, and an early warning signal is triggered simultaneously. If the deviation of the pattern does not exceed the deviation threshold, it indicates that the real-time movement trajectory of the target ocean vessel conforms to the normal driving mode, and the target ocean vessel is considered to be driving and docking normally in the port.

8. The port vessel information management method based on image processing according to claim 7, characterized in that, The step of performing confidence verification on the deviation behavior includes: The duration of the target ocean vessel's deviation behavior within the port area is obtained, and the duration of the deviation is compared with a preset confidence threshold for deviation duration. If the duration of the deviation exceeds the confidence threshold for the deviation duration, it will be directly judged as abnormal behavior. If the duration of continuous deviation does not exceed the confidence threshold for deviation duration, the frequency of deviation of the target ocean vessel after entering the port area is counted, and the deviation frequency is compared with the preset frequency confidence threshold. When the frequency of deviation exceeds the frequency confidence threshold, it is directly judged as abnormal behavior; When the deviation frequency does not exceed the frequency confidence threshold, the trajectory trend prediction mechanism is executed for the target ocean vessel; When the trajectory trend prediction mechanism is executed, it starts with the real-time motion feature vector and extracts the trajectory segment before the deviation occurs to construct the prediction function; Predict the sequence of predicted trajectory points of the target ocean-going vessel within the demand forecasting time window based on the prediction function; The predicted trajectory point sequence is cross-validated with the motion constraints. If the validation result does not meet the motion constraints, the deviation behavior is determined to be abnormal behavior; otherwise, it is recorded as occasional deviation behavior.

9. A port and ocean vessel information management system based on image processing, characterized in that, The port vessel information management method based on image processing according to any one of claims 1 to 8 includes: The initialization module is used to load and display the port site map, and to mark the location coordinates and identification of all target ocean-going vessels on the map in real time; The trajectory output module is used to collect real-time video streams from the port and perform target detection and tracking based on the video streams to generate the motion trajectory of each ocean-going vessel. The constraint determination module is used to identify and associate the target ocean vessel's identity information and the motion restriction range corresponding to the identity information, and then determine the motion constraint conditions of the target ocean vessel based on the motion restriction range. The trajectory judgment module is used to identify whether the motion trajectory of each ship is compliant in real time based on motion constraints, and to trigger alarm information when a violation is identified. When a normal behavior is identified, the module continuously collects ship motion trajectory data and summarizes it into a ship motion trajectory dataset. The behavior analysis module is used to analyze the behavior patterns of ocean-going vessels based on the ocean-going vessel trajectory dataset, identify and output abnormal behavior patterns of ocean-going vessels, and send route deviation warning signals to the corresponding ocean-going vessels under abnormal behavior patterns.

10. An electronic device, characterized in that, The electronic device includes: At least one processor; and a memory communicatively connected to the at least one processor; The memory stores a computer program that can be executed by the at least one processor, which is then executed by the at least one processor to enable the at least one processor to perform the image processing-based port vessel information management method according to any one of claims 1 to 8.

Images