A port berth management method and system based on video GIS data fusion

Abstract

The present invention discloses a port berth management method and system based on video GIS data fusion. By acquiring the GIS data of the port and the visual monitoring requirements of the berth area, the camera layout position is determined, and a port berth monitoring system is constructed. The video image data and GIS data collected by the system are used to determine the event monitoring area of ​​the port berth area. The port ship entry and exit data is obtained in real time, and the anchored ship events are monitored based on the video recognition algorithm, and compared with the port ship management strategy to determine the dynamic deviation of the ship. When the deviation occurs, an early warning operation is performed to generate a port berth early warning plan. The method improves the intelligence and safety of port berth management.

Description

Technical Field

[0001] The present invention relates to the technical field of port management, and in particular to a port berth management method and system based on video GIS data fusion. Background Art

[0002] Traditional port berth management mainly relies on manual scheduling and simple electronic information systems. However, this method has many problems such as untimely information acquisition, insufficient data accuracy, and monitoring blind spots, which cannot meet the needs of efficient management of modern ports.

[0003] In recent years, with the rapid development of video surveillance technology and geographic information system (GIS) technology, the integration of these two technologies in port berth management has become a new research direction. GIS technology can accurately provide information such as the spatial location and facility layout of the port berth area, while video surveillance technology can obtain dynamic image data of the port berth in real time. The combination of the two can effectively realize real-time monitoring and management of port berths.

[0004] However, the existing port berth management system still has a lot of room for improvement in terms of multi-source data fusion processing, automatic event identification and early warning. For example, how to accurately integrate video data and GIS data to improve the system's monitoring accuracy of ship dynamics; how to use video recognition algorithms to quickly detect events such as ship berthing and anchoring; how to provide effective early warning measures in a timely manner when the actual behavior of the ship deviates from the preset port management strategy, are all the key points and difficulties of current research.

[0005] Therefore, in view of the shortcomings of the existing technology, a port berth management method and system based on video GIS data fusion is proposed to improve the intelligence, precision and real-time level of port berth management, which has important research value and application prospects. Summary of the invention

[0006] In order to solve at least one of the above technical problems, the present invention proposes a port berth management method and system based on video GIS data fusion.

[0007] The first aspect of the present invention provides a port berth management method based on video GIS data fusion, comprising:

[0008] Acquire the GIS data of the port and the visual monitoring demand data of the port berth area, determine the camera layout position of the port berth area according to the GIS data and the visual monitoring demand data, and build a port berth monitoring system;

[0009] Acquiring video image data of a port berth area according to the port berth monitoring system, and determining an event monitoring area of ​​the port berth area according to the video image data and GIS data;

[0010] Acquire port ship entry and exit data in real time, and determine port ship management strategies based on the port ship entry and exit data;

[0011] Determine anchored vessel events in the event monitoring area based on a video recognition algorithm;

[0012] The anchored ship event is compared with the port ship management strategy to determine the ship dynamic deviation between the anchored ship event and the port ship management strategy, and an early warning operation is performed according to the ship dynamic deviation to obtain a port berth early warning plan.

[0013] In this solution, the GIS data of the port and the visual monitoring demand data of the port berth area are obtained, the camera layout positions of the port berth area are determined according to the GIS data and the visual monitoring demand data, and the port berth monitoring system is constructed, specifically:

[0014] Acquire GIS data of the port, the GIS data including location and height information of port buildings and facilities, and construct a spatial three-dimensional model of the port based on the GIS data;

[0015] Selecting a port berth area in the three-dimensional spatial model to determine the port berth area in the three-dimensional spatial model;

[0016] Performing light simulation projection in the port berth area in the three-dimensional spatial model, marking the area in the port that can receive the light emitted from the port berth area as the pre-selected camera installation position, and marking the area that cannot receive the light as the camera installation restricted area;

[0017] Obtaining port monitoring clarity requirement threshold information and camera monitoring clarity data for objects at different distances, and performing secondary selection on the pre-selected camera installable area according to the port monitoring clarity requirement threshold information and the camera monitoring clarity data for objects at different distances to obtain a camera installable position;

[0018] Obtaining visual surveillance demand data for the port berth area, wherein the visual surveillance demand data for the port berth area includes video surveillance coverage requirements and video surveillance blind area ratio restrictions for the port berth area;

[0019] A particle swarm optimization algorithm is introduced, and the visual monitoring demand data of the port berth area is used as the optimization target of the particle swarm optimization algorithm. According to the particle swarm optimization algorithm, a spatial search is performed in the camera installation positions to determine the number and location of cameras in the port berth area. Cameras are deployed in the port berth area according to the number and location of cameras, and a port berth monitoring system is constructed.

[0020] In this solution, the video image data of the port berth area is obtained according to the port berth monitoring system, and the event monitoring area of ​​the port berth area is determined according to the video image data and GIS data, specifically:

[0021] Selecting a preset number of equipment facilities with fixed positions from the port berth area as point reference objects in the port berth area, and obtaining GIS plane positioning information of the point reference objects and GIS plane positioning information of the vertices of the port berth area boundaries;

[0022] Acquire video image data of the port berth area according to the port berth monitoring system, identify point reference objects in the port berth area according to the video image data, mark the point reference objects in the video image data, and obtain the point reference objects in the video;

[0023] Obtain the video position of the point reference object in the video in the video image, compare the GIS plane positioning information of each point reference object with the video position, calculate the relative deviation between the video position of each point reference object and the GIS plane positioning information, and obtain the video picture deviation information;

[0024] Performing deviation analysis on the GIS plane positioning information of the port berth area boundary vertices according to the video screen position deviation information, determining the actual positions of the port berth area boundary vertices in the video image, and drawing the port berth area boundary in the video image data according to the actual positions;

[0025] The port berth area drawn in the video is marked as the event monitoring area of ​​the port berth area.

[0026] In this solution, the real-time acquisition of port ship entry and exit data and the determination of the port ship management strategy based on the port ship entry and exit data are specifically as follows:

[0027] Real-time acquisition of AIS data of ships within the preset range of the port, including the real-time position, name, type and draft of the ship;

[0028] Determine the port entry and exit status, estimated arrival time or estimated departure time of the ship according to the AIS data, and obtain port ship entry and exit data;

[0029] According to the port ship entry and exit data, if it is an outbound ship, determine the outbound route of the outbound ship according to the berth position of the outbound ship;

[0030] If the ship is an incoming ship, obtain basic information of each port berth, the basic information includes the current use status of the berth, the water level depth of the berth, and the type of ships that can be docked at the berth;

[0031] Acquiring priority data of ships entering the port, determining the berthing priority of the ships entering the port according to the priority data, and determining the berthing order of the ships according to the berthing priority;

[0032] According to the port ship entry and exit data and the basic information of the port berths, berth matching is performed for the incoming ship to determine the berth of the ship, and the entry route of the incoming ship is determined according to the berth;

[0033] If the incoming ship cannot match a berth, the incoming ship is marked as a waiting ship, the type of the waiting ship and the berth number that can berth the waiting ship type are obtained, and the available berthing time of the waiting ship is calculated according to the estimated departure time of the ship in each berth number;

[0034] According to the real-time position of the ship waiting to berth, if the real-time position of the ship waiting to berth is greater than the preset distance from the port, a recommended sailing speed is given to the ship waiting to berth according to the real-time position of the ship waiting to berth and the available berthing time;

[0035] If the real-time position of the ship waiting to berth is less than a preset distance from the port, obtain the navigation route information of the port ship in the future preset time period, search for the non-navigation route area according to the navigation route information, recommend the waiting berthing position of the ship waiting to berth according to the non-navigation route area, and obtain the recommended waiting berthing position;

[0036] The outbound routes of outbound ships, the inbound routes of incoming ships, and the recommended sailing speeds or recommended waiting berthing positions of ships waiting to berth if the incoming ships cannot match the berths are integrated to build a port ship management strategy.

[0037] In this solution, the video recognition algorithm is used to determine the anchored ship event in the event monitoring area, specifically:

[0038] Acquire image feature data of ships entering and leaving the port, the image feature data including ship name features, ship number features, ship edge features, size features, and shape features, and construct a ship feature database with the feature data;

[0039] Acquire real-time video image data of the port berth area according to the port berth monitoring system, and extract video frame data of the real-time video image data;

[0040] Extracting features of ships in the event monitoring area in the video frame data according to the Canny edge detection algorithm to obtain ship feature data;

[0041] Comparing and matching the ship characteristic data with a ship characteristic database to identify anchored ships in an event monitoring area;

[0042] Selecting ship feature points from the ship feature data based on the Harris corner detection algorithm, and calculating the optical flow vectors of the ship feature points in the event monitoring area in each video frame using the Lucas-Kanade optical flow method;

[0043] For each ship feature point, the overall displacement of the anchored ship in the event monitoring area in the video frame data is determined by averaging the optical flow vectors of the ship feature points in the event monitoring area in all video frame data;

[0044] The motion trajectory of the anchored ship is drawn according to the overall displacement, and the anchored ship events in the event monitoring area are determined according to the motion trajectory. The anchored ship events include mooring, shifting, leaving, and dropping anchor, and also include the time when the anchored ship events occur.

[0045] In this solution, the anchored ship event is compared with the port ship management strategy to determine the ship dynamic deviation between the anchored ship event and the port ship management strategy, and an early warning operation is performed according to the ship dynamic deviation to obtain a port berth early warning solution, which is specifically:

[0046] Comparing the anchored ship event with the port ship management strategy, determining the ship dynamic deviation between the anchored ship event and the port ship management strategy, wherein the ship dynamic deviation includes the deviation ship, the deviation type, and the deviation degree;

[0047] An early warning operation is performed on the ship according to the dynamic deviation of the ship to obtain a ship early warning plan.

[0048] The second aspect of the present invention also provides a port berth management system based on video GIS data fusion, the system comprising: a memory, a processor, the memory comprising a port berth management method program based on video GIS data fusion, the port berth management method program based on video GIS data fusion when executed by the processor, implements the following steps:

[0049] Acquire the GIS data of the port and the visual monitoring demand data of the port berth area, determine the camera layout position of the port berth area according to the GIS data and the visual monitoring demand data, and build a port berth monitoring system;

[0050] Acquiring video image data of a port berth area according to the port berth monitoring system, and determining an event monitoring area of ​​the port berth area according to the video image data and GIS data;

[0051] Acquire port ship entry and exit data in real time, and determine port ship management strategies based on the port ship entry and exit data;

[0052] Determine anchored vessel events in the event monitoring area based on a video recognition algorithm;

[0053] The anchored ship event is compared with the port ship management strategy to determine the ship dynamic deviation between the anchored ship event and the port ship management strategy, and an early warning operation is performed according to the ship dynamic deviation to obtain a port berth early warning plan.

[0054] The present invention discloses a port berth management method and system based on video GIS data fusion. By acquiring the GIS data of the port and the visual monitoring requirements of the berth area, the camera layout position is determined, and a port berth monitoring system is constructed. The video image data and GIS data collected by the system are used to determine the event monitoring area of ​​the port berth area. The port ship entry and exit data is obtained in real time, and the anchored ship events are monitored based on the video recognition algorithm, and compared with the port ship management strategy to determine the dynamic deviation of the ship. When the deviation occurs, an early warning operation is performed to generate a port berth early warning plan. The method improves the intelligence and safety of port berth management. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1 A flow chart of a port berth management method based on video GIS data fusion according to the present invention is shown;

[0056] Figure 2 A flow chart showing the present invention for determining an event monitoring area for a port berth area;

[0057] Figure 3 The flowchart of the port berth early warning scheme obtained by the present invention is shown;

[0058] Figure 4 A block diagram of a port berth management system based on video GIS data fusion according to the present invention is shown. DETAILED DESCRIPTION

[0059] In order to more clearly understand the above-mentioned purpose, features and advantages of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be noted that the embodiments of the present application and the features in the embodiments can be combined with each other without conflict.

[0060] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Therefore, the protection scope of the present invention is not limited to the specific embodiments disclosed below.

[0061] Figure 1 A flow chart of a port berth management method based on video GIS data fusion according to the present invention is shown.

[0062] like Figure 1As shown, the first aspect of the present invention provides a port berth management method based on video GIS data fusion, comprising:

[0063] S102, obtaining GIS data of the port and visual monitoring demand data of the port berth area, determining the camera layout positions of the port berth area according to the GIS data and the visual monitoring demand data, and constructing a port berth monitoring system;

[0064] S104, acquiring video image data of a port berth area according to the port berth monitoring system, and determining an event monitoring area of ​​the port berth area according to the video image data and GIS data;

[0065] S106, acquiring port ship entry and exit data in real time, and determining a port ship management strategy according to the port ship entry and exit data;

[0066] S108, determining an event of a moored ship in an event monitoring area based on a video recognition algorithm;

[0067] S110, comparing the anchored ship event with the port ship management strategy, determining the ship dynamic deviation between the anchored ship event and the port ship management strategy, performing an early warning operation according to the ship dynamic deviation, and obtaining a port berth early warning plan.

[0068] It should be noted that the camera layout positions in the port berth area are determined through the GIS data of the port berth area and the visual monitoring demand data, and a port berth monitoring system is constructed to ensure that the cameras can effectively cover the key berth areas, reduce monitoring blind spots, achieve comprehensive monitoring, and improve the effectiveness and accuracy of port berth monitoring; through the combination of video images and GIS data, the monitoring range of the berth area can be delineated in real time and accurately in the acquired video image data, and the key areas can be located in a timely manner through automatic identification of the event monitoring area. This integration enables a quick response when an incident occurs, improving the efficiency and accuracy of port incident handling. By determining the event monitoring area and only identifying events in the event monitoring area, the efficiency of event identification can be greatly improved. By acquiring the ship's entry and exit data in real time, the system can grasp the ship's real-time position, entry and exit status and other key information, thereby providing a scientific basis for ship scheduling and formulating reasonable management strategies, which not only improves the utilization rate of berth resources, but also reduces the waiting time of ships in ports, and improves the overall operational efficiency of ports. By determining anchored ship events through video recognition algorithms, by comparing the actual behavior of ships with management strategies, it is possible to quickly discover deviations in ship behavior, determine whether there are abnormal conditions, and issue warnings in a timely manner based on the size of the deviation, effectively improving the initiative of port safety management, helping to prevent the risks brought about by ship deviations from plans, and realizing dynamic warnings and risk control of port operations.

[0069] According to an embodiment of the present invention, the GIS data of the port and the visual monitoring demand data of the port berth area are obtained, the camera layout positions of the port berth area are determined according to the GIS data and the visual monitoring demand data, and the port berth monitoring system is constructed, specifically:

[0070] Acquire GIS data of the port, the GIS data including location and height information of port buildings and facilities, and construct a spatial three-dimensional model of the port based on the GIS data;

[0071] Selecting a port berth area in the three-dimensional spatial model to determine the port berth area in the three-dimensional spatial model;

[0072] Performing light simulation projection in the port berth area in the three-dimensional spatial model, marking the area in the port that can receive the light emitted from the port berth area as the pre-selected camera installation position, and marking the area that cannot receive the light as the camera installation restricted area;

[0073] Obtaining port monitoring clarity requirement threshold information and camera monitoring clarity data for objects at different distances, and performing secondary selection on the pre-selected camera installable area according to the port monitoring clarity requirement threshold information and the camera monitoring clarity data for objects at different distances to obtain a camera installable position;

[0074] Obtaining visual surveillance demand data for the port berth area, wherein the visual surveillance demand data for the port berth area includes video surveillance coverage requirements and video surveillance blind area ratio restrictions for the port berth area;

[0075] A particle swarm optimization algorithm is introduced, and the visual monitoring demand data of the port berth area is used as the optimization target of the particle swarm optimization algorithm. According to the particle swarm optimization algorithm, a spatial search is performed in the camera installation positions to determine the number and location of cameras in the port berth area. Cameras are deployed in the port berth area according to the number and location of cameras, and a port berth monitoring system is constructed.

[0076] It should be noted that by obtaining the GIS data of the port, an accurate three-dimensional spatial model can be constructed to clearly reflect the location and height of the buildings and facilities in the port; the light simulation projection is to regard the port berth area as a light source, and simulate the light irradiation around the port berth area. The position where the light can be received in the port is considered that the camera will not be blocked by the port facilities; by analyzing the camera monitoring clarity requirements and the actual monitoring distance, a secondary selection is made to ensure that the selected camera position can meet the clarity requirements. This step optimizes the layout of the camera, so that the monitoring screen can achieve the expected effect at any distance, improves the quality of video monitoring, and ensures the accurate capture of key events; the particle swarm optimization algorithm is used to optimize the number and location of the camera to ensure that the camera layout plan not only meets the monitoring requirements, but also achieves the optimal layout effect. The application of this intelligent algorithm can reduce the number of cameras and reduce system costs, while improving the comprehensiveness of monitoring coverage and the efficiency of layout, and improving the performance and economy of the monitoring system.

[0077] Figure 2 A flow chart of determining an event monitoring area of ​​a port berth area according to the present invention is shown.

[0078] According to an embodiment of the present invention, the video image data of the port berth area is obtained according to the port berth monitoring system, and the event monitoring area of ​​the port berth area is determined according to the video image data and GIS data, specifically:

[0079] S202, selecting a preset number of equipment facilities with fixed positions from the port berth area as point reference objects of the port berth area, and obtaining GIS plane positioning information of the point reference objects and GIS plane positioning information of the vertices of the port berth area boundaries;

[0080] S204, acquiring video image data of a port berth area according to the port berth monitoring system, identifying point reference objects in the port berth area according to the video image data, marking the point reference objects in the video image data, and obtaining point reference objects in the video;

[0081] S206, obtaining the video position of the point reference object in the video in the video image, comparing the GIS plane positioning information of each point reference object with the video position, calculating the relative deviation between the video position of each point reference object and the GIS plane positioning information, and obtaining the video picture deviation information;

[0082] S208, performing deviation analysis on the GIS plane positioning information of the port berth area boundary vertices according to the video screen position deviation information, determining the actual positions of the port berth area boundary vertices in the video image, and drawing the port berth area boundary in the video image data according to the actual positions;

[0083] S210, marking the port berth area drawn in the video as an event monitoring area of ​​the port berth area.

[0084] It should be noted that, since the camera equipment of the port berth monitoring system monitors the port berth area at a certain angle and height, the position of the port berth area in the video image will deviate from the GIS plane positioning information. Therefore, by selecting fixed equipment and facilities as point reference objects and obtaining their GIS plane positioning information, the point reference objects are identified and marked using video image data, thereby realizing the synchronous mapping of the video monitoring screen and the actual space; by calculating the relative deviation between the position of the point reference object in the video and its GIS positioning information, the calculation of the deviation information can timely discover the position error in the monitoring, and ensure the high-precision matching of the video monitoring data with the actual geographical location; by analyzing and correcting the position deviation of the video image, the port berth area can be accurately determined. The boundary position of the domain ensures that the monitoring system can clearly and accurately draw the actual boundary of the berth area, effectively avoids the problem of misalignment or incomplete coverage of the monitoring area, and improves the accuracy and comprehensiveness of the monitoring; the port berth area drawn in the video is set as the event monitoring area, which can realize the accurate monitoring of ship activities in a specific area; the port berth area is the overall area including all berths and berth facilities; the event monitoring area is the video screen area drawn in the video data where the ship needs to be monitored; the preset number of equipment and facilities with fixed positions include port buildings, such as control towers, lighthouses or signal towers at the dock, fixed berth facilities, including bollards, etc.; the relative deviation includes angle deviation, distance deviation in the X direction and distance deviation in the Y direction.

[0085] According to an embodiment of the present invention, the real-time acquisition of port ship entry and exit data and the determination of the port ship management strategy according to the port ship entry and exit data are specifically as follows:

[0086] Real-time acquisition of AIS data of ships within the preset range of the port, including the real-time position, name, type and draft of the ship;

[0087] Determine the port entry and exit status, estimated arrival time or estimated departure time of the ship according to the AIS data, and obtain port ship entry and exit data;

[0088] According to the port ship entry and exit data, if it is an outbound ship, determine the outbound route of the outbound ship according to the berth position of the outbound ship;

[0089] If the ship is an incoming ship, obtain basic information of each port berth, the basic information includes the current use status of the berth, the water level depth of the berth, and the type of ships that can be docked at the berth;

[0090] Acquiring priority data of ships entering the port, determining the berthing priority of the ships entering the port according to the priority data, and determining the berthing order of the ships according to the berthing priority;

[0091] According to the port ship entry and exit data and the basic information of the port berths, berth matching is performed for the incoming ship to determine the berth of the ship, and the entry route of the incoming ship is determined according to the berth;

[0092] If the incoming ship cannot match a berth, the incoming ship is marked as a waiting ship, the type of the waiting ship and the berth number that can berth the waiting ship type are obtained, and the available berthing time of the waiting ship is calculated according to the estimated departure time of the ship in each berth number;

[0093] According to the real-time position of the ship waiting to berth, if the real-time position of the ship waiting to berth is greater than the preset distance from the port, a recommended sailing speed is given to the ship waiting to berth according to the real-time position of the ship waiting to berth and the available berthing time;

[0094] If the real-time position of the ship waiting to berth is less than a preset distance from the port, obtain the navigation route information of the port ship in the future preset time period, search for the non-navigation route area according to the navigation route information, recommend the waiting berthing position of the ship waiting to berth according to the non-navigation route area, and obtain the recommended waiting berthing position;

[0095] The outbound routes of outbound ships, the inbound routes of incoming ships, and the recommended sailing speeds or recommended waiting berthing positions of ships waiting to berth if the incoming ships cannot match the berths are integrated to build a port ship management strategy.

[0096] It should be noted that by acquiring AIS data in real time, the system can accurately track the dynamic situation of ships near the port and obtain specific information about the ships, such as type and draft depth; based on the AIS data of the ship, the system can predict the entry and exit time of the ship in real time, ensure that the port can make management decisions in advance, effectively plan the ship's entry, berthing, exit and other operations, avoid congestion or waste of resources, and improve the overall operation efficiency of the port; provide optimized exit route planning for outbound ships, reduce navigation conflicts between ships and other ships, and ensure smooth traffic inside and outside the port; by acquiring real-time information on berths, ensure that appropriate berths are allocated to incoming ships, and avoid ship docking problems caused by berth incompatibility (such as water level depth or type mismatch); reasonably arrange the docking order according to the ship priority, which can effectively reduce the waiting time of high-priority ships and ensure that emergency or special ships (such as large cargo ships or special mission ships) can be handled first; for ships that cannot match berths temporarily, the system provides waiting berthing solutions, including recommended sailing speed and waiting time. Through this scheduling optimization, the risk of waiting ships staying disorderly near the port is reduced; by recommending waiting areas, waiting ships are prevented from occupying busy sailing routes, forming a complete port ship management strategy.

[0097] According to an embodiment of the present invention, the determining of the anchored ship event in the event monitoring area based on the video recognition algorithm is specifically:

[0098] Acquire image feature data of ships entering and leaving the port, the image feature data including ship name features, ship number features, ship edge features, size features, and shape features, and construct a ship feature database with the feature data;

[0099] Acquire real-time video image data of the port berth area according to the port berth monitoring system, and extract video frame data of the real-time video image data;

[0100] Extracting features of ships in the event monitoring area in the video frame data according to the Canny edge detection algorithm to obtain ship feature data;

[0101] Comparing and matching the ship characteristic data with a ship characteristic database to identify anchored ships in an event monitoring area;

[0102] Selecting ship feature points from the ship feature data based on the Harris corner detection algorithm, and calculating the optical flow vectors of the ship feature points in the event monitoring area in each video frame using the Lucas-Kanade optical flow method;

[0103] For each ship feature point, the overall displacement of the anchored ship in the event monitoring area in the video frame data is determined by averaging the optical flow vectors of the ship feature points in the event monitoring area in all video frame data;

[0104] The motion trajectory of the anchored ship is drawn according to the overall displacement, and the anchored ship events in the event monitoring area are determined according to the motion trajectory. The anchored ship events include mooring, shifting, leaving, and dropping anchor, and also include the time when the anchored ship events occur.

[0105] It should be noted that by extracting the image features of the ship (such as ship name, number, edge, size and shape, etc.) and building a ship feature database, the system can accurately identify and distinguish different ships, reduce the error of ship identity recognition in the berth management process, and ensure that the information of each ship when entering and leaving the port is accurately recorded and matched; according to the Canny edge detection algorithm, the edge features of the ship in the video frame can be effectively extracted, the clarity of the ship outline can be enhanced, and the outline and boundary of the ship can still be accurately identified in a complex background environment (such as weather changes, light changes), which improves the robustness and accuracy of ship recognition; through the Harris corner detection algorithm and the Lucas-Kanade optical flow method, the system can accurately track the movement changes of the ship's feature points in the video, and analyze the movement trajectory and state of the ship in real time. The calculation of the optical flow vector can reflect the displacement of the ship in each video frame; through the analysis of the ship's motion trajectory, it can accurately determine whether the ship is in a state of berthing, shifting, leaving or anchoring, and record these events and their occurrence time in real time, providing detailed data support for subsequent event analysis and management decisions, which is helpful to improve the efficiency of port safety management and resource scheduling.

[0106] Figure 3 The flowchart of the port berth early warning scheme obtained by the present invention is shown.

[0107] According to an embodiment of the present invention, the anchored ship event is compared with the port ship management strategy, the ship dynamic deviation between the anchored ship event and the port ship management strategy is determined, and an early warning operation is performed according to the ship dynamic deviation to obtain a port berth early warning plan, specifically:

[0108] S302, comparing the anchored ship event with the port ship management strategy to determine the ship dynamic deviation between the anchored ship event and the port ship management strategy, wherein the ship dynamic deviation includes the deviation ship, deviation type, and deviation degree;

[0109] S304, performing an early warning operation on the ship according to the ship dynamic deviation to obtain a ship early warning plan.

[0110] It should be noted that by comparing the actual events of anchored ships (such as berthing, shifting, leaving berth, etc.) with the pre-set port ship management strategy, the system can detect deviations in ship dynamics in real time. This comparison process can quickly identify abnormal ship behaviors, such as ships not berthing or leaving berths at the scheduled time or path, thereby improving the accuracy of port management; by analyzing the specific information of ship dynamic deviations (such as which ship has deviated, the type of deviation such as abnormal anchoring, delayed departure, etc., and the degree of deviation), it can identify the problem in more detail, improve the accuracy of abnormal ship event identification, and facilitate port management personnel to take targeted measures to avoid unnecessary impact on other ships and port operations; when it is found that the actual dynamics of the ship deviate from the management strategy, the corresponding warning operation can be triggered according to the deviation; based on the dynamic deviation of the ship, the system generates a specific ship warning plan, such as adjusting the order of ship berthing, adjusting the navigation route, and reminding relevant personnel to deal with possible emergencies. Through these early warning measures, port management can take measures in advance to prevent potential problems, avoid affecting normal operations, and improve the safety and operational efficiency of the port; the dynamic deviation of the ship includes whether the actual berthing position of the ship in the event is consistent with the scheduled berth in the strategy, whether the time of the ship entering or leaving the port is in line with the time window set in the strategy, whether the type of ship actually berthed is consistent with the suitable type of berth (such as whether the length and draft of the ship are suitable for the current berthing conditions), checking the berthing order of ships entering the port to confirm whether it complies with the priority sorting set in the system, and analyzing the offset of the anchor point and actual position of the anchored ship to determine whether it deviates from the permitted range.

[0111] Figure 4 A block diagram of a port berth management system based on video GIS data fusion according to the present invention is shown.

[0112] The second aspect of the present invention further provides a port berth management system 4 based on video GIS data fusion, the system comprising: a memory 41, a processor 42, the memory comprising a port berth management method program based on video GIS data fusion, the port berth management method program based on video GIS data fusion when executed by the processor, implements the following steps:

[0113] Acquire the GIS data of the port and the visual monitoring demand data of the port berth area, determine the camera layout position of the port berth area according to the GIS data and the visual monitoring demand data, and build a port berth monitoring system;

[0114] Acquiring video image data of a port berth area according to the port berth monitoring system, and determining an event monitoring area of ​​the port berth area according to the video image data and GIS data;

[0115] Acquire port ship entry and exit data in real time, and determine port ship management strategies based on the port ship entry and exit data;

[0116] Determine anchored vessel events in the event monitoring area based on a video recognition algorithm;

[0117] The anchored ship event is compared with the port ship management strategy to determine the ship dynamic deviation between the anchored ship event and the port ship management strategy, and an early warning operation is performed according to the ship dynamic deviation to obtain a port berth early warning plan.

[0118] The present invention discloses a port berth management method and system based on video GIS data fusion. By acquiring the GIS data of the port and the visual monitoring requirements of the berth area, the camera layout position is determined, and a port berth monitoring system is constructed. The video image data and GIS data collected by the system are used to determine the event monitoring area of ​​the port berth area. The port ship entry and exit data is obtained in real time, and the anchored ship events are monitored based on the video recognition algorithm, and compared with the port ship management strategy to determine the dynamic deviation of the ship. When the deviation occurs, an early warning operation is performed to generate a port berth early warning plan. The method improves the intelligence and safety of port berth management.

[0119] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0120] The units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units; they may be located in one place or distributed on multiple network units; some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0121] In addition, all functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may be separately used as a unit, or two or more units may be integrated into one unit; the above-mentioned integrated units may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0122] A person of ordinary skill in the art can understand that: all or part of the steps of implementing the above method embodiment can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above method embodiment; and the aforementioned storage medium includes: a mobile storage device, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and other media that can store program codes.

[0123] Alternatively, if the above-mentioned integrated unit of the present invention is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present invention can be essentially or partly reflected in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, ROM, RAM, magnetic disks or optical disks.

[0124] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A port berth management method based on video GIS data fusion, characterized in that: The following steps are involved: Acquire the GIS data of the port and the visual monitoring demand data of the port berth area, determine the camera layout position of the port berth area according to the GIS data and the visual monitoring demand data, and build a port berth monitoring system; Acquiring video image data of a port berth area according to the port berth monitoring system, and determining an event monitoring area of ​​the port berth area according to the video image data and GIS data; Acquire port ship entry and exit data in real time, and determine port ship management strategies based on the port ship entry and exit data; Determine anchored vessel events in the event monitoring area based on a video recognition algorithm; Comparing the anchored ship event with the port ship management strategy, determining the ship dynamic deviation between the anchored ship event and the port ship management strategy, performing early warning operations according to the ship dynamic deviation, and obtaining a port berth early warning plan; The obtaining of the port's GIS data and the visual monitoring demand data of the port's berth area, determining the camera layout positions of the port's berth area according to the GIS data and the visual monitoring demand data, and constructing the port's berth monitoring system, specifically includes: Acquire GIS data of the port, the GIS data including location and height information of port buildings and facilities, and construct a spatial three-dimensional model of the port based on the GIS data; Selecting a port berth area in the three-dimensional spatial model to determine the port berth area in the three-dimensional spatial model; Performing light simulation projection in the port berth area in the three-dimensional spatial model, marking the area in the port that can receive the light emitted from the port berth area as the pre-selected camera installation position, and marking the area that cannot receive the light as the camera installation restricted area; Obtaining port monitoring clarity requirement threshold information and camera monitoring clarity data for objects at different distances, and performing secondary selection on the pre-selected camera installable area according to the port monitoring clarity requirement threshold information and the camera monitoring clarity data for objects at different distances to obtain a camera installable position; Obtaining visual surveillance demand data for the port berth area, wherein the visual surveillance demand data for the port berth area includes video surveillance coverage requirements and video surveillance blind area ratio restrictions for the port berth area; A particle swarm optimization algorithm is introduced, and the visual monitoring demand data of the port berth area is used as the optimization target of the particle swarm optimization algorithm. According to the particle swarm optimization algorithm, a spatial search is performed in the camera installation positions to determine the number and location of cameras in the port berth area. Cameras are deployed in the port berth area according to the number and location of cameras, and a port berth monitoring system is constructed.

2. A port berth management method based on video GIS data fusion according to claim 1, characterized in that: The method of acquiring video image data of a port berth area according to the port berth monitoring system and determining an event monitoring area of ​​the port berth area according to the video image data and GIS data is specifically as follows: Selecting a preset number of equipment facilities with fixed positions from the port berth area as point reference objects in the port berth area, and obtaining GIS plane positioning information of the point reference objects and GIS plane positioning information of the vertices of the port berth area boundaries; Acquire video image data of the port berth area according to the port berth monitoring system, identify point reference objects in the port berth area according to the video image data, mark the point reference objects in the video image data, and obtain the point reference objects in the video; Obtain the video position of the point reference object in the video in the video image, compare the GIS plane positioning information of each point reference object with the video position, calculate the relative deviation between the video position of each point reference object and the GIS plane positioning information, and obtain the video picture deviation information; Performing deviation analysis on the GIS plane positioning information of the port berth area boundary vertices according to the video picture deviation information, determining the actual positions of the port berth area boundary vertices in the video image, and drawing the port berth area boundary in the video image data according to the actual positions; The port berth area drawn in the video is marked as the event monitoring area of ​​the port berth area.

3. A port berth management method based on video GIS data fusion according to claim 1, characterized in that: The real-time acquisition of port ship entry and exit data and the determination of the port ship management strategy according to the port ship entry and exit data are specifically as follows: Real-time acquisition of AIS data of ships within the preset range of the port, including the real-time position, name, type and draft of the ship; Determine the port entry and exit status, estimated arrival time or estimated departure time of the ship according to the AIS data, and obtain port ship entry and exit data; According to the port ship entry and exit data, if it is an outbound ship, determine the outbound route of the outbound ship according to the berth position of the outbound ship; If the ship is an incoming ship, obtain basic information of each port berth, the basic information includes the current use status of the berth, the water level depth of the berth, and the type of ships that can be docked at the berth; Acquiring priority data of ships entering the port, determining the berthing priority of the ships entering the port according to the priority data, and determining the berthing order of the ships according to the berthing priority; According to the port ship entry and exit data and the basic information of the port berths, berth matching is performed for the incoming ship, the berth of the ship is determined, and the entry route of the incoming ship is determined according to the berth; If the incoming ship cannot match a berth, the incoming ship is marked as a waiting ship, the type of the waiting ship and the berth number that can berth the waiting ship type are obtained, and the available berthing time of the waiting ship is calculated according to the estimated departure time of the ship in each berth number; According to the real-time position of the ship waiting to berth, if the real-time position of the ship waiting to berth is greater than the preset distance from the port, a recommended sailing speed is given to the ship waiting to berth according to the real-time position of the ship waiting to berth and the available berthing time; If the real-time position of the waiting ship is less than a preset distance from the port, obtain the navigation route information of the port ship in the future preset time period, search for the non-navigation route area according to the navigation route information, recommend the waiting berthing position of the waiting ship according to the non-navigation route area, and obtain the recommended waiting berthing position; The outbound routes of outbound ships, the inbound routes of incoming ships, and the recommended sailing speeds or recommended waiting berthing positions of ships waiting to berth if the incoming ships cannot match the berths are integrated to build a port ship management strategy.

4. The port berth management method based on video GIS data fusion according to claim 1 is characterized in that: The method of determining the anchored ship event in the event monitoring area based on the video recognition algorithm is specifically as follows: Acquire image feature data of ships entering and leaving the port, the image feature data including ship name features, ship number features, ship edge features, size features, and shape features, and construct a ship feature database with the feature data; Acquire real-time video image data of the port berth area according to the port berth monitoring system, and extract video frame data of the real-time video image data; Extracting features of ships in the event monitoring area in the video frame data according to the Canny edge detection algorithm to obtain ship feature data; Comparing and matching the ship characteristic data with a ship characteristic database to identify anchored ships in an event monitoring area; Selecting ship feature points from the ship feature data based on the Harris corner detection algorithm, and calculating the optical flow vectors of the ship feature points in the event monitoring area in each video frame using the Lucas-Kanade optical flow method; For each ship feature point, the overall displacement of the anchored ship in the event monitoring area in the video frame data is determined by averaging the optical flow vectors of the ship feature points in the event monitoring area in all video frame data; The motion trajectory of the anchored ship is drawn according to the overall displacement, and the anchored ship events in the event monitoring area are determined according to the motion trajectory. The anchored ship events include mooring, shifting, leaving, and dropping anchor, and also include the time when the anchored ship events occur.

5. The port berth management method based on video GIS data fusion according to claim 1 is characterized in that: The anchored ship event is compared with the port ship management strategy, the ship dynamic deviation between the anchored ship event and the port ship management strategy is determined, and an early warning operation is performed according to the ship dynamic deviation to obtain a port berth early warning plan, specifically: Comparing the anchored ship event with the port ship management strategy, determining the ship dynamic deviation between the anchored ship event and the port ship management strategy, wherein the ship dynamic deviation includes the deviation ship, the deviation type, and the deviation degree; An early warning operation is performed on the ship according to the dynamic deviation of the ship to obtain a ship early warning plan.

6. A port berth management system based on video GIS data fusion, characterized in that: The port berth management system based on video GIS data fusion includes a storage and a processor, wherein the storage includes a port berth management method program based on video GIS data fusion, and when the port berth management method program based on video GIS data fusion is executed by the processor, the following steps are implemented: Acquire the GIS data of the port and the visual monitoring demand data of the port berth area, determine the camera layout position of the port berth area according to the GIS data and the visual monitoring demand data, and build a port berth monitoring system; Acquiring video image data of a port berth area according to the port berth monitoring system, and determining an event monitoring area of ​​the port berth area according to the video image data and GIS data; Acquire port ship entry and exit data in real time, and determine port ship management strategies based on the port ship entry and exit data; Determine anchored vessel events in the event monitoring area based on a video recognition algorithm; Comparing the anchored ship event with the port ship management strategy, determining the ship dynamic deviation between the anchored ship event and the port ship management strategy, performing early warning operations according to the ship dynamic deviation, and obtaining a port berth early warning plan; The obtaining of the port's GIS data and the visual monitoring demand data of the port's berth area, determining the camera layout positions of the port's berth area according to the GIS data and the visual monitoring demand data, and constructing the port's berth monitoring system, specifically includes: Acquire GIS data of the port, the GIS data including location and height information of port buildings and facilities, and construct a spatial three-dimensional model of the port based on the GIS data; Selecting a port berth area in the three-dimensional spatial model to determine the port berth area in the three-dimensional spatial model; Performing light simulation projection in the port berth area in the three-dimensional spatial model, marking the area in the port that can receive the light emitted from the port berth area as the pre-selected camera installation position, and marking the area that cannot receive the light as the camera installation restricted area; Obtaining port monitoring clarity requirement threshold information and camera monitoring clarity data for objects at different distances, and performing secondary selection on the pre-selected camera installable area according to the port monitoring clarity requirement threshold information and the camera monitoring clarity data for objects at different distances to obtain a camera installable position; Obtaining visual surveillance demand data for the port berth area, wherein the visual surveillance demand data for the port berth area includes video surveillance coverage requirements and video surveillance blind area ratio restrictions for the port berth area; A particle swarm optimization algorithm is introduced, and the visual monitoring demand data of the port berth area is used as the optimization target of the particle swarm optimization algorithm. According to the particle swarm optimization algorithm, a spatial search is performed in the camera installation positions to determine the number and location of cameras in the port berth area. Cameras are deployed in the port berth area according to the number and location of cameras, and a port berth monitoring system is constructed.

Images