A method and system for detecting the safety of ships passing through locks

Abstract

This application relates to the field of ship inspection technology and discloses a method and system for ship lock passage safety inspection. The method includes real-time acquisition and analysis of ship image data to separate the ship target; identification of key feature points characterizing the core morphology and structure of the ship target, and construction of a spatial distribution map containing the relative positional relationships of these key feature points; matching the spatial distribution map with a pre-stored standard ship 3D model library, and obtaining the ship target's passage information based on the matched standard 3D model; and conducting a lock passage safety assessment based on the passage information. By performing in-depth analysis of the ship image data, identifying key feature points and constructing a spatial distribution map, and then matching it with a standard 3D model, accurate ship passage information is obtained. This effectively solves the problem of size and positional information deviations caused by inaccurate extraction of the ship's hull contour in existing technologies, significantly improving the reliability of lock passage safety assessment.

Description

Technical Field

[0001] This application relates to the field of ship inspection technology, and more specifically, to a method and system for ship lock passage safety inspection. Background Technology

[0002] In modern automatic lock detection systems, the traditional approach involves acquiring images of the ship using image acquisition equipment, extracting the ship's two-dimensional contour, estimating the ship's yaw angle using equipment parameters, rotating and correcting the contour to simulate an ideal observation posture, and then converting the two-dimensional contour into actual dimensions through projection calculations. Finally, the dimensions are compared with the lock's rated standards to determine safety. This method is usable under ideal conditions (monopolized cargo ships, uniform lighting, and no parallel ships). However, this traditional method, which relies on the complete ship contour, faces many challenges in practical applications. Engineering vessels and multihull ships often have non-standard structures such as crane booms and probes. These structures may be mistakenly included in the ship's outline, or dynamic adjustments may cause significant differences in the outline of the same ship at different times, resulting in distorted dimensional calculations and inaccurate attitude (such as yaw angle) estimations. When a ship passes through a lock, factors such as water flow and wind may cause continuous, minute attitude adjustments, leading to discrepancies between the captured instantaneous attitude information and the actual situation, affecting the accuracy of rotation correction. Reflective material on the hull and shadows cast by complex structures can easily cause broken, blurred, or noisy outlines, compromising the geometric basis of subsequent calculations. When multiple ships pass through locks in parallel, traditional methods may fail to effectively separate the targets, leading to outline merging or missed detections, resulting in dimensional misjudgments. These problems cause significant deviations in dimensional and position calculations using traditional methods in complex scenarios, easily leading to false alarms or missed detections, seriously affecting the reliability of ship lock passage safety detection. Summary of the Invention

[0003] To address the shortcomings of existing technologies, this application provides a method and system for detecting ship lock passage safety. This method can solve the problem that existing ship lock passage detection methods suffer from inaccurate ship contour extraction and attitude estimation, leading to deviations in size and position information, which in turn affects the reliability of lock passage safety assessment when dealing with complex ship types, dynamic configurations, and multi-objective scenarios.

[0004] Firstly, this application provides a method for detecting the safety of ships passing through locks, including:

[0005] Real-time acquisition and analysis of ship image data to separate ship targets;

[0006] Identify key feature points that characterize the core shape and structure of the ship target, and construct a spatial distribution map containing the relative positional relationships of the key feature points;

[0007] The spatial distribution map is matched with a pre-stored library of standard 3D ship models, and the passage information of the ship target is obtained based on the matched standard 3D model.

[0008] Safety assessment of gate passage based on passage information.

[0009] By performing in-depth analysis of ship image data, identifying key feature points and constructing a spatial distribution map, and then matching it with a standard 3D model, accurate ship passage information can be obtained. This effectively solves the problem of size and position information deviation caused by inaccurate extraction of ship hull contours in existing technologies, and significantly improves the reliability of lock passage safety assessment.

[0010] Furthermore, this application also proposes that the method includes:

[0011] If the analysis shows that the feature parameters of a ship in the ship image data exceed those of any single ship in the standard ship 3D model library, it is determined to be a composite ship target.

[0012] Identify key feature points of composite ship targets and construct a composite spatial distribution map;

[0013] The composite spatial distribution map is matched with the standard ship 3D model library and the pre-stored formation connection rule library, and a virtual formation model is constructed based on the matched standard 3D model and formation connection rules.

[0014] Iteratively optimize the virtual formation model and obtain the passage information of each ship in the composite ship target;

[0015] Safety assessment of gate passage based on passage information.

[0016] By constructing a virtual formation model and iteratively optimizing it, it is possible to effectively identify and process complex vessel targets, ensuring that the passage information of each vessel can still be accurately obtained in complex scenarios where multiple vessels pass through locks in parallel or in formation. This overcomes the limitations of existing technologies in multi-target scenarios, such as target confusion and inaccurate information.

[0017] Furthermore, this application also proposes that the method further includes:

[0018] The standard ship 3D model library includes a core hull model and additional component models. The additional component models include adjustable geometric parameters and their constraint ranges.

[0019] Based on the spatial distribution map or composite spatial distribution map, a preliminary matching is performed with the core hull model and additional component models. The matched core hull model and additional component models are then virtually assembled, and the parameters of the core hull model and additional component models are optimized and adjusted.

[0020] This application enables more precise processing of ships with additional components. Through virtual assembly and parameter optimization, it makes model matching more accurate and effectively solves the problem of detection failure caused by non-standard or complex geometries of ships or auxiliary equipment being in a dynamic configuration state in the prior art.

[0021] Furthermore, this application also proposes that the method includes:

[0022] Adjustable geometric parameters for multiple candidate add-on component models of spatial distribution maps or composite spatial distribution maps;

[0023] The identified key feature points are jointly matched and optimized with the adjustable geometric parameters of multiple candidate add-on component models, and the reprojection error of the joint matching is recorded.

[0024] By comparing the reprojection error and the constraint range of the adjustable geometric parameters of the additional component model, the joint matching result with the smallest reprojection error and the constraint range satisfying the physical constraints is selected.

[0025] By jointly matching and optimizing multiple candidate models, the accuracy and robustness of additional component identification are further improved, ensuring that reliable geometric information can still be obtained under local optical interference caused by the interaction of complex surfaces and light.

[0026] Furthermore, this application also proposes steps for acquiring and analyzing ship image data in real time to separate ship targets, including:

[0027] Acquire consecutive image frames corresponding to ship image data, and identify fixed reference regions from consecutive image frames;

[0028] Track feature points within a fixed reference area and estimate the overall background motion of consecutive image frames based on the movement of the feature points;

[0029] Background motion compensation is performed on consecutive image frames to eliminate the influence of overall background motion. In the consecutive image frames after background motion compensation, the motion vector of the moving foreground object is identified and extracted.

[0030] Cluster analysis is performed on the motion vectors of moving objects in the foreground to identify and separate ship targets.

[0031] By using motion vector analysis and clustering, the interference of background motion on ship target recognition can be effectively eliminated, enabling accurate separation of ship targets and laying the foundation for subsequent key feature point recognition, thus improving the system's adaptability in dynamic environments.

[0032] Furthermore, this application also proposes a step for identifying and separating ship targets by performing cluster analysis on the motion vectors of moving foreground objects:

[0033] Cluster analysis is performed on the motion vectors of moving objects in the foreground based on the similarity of their spatial distribution and motion characteristics to identify and separate ship targets.

[0034] By comprehensively considering the spatial distribution of motion vectors and the similarity of motion characteristics, more accurate ship target clustering and separation are achieved, further improving the accuracy of target identification. In particular, when multiple targets pass through the lock in parallel, target confusion can be effectively avoided.

[0035] Furthermore, this application proposes that the steps for identifying key feature points include:

[0036] For candidate key feature points for ship target identification, features are extracted from the candidate key feature points, and the features are matched with the preset key feature point template to determine the initial key feature points, and the identification confidence of each initial key feature point is obtained.

[0037] For initial key feature points with a confidence level below a preset threshold, the image position of the initial key feature points with a confidence level above a preset threshold is calculated based on the initial key feature points with a confidence level above a preset threshold and the preset spatial geometric relationship between the predefined key feature points.

[0038] The key feature points are obtained by merging the initial key feature points with a confidence level higher than a preset threshold and the calculated image locations.

[0039] By combining confidence assessment and spatial geometric relationship calculation, the problem of incomplete or inaccurate identification of key feature points caused by local interference is effectively compensated, ensuring that a comprehensive and reliable set of key feature points is obtained, thus improving the robustness of feature recognition.

[0040] Furthermore, this application also proposes that the method includes:

[0041] When the number of initial key feature points with a confidence level higher than the preset threshold is insufficient, the motion trajectory of the initial key feature points with a confidence level higher than the preset threshold is obtained.

[0042] Calculate the motion state of the ship target based on its trajectory;

[0043] Based on the motion state and the preset spatial geometric relationship between key feature points, the image position of the initial key feature points with a confidence level lower than a preset threshold is calculated.

[0044] By utilizing the ship's motion trajectory and motion state information, supplementary calculations are performed when key feature point identification is insufficient, further enhancing the reliability of key feature point identification. This is especially suitable for scenarios where ships undergo minor but continuous attitude adjustments during lock passage.

[0045] Furthermore, this application also proposes that the steps for conducting a gate security assessment based on passage information include:

[0046] The passage information is compared with the lock's rated passage size and preset safety margin;

[0047] The passage information includes passage dimensions, real-time attitude information, and location information.

[0048] This application, based on comprehensive and accurate passage information, compares the rated dimensions and safety margins of the lock to achieve precise lock passage safety assessment, effectively avoiding false alarms or omissions caused by inaccurate information, and improving the accuracy and reliability of the assessment.

[0049] Secondly, this application also proposes a ship lock passage safety detection system for performing the above-mentioned ship lock passage safety detection method, the system comprising:

[0050] Image analysis unit: used to acquire and analyze ship image data in real time and separate ship targets;

[0051] Feature recognition unit: used to identify key feature points that characterize the core shape and structure of the ship target, and to construct a spatial distribution map containing the relative positional relationships of the key feature points;

[0052] Model matching unit: used to match the spatial distribution map with a pre-stored library of standard ship 3D models, and obtain the passage information of the ship target based on the matched standard 3D model;

[0053] Safety assessment unit: Used to conduct gate passage safety assessment based on passage information.

[0054] In summary, this application provides a method and system for ship lock passage safety detection. It acquires and analyzes ship image data in real time to separate the ship target, then identifies key feature points of the ship target's core shape and structure and constructs a spatial distribution map. Subsequently, this spatial distribution map is matched with a pre-stored standard 3D ship model library. Based on the matched standard 3D model, the ship target's passage information is obtained, and finally, a lock passage safety assessment is performed based on this information. This effectively solves the problem in existing technologies where the diverse types of ships, complex and variable structures, environmental factors, and interference from multi-target scenes make it difficult to extract accurate geometric information of the ship, thus affecting the reliability of lock passage safety assessments. This application, by identifying key feature points of the ship and constructing its spatial distribution map, avoids the excessive reliance on the complete hull outline in traditional methods, thereby effectively avoiding the impact of non-measurement-essential auxiliary structures, dynamic configurations, and continuous attitude adjustments on the stability of outline and attitude information. Through precise matching with the standard 3D model library, more accurate ship dimensions, real-time attitude, and position information can be obtained, overcoming the shortcomings of existing "rotation + projection" methods with large calculation deviations in complex ship types and dynamic scenes. Therefore, this application can significantly improve the accuracy and reliability of ship lock passage safety assessment, reduce false alarms and false alarms, and improve the efficiency and safety of lock passage. Attached Figure Description

[0055] Figure 1 This is a flowchart illustrating a ship lock passage safety detection method provided in an embodiment of this application.

[0056] Figure 2 This is a schematic diagram of a ship lock passage safety detection system provided in an embodiment of this application.

[0057] Labeling explanation: 210, Image analysis unit; 220, Feature recognition unit; 230, Model matching unit; 240, Security assessment unit. Detailed Implementation

[0058] The technical solutions of this application will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of this application, and not all embodiments. The components of this application described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0059] It should be noted that similar reference numerals and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. Furthermore, in the description of this application, terms such as "first," "second," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0060] In modern inland waterway transportation systems, locks serve as crucial hubs connecting waterways at different water levels, ensuring both navigation efficiency and safety. However, in practical applications, due to the diverse types and complex structures of vessels, as well as interference from environmental factors and multi-target scenarios, existing methods face numerous challenges in extracting accurate vessel geometric information, leading to decreased system reliability. Specifically, existing vessel inspection programs based on "rotation + projection" can work effectively under ideal conditions. However, when the vessel under inspection has non-standard, complex geometry (e.g., with large auxiliary equipment or multi-hull structures), or when its auxiliary equipment is in a dynamically configured state, or when the vessel undergoes minor but continuous attitude adjustments (e.g., yaw angle changes) during lock passage, the program faces the risk of failure. The specific technical problem lies in the possibility that the program may incorrectly include non-measurement-essential auxiliary structures within the hull contour range when identifying the hull profile, or that dynamic configuration and continuous attitude adjustments lead to instability in the contour and attitude information. Furthermore, local optical interference caused by complex surfaces and light interactions, and even target confusion when multiple vessels pass through the lock in parallel, result in severely inaccurate initial hull profile extraction and yaw angle estimation. This directly leads to the inability of subsequent "rotation" and "projection" steps to transform based on accurate geometric data, resulting in significant deviations in the calculated ship size and position information. This makes it impossible to reliably determine whether a ship can safely pass through the lock, leading to false alarms or missed alarms. Consequently, the core functions of the entire automated detection system are severely affected when facing complex ship types and dynamic scenarios.

[0061] Regarding this, firstly, see... Figure 1 This application proposes a method for safety inspection of ships passing through locks, including:

[0062] Real-time acquisition and analysis of ship image data to separate ship targets;

[0063] Identify key feature points that characterize the core shape and structure of the ship target, and construct a spatial distribution map containing the relative positional relationships of the key feature points;

[0064] The spatial distribution map is matched with a pre-stored library of standard 3D ship models, and the passage information of the ship target is obtained based on the matched standard 3D model.

[0065] Safety assessment of gate passage based on passage information.

[0066] By introducing key feature point recognition and spatial distribution map construction, and combining it with a standard ship 3D model library for matching, the passage information of ships can be obtained more accurately, thereby effectively improving the accuracy and reliability of lock passage safety assessment and solving the problem of decreased detection accuracy caused by factors such as complex ship morphology and environmental interference in existing technologies. Ship image data refers to ship image information acquired in real time by cameras or other imaging devices, which can be a continuous video stream or a series of discrete image frames. The ship target is the hull itself and its attached structures that need to be identified and analyzed in the image. Key feature points are points on the ship that are highly identifiable and can characterize the core morphology and structure of the ship, such as the bow, stern, specific locations on the hull, and the top of the bridge. The spatial distribution map is a geometric model constructed based on these key feature points and their relative positional relationships, used to describe the two-dimensional or three-dimensional morphology of the ship. The standard ship 3D model library is a pre-stored database containing various typical ship 3D geometric models and their related parameters. Each model includes the ship's precise dimensions, structural information, and possible passage restrictions. Passage information refers to the key data required for a vessel to pass through a lock, including but not limited to the vessel's real-time dimensions (length, width, height), attitude (pitch, roll, yaw angle), and precise position within the lock. Based on this passage information, combined with the lock's rated passage size and safety margin, it is determined whether the vessel can pass through the lock safely and smoothly.

[0067] In one implementation, high-definition cameras can be deployed in the lock area to continuously capture video streams or image sequences of ships. These images are then analyzed, employing background subtraction techniques. This involves comparing the current image frame with a pre-defined background model to identify moving ship targets in the image. For example, a background image of the lock when no ships are passing through can be pre-captured and used as the background model. When a ship passes through, the real-time captured image is compared with the background model at the pixel level; areas with significant differences are considered foreground ship targets. Subsequently, morphological operations (such as dilation and erosion) and connected component analysis are used to integrate these foreground regions, thus completely separating the ship targets. Deep learning models, such as convolutional neural networks (CNNs), can be used to process the separated ship target images. After training with a large number of ship images, specific key feature points on the ship can be automatically identified, such as specific markers on the bow, stern, sides of the hull, and mast tops. Each identified key feature point is assigned an image coordinate. Then, based on these image coordinates, a two-dimensional or three-dimensional spatial distribution map can be constructed. For example, the Euclidean distance and relative angle between any two key feature points can be calculated to form a set of feature vectors describing the geometry of a ship, i.e., a spatial distribution map.

[0068] In one implementation, the constructed spatial distribution map can be used as query input and compared with each pre-stored model in a standard ship 3D model library. The matching algorithm can employ a feature point matching method, such as the Iterative Closest Point (ICP) algorithm or its variants. This algorithm iterates continuously to find the optimal geometric transformation (including rotation, translation, and scaling) between the spatial distribution map and the standard 3D model to minimize the error between them. When the matching degree reaches a preset threshold, it is considered that the standard 3D model that best matches the current ship target has been found. Once the match is successful, the ship's precise passage information, such as the ship's rated length, width, draft, and geometric center position in standard attitude, can be directly extracted from the standard 3D model. The ship passage information obtained from the standard 3D model (e.g., the ship's real-time length, width, height, and attitude) is compared with the lock's rated passage dimensions (e.g., the lock's effective length, width, and water depth) and a preset safety margin. If any dimensional parameter (such as length or width) or attitude parameter (such as yaw angle) of the vessel exceeds the allowable range of the lock, or if the safe distance between the vessel and the lock walls or gates is insufficient, the system will determine that there is a safety risk and issue an alarm. Conversely, if all parameters are within the safe range, the system will determine that passage is safe.

[0069] This application, by introducing key feature point identification and spatial distribution map construction, and combining it with a standard ship 3D model library for matching, can more accurately obtain ship passage information, thereby effectively improving the accuracy and reliability of lock passage safety assessment. Traditional ship detection methods, such as "rotation + projection" technology, often struggle to accurately extract the ship's contour and attitude information when faced with complex ship shapes, dynamically configured auxiliary equipment, and multi-target scenarios, leading to deviations in subsequent size and position calculations. For example, when a ship carries large auxiliary equipment or its attitude undergoes slight changes, traditional methods may include non-measurement-essential structures within the contour range, or cause contour information distortion due to attitude instability, thus affecting the accuracy of lock passage safety assessment. This application innovatively introduces key feature point identification and spatial distribution map construction. By identifying key feature points characterizing the core shape and structure of the ship and constructing a spatial distribution map containing the relative positional relationships of these key feature points, it can more robustly capture the ship's essential geometric features. Even in cases of complex ship shapes or partial occlusion, it can effectively avoid misjudging non-core structures as the hull contour, thereby overcoming the limitations of traditional methods in contour extraction. This application matches the constructed spatial distribution map with a pre-stored library of standard 3D ship models. This 3D model-based matching method can directly obtain accurate passage information for ships from the matched standard 3D models, including their standard dimensions and attitude parameters. This is more accurate and comprehensive than traditional methods that rely solely on 2D images for size estimation, especially when dealing with changes in ship attitude, providing more reliable 3D geometric data. Because the passage information comes from matching with standard 3D models, its accuracy is far higher than the easily disturbed 2D contour estimation in traditional methods, thus effectively avoiding false alarms or missed alarms and ensuring the safety of ships passing through locks.

[0070] This application further proposes that the above-mentioned methods also include:

[0071] If the analysis shows that the feature parameters of a ship in the ship image data exceed those of any single ship in the standard ship 3D model library, it is determined to be a composite ship target.

[0072] Identify key feature points of composite ship targets and construct a composite spatial distribution map;

[0073] The composite spatial distribution map is matched with the standard ship 3D model library and the pre-stored formation connection rule library, and a virtual formation model is constructed based on the matched standard 3D model and formation connection rules.

[0074] Iteratively optimize the virtual formation model and obtain the passage information of each ship in the composite ship target;

[0075] Safety assessment of gate passage based on passage information.

[0076] Specifically, after acquiring and analyzing ship image data in real time, the feature parameters of the identified ship targets in the images are analyzed in depth. These feature parameters include, but are not limited to, the ship's overall length, width, height, draft, hull outline, and any possible multiple hull structures. If these analyzed feature parameters exceed the preset range or matching threshold of any single ship model in the pre-stored standard ship 3D model library—for example, if a ship's length far exceeds any known standard ship type, or if the image shows obvious signs of multiple hull connections—then the target is determined to be a composite ship target, ensuring accurate identification of non-standard or multi-ship combinations.

[0077] Once identified as a composite vessel target, a more detailed identification of key feature points is performed. These key feature points include not only the core morphological and structural features of each individual vessel or component constituting the composite target, but also their connection points and relative boundary points, comprehensively representing the overall structure and internal composition of the composite target. Based on these identified key feature points, a composite spatial distribution map is constructed, accurately depicting the relative positional relationships between all key feature points in the composite target, thus forming a complete and detailed spatial representation of the composite target. Subsequently, the constructed composite spatial distribution map is jointly matched with a pre-stored standard vessel 3D model library and a pre-stored formation connection rule library. The standard vessel 3D model library contains detailed 3D model data of various individual vessels (such as tugboats, barges, passenger ships, etc.); while the formation connection rule library stores rules on possible connection methods, relative position constraints, and formation configurations between different types of vessels, such as towing formations, pushing formations, and parallel formations. By simultaneously matching the 3D models and formation rules, it is possible to identify which specific standard vessel models constitute the composite target and how they are connected and arranged. Based on the matched standard 3D model and formation connection rules, a virtual formation model is constructed to accurately simulate the physical composition and spatial layout of composite ship targets in reality.

[0078] Based on this, the constructed virtual formation model is iteratively optimized. By continuously adjusting parameters such as the attitude, relative position, and connection points of each component vessel in the virtual formation model, it achieves optimal consistency with the actual observed composite spatial distribution map, thereby improving the model's accuracy. Through iterative optimization, the passage information of each component vessel in the composite vessel target can be obtained more accurately, such as the individual dimensions, attitude, and draft of each vessel, as well as their relative positions in the formation and the overall size and shape of the entire formation. Finally, based on the obtained passage information of each vessel in the composite vessel target, a lock passage safety assessment is conducted. This means that the assessment is no longer based solely on a vague whole, but can take into account the specific circumstances of each component in the composite target, as well as the overall size and characteristics of their combination, thus providing a more refined and accurate safety assessment.

[0079] Through the above technical solution, this application can effectively identify and accurately process formations of multiple vessels or composite vessel targets with non-standard shapes, significantly improving the applicability and robustness of the vessel lock passage safety detection system in complex real-world application scenarios. It avoids evaluation errors caused by simply treating composite targets as single entities, and can obtain detailed passage information for each component vessel within a composite target, enabling more refined safety assessments. Therefore, it can significantly reduce the risk of misjudgment due to complex vessel shapes or formation passage, ensuring the safety of lock passage, improving lock operation efficiency, and providing lock management departments with more reliable decision-making basis.

[0080] This application further proposes that the above method also includes:

[0081] The standard ship 3D model library includes a core hull model and additional component models. The additional component models include adjustable geometric parameters and their constraint ranges.

[0082] Based on the spatial distribution map or composite spatial distribution map, a preliminary matching is performed with the core hull model and additional component models. The matched core hull model and additional component models are then virtually assembled, and the parameters of the core hull model and additional component models are optimized and adjusted.

[0083] The standard ship 3D model library is further subdivided into core hull models and add-on component models. The core hull model typically refers to the ship's main structure, whose form is relatively fixed and forms the basis for ship identification. Add-on component models refer to variable structures attached to the core hull, such as radomes, antennas, cranes, container stacks, and special cargo. These components may have adjustable geometric parameters, such as length, width, height, angle, or position, and these parameters have specific constraints. These constraints define the physically feasible size and configuration limitations of the components. Specifically, based on real-time acquired ship spatial distribution maps or composite spatial distribution maps, a preliminary matching is first performed with the core hull model and add-on component models to quickly identify the closest hull type and possible add-on component types. Once the preliminary matching is complete, the matched core hull model and add-on component model are virtually assembled. Virtual assembly refers to the combination of identified core hull and additional components in a digital environment according to their preset connection relationships and spatial positions. Based on observed ship image data, the parameters of the core hull model and the additional component models are optimized and adjusted to ensure that the virtually assembled model more closely matches the shape and size of the actual ship. For example, the size parameters of the additional components or their relative positions on the core hull can be adjusted to better fit key feature points.

[0084] Through the above technical solutions, this application can significantly improve the accuracy of ship target identification and model matching, especially when dealing with ships with complex structures, variable components, or non-standard configurations. By virtually assembling and optimizing the parameters of the core hull model and additional component models, the ship's passage dimensions and real-time attitude information can be obtained more accurately, thus providing a more reliable and detailed data foundation for subsequent lock passage safety assessments, effectively improving the adaptability and accuracy of ship lock passage safety detection.

[0085] This application further proposes that the above method also includes:

[0086] Adjustable geometric parameters for multiple candidate add-on component models of spatial distribution maps or composite spatial distribution maps;

[0087] The identified key feature points are jointly matched and optimized with the adjustable geometric parameters of multiple candidate add-on component models, and the reprojection error of the joint matching is recorded.

[0088] By comparing the reprojection error and the constraint range of the adjustable geometric parameters of the additional component model, the joint matching result with the smallest reprojection error and the constraint range satisfying the physical constraints is selected.

[0089] Specifically, when performing 3D model matching on ship targets, to more accurately determine the additional component model, it is first necessary to obtain the adjustable geometric parameters of multiple candidate additional component models corresponding to the aforementioned spatial distribution map or composite spatial distribution map. This means that instead of simply trying to fit a single additional component model, multiple variations or configurations of the additional component may be considered, such as additional structures with different lengths, widths, and heights, or installation methods with different angles and positions. These candidate parameters can be pre-stored in a model library or dynamically generated based on preliminary matching results. Further, the identified key feature points are jointly matched and optimized with the adjustable geometric parameters of the multiple candidate additional component models. This process aims to find the optimal correspondence between each candidate additional component model and the actually observed key feature points. Joint matching optimization can be achieved using various techniques such as the Iterative Closest Point (ICP) algorithm and nonlinear least squares optimization to minimize the difference between the model and the observed data, and to record the reprojection error generated by each joint match. Reprojection error refers to the distance error between the points on the 3D model and the corresponding key feature points in the actual image after the points are projected onto the 2D image plane. The smaller the error value, the better the model fits the actual observation data.

[0090] The constraints define the physical limitations on the size, shape, or installation location of the add-on model. For example, the length of a crane boom cannot exceed the actual load-bearing capacity of the ship, or its rotation angle cannot penetrate the hull. By comprehensively considering reprojection error and physical constraints, the joint matching result with the smallest reprojection error and the constraint range satisfying the physical constraints is selected. This means that even if a candidate model has a very small reprojection error, if its geometric parameters exceed the physical constraints, the result will be excluded, thus ensuring that the finally selected add-on model conforms to both observational data and actual physical laws.

[0091] Through the above technical solution, this application can significantly improve the accuracy and reliability of ship add-on component model identification. Especially when dealing with complex or irregular add-ons, it can effectively avoid errors caused by single matching or local optimization, ensuring that the identified add-on component model and its parameters more accurately reflect the actual shape of the ship. As a result, the obtained ship passage information will be more accurate, thereby substantially improving the reliability of lock passage safety assessment and effectively reducing the safety risks caused by inaccurate model identification.

[0092] Furthermore, the steps for acquiring and analyzing ship image data in real time to separate ship targets include:

[0093] Acquire consecutive image frames corresponding to ship image data, and identify fixed reference regions from consecutive image frames;

[0094] Track feature points within a fixed reference area and estimate the overall background motion of consecutive image frames based on the movement of the feature points;

[0095] Background motion compensation is performed on consecutive image frames to eliminate the influence of overall background motion. In the consecutive image frames after background motion compensation, the motion vector of the moving foreground object is identified and extracted.

[0096] Cluster analysis is performed on the motion vectors of moving objects in the foreground to identify and separate ship targets.

[0097] The acquisition of continuous image frames corresponding to ship image data can be achieved in real time by visual sensors installed in the lock area, such as high-definition cameras or infrared cameras, acquiring video streams or image frame sequences. Identifying fixed reference regions from continuous image frames refers to identifying areas in the image scene that remain relatively stationary in space, such as lock walls, fixed structures on the shore, or pre-set calibration markers. These fixed reference regions can be pre-defined through system calibration or automatically identified during image processing by analyzing image content, for example, selecting regions with rich texture and minimal variation between frames. Further, tracking feature points within the fixed reference regions involves extracting a series of discriminative feature points within the identified fixed reference regions using feature point detection algorithms (such as SIFT, SURF, ORB, or FAST corner detectors). Subsequently, feature point tracking algorithms (such as KLT optical flow or feature matching algorithms) are used to track the motion trajectories of these feature points between continuous image frames. Estimating the overall background motion of consecutive image frames based on the movement of feature points refers to calculating the overall motion parameters of the entire image background using a motion estimation model (such as affine transformation, perspective transformation, or rigid body transformation) based on the displacement and changes of these tracked fixed feature points. This overall background motion may be caused by factors such as slight camera shake, wind effects, or vibration of the observation platform.

[0098] Therefore, background motion compensation for consecutive image frames to eliminate the influence of overall background motion refers to performing corresponding geometric transformations (such as translation, rotation, and scaling) on ​​each frame of the image based on the estimated overall background motion parameters, so that the background in the image remains relatively static between consecutive frames. This compensation operation effectively removes dynamic interference from the background. In the consecutive image frames after background motion compensation, identifying and extracting the motion vectors of foreground moving objects means that in the image sequence after the background has stabilized, any significant motion will mainly originate from moving objects in the foreground. At this point, techniques such as inter-frame differencing, optical flow calculation, or background subtraction can be used to identify these foreground moving objects and extract their motion vectors on the image plane. These vectors represent the displacement direction and magnitude of the objects between consecutive frames. Specifically, clustering analysis of the motion vectors of foreground moving objects to identify and separate ship targets involves using the extracted motion vectors of all foreground moving objects as input and analyzing them using clustering algorithms (such as K-means, DBSCAN, Mean-Shift, or spectral clustering). Motion vectors with similar motion directions, velocities, and spatial proximity are usually grouped into the same moving object. This clustering analysis allows motion vectors belonging to the same vessel to be grouped together, enabling accurate identification of individual vessel targets and separation from other moving objects (such as floating objects on the water, birds, etc.).

[0099] The above technical solution effectively addresses background motion caused by factors such as camera shake, environmental wind, or platform vibration in practical applications, significantly improving the accuracy and robustness of ship target separation. Compared to direct target recognition in the original image, this application greatly simplifies the complexity of foreground target extraction and reduces the false recognition rate through background motion compensation. Furthermore, by clustering motion vectors, ship targets among multiple moving objects can be more accurately distinguished and isolated, providing high-quality input data for subsequent refined analysis, thereby improving the reliability of the entire ship lock passage safety detection method.

[0100] Furthermore, the steps of performing cluster analysis on the motion vectors of moving objects in the foreground to identify and separate ship targets include:

[0101] Cluster analysis is performed on the motion vectors of moving objects in the foreground based on the similarity of their spatial distribution and motion characteristics to identify and separate ship targets.

[0102] Spatial distribution similarity of motion vectors refers to the proximity of pixel positions or regions of moving objects in an image frame. For example, motion vectors belonging to the same ship target usually exhibit a continuous or closely spaced distribution in the image space. Motion feature similarity of motion vectors refers to the similarity of dynamic attributes such as motion direction and velocity represented by these motion vectors. For example, points within the same ship target typically have similar motion directions and velocities over a short period. In practical applications, various clustering algorithms can be used to achieve this process, such as K-means, DBSCAN, or hierarchical clustering. These algorithms comprehensively consider the positional information of each motion vector in the image (reflecting spatial distribution similarity) and its corresponding motion parameters (reflecting motion feature similarity). By defining appropriate distance metrics or similarity functions, motion vectors belonging to the same ship target are grouped together, while motion vectors from different ships or background noise are separated.

[0103] This application, by comprehensively considering the spatial distribution similarity and motion characteristic similarity of motion vectors, can more accurately aggregate motion vectors belonging to the same ship target. Specifically, as a whole, the motion vectors of the various parts of a ship are continuously and closely distributed in space, and their motion directions and speeds tend to be consistent. Therefore, spatial distribution similarity can effectively eliminate noise points that are discontinuous or scattered in space; while motion characteristic similarity can distinguish objects that are spatially close but have different motion patterns, such as other moving objects passing by the ship or water ripples. This dual-constraint clustering method enables ship targets to be accurately identified and separated from complex backgrounds and interference. When clustering the motion vectors of foreground moving objects, the inherent physical characteristics of ship targets can be utilized more effectively, namely, their spatial continuity and motion consistency as a whole. As a result, the accuracy and robustness of ship target identification can be significantly improved, reducing misidentification or omissions caused by complex backgrounds, changes in lighting, or the presence of other moving objects, thus providing more reliable basic data for subsequent ship traffic information acquisition and safety assessment.

[0104] This application further proposes that the steps for identifying key feature points include:

[0105] For candidate key feature points for ship target identification, features are extracted from the candidate key feature points, and the features are matched with the preset key feature point template to determine the initial key feature points, and the identification confidence of each initial key feature point is obtained.

[0106] For initial key feature points with a confidence level below a preset threshold, the image position of the initial key feature points with a confidence level above a preset threshold is calculated based on the initial key feature points with a confidence level above a preset threshold and the preset spatial geometric relationship between the predefined key feature points.

[0107] The key feature points are obtained by merging the initial key feature points with a confidence level higher than a preset threshold and the calculated image locations.

[0108] Specifically, when identifying candidate key feature points, various image processing techniques can be employed, such as corner detection algorithms (e.g., Harris corner, Shi-Tomasi corner), Scale Invariant Feature Transform (SIFT), or Speed-Up Robust Feature Transform (SURF), to extract potential, discriminative feature points from ship image data. Subsequently, descriptors, such as Histogram of Gradient Orientation (HOG) or Binary Robust Independent Fundamental Features (BRIEF), are extracted from these candidate key feature points to characterize their local texture and structural information. The extracted features are then matched against a pre-defined key feature point template, which can be pre-trained using a large number of standard ship images and contains feature descriptions of typical key feature points from different ship types and perspectives. The matching process can utilize classifiers such as Euclidean distance, Hamming distance, or Support Vector Machine (SVM) to determine initial key feature points, and a recognition confidence score is calculated for each initial key feature point. This confidence score reflects the reliability of the match or the probability of the feature point's existence.

[0109] The preset threshold can be set according to the actual application scenario and the required recognition accuracy, for example, it can be set to 0.7 or 0.8. When the recognition confidence of an initial key feature point is lower than the preset threshold, it indicates that its recognition result may not be reliable enough. At this time, this application uses those initial key feature points with recognition confidence higher than the preset threshold as reliable reference points. At the same time, it combines the preset spatial geometric relationships between predefined key feature points. For example, key parts of a ship such as the bow, stern, mast, and funnel have relatively fixed position and distance relationships in three-dimensional space. These geometric relationships can be obtained and stored in advance from the ship's three-dimensional model or CAD data. Based on these high-confidence reference points and known geometric relationships, geometric constraints, triangulation, perspective-N-point (PnP) algorithms, or optimization methods can be used to calculate the precise position of those low-confidence initial key feature points in the image. For example, if the mast and bow are known to be high-confidence points, and there is a fixed relative positional relationship between them and a certain low-confidence porthole point, the position of the porthole point can be calculated based on the position of the mast and bow in the image. Finally, all initial key feature points with identification confidence levels higher than a preset threshold are merged with the image locations of low-confidence key feature points obtained through deduction to form a complete and accurate set of key feature points, which is then used to construct a spatial distribution map.

[0110] Through the above technical solution, this application can significantly improve the robustness and accuracy of key feature point identification for ships. Even in cases of poor image quality, occlusion, or complex backgrounds, it can ensure the acquisition of a complete and reliable set of key feature points. This avoids deviations in the construction of ship spatial distribution maps due to inaccurate or missing key feature point identification, thereby improving the accuracy of ship passage information acquisition. Ultimately, this makes lock passage safety assessment more reliable and accurate, effectively reducing safety risks during ship passage through locks.

[0111] This application further proposes, as well as:

[0112] When the number of initial key feature points with a confidence level higher than the preset threshold is insufficient, the motion trajectory of the initial key feature points with a confidence level higher than the preset threshold is obtained.

[0113] Calculate the motion state of the ship target based on its trajectory;

[0114] Based on the motion state and the preset spatial geometric relationship between key feature points, the image position of the initial key feature points with a confidence level lower than a preset threshold is calculated.

[0115] In some cases, after identifying candidate key feature points and determining initial key feature points for a ship target, the number of key feature points with high recognition confidence obtained by matching a preset key feature point template fails to meet the preset minimum requirement; for example, it may be insufficient for reliable estimation based on purely geometric relationships. In such cases, traditional estimation methods based on geometric relationships may fail or have reduced accuracy. For those initial key feature points that, although insufficient in number, still have high recognition confidence, they are tracked through continuous image frames, recording their positional changes over time to form their respective motion trajectories. This can be achieved using tracking algorithms such as Kalman filtering, particle filtering, or optical flow to ensure the smoothness and accuracy of the trajectories. Using the motion trajectories of these high-confidence key feature points, the overall motion trend of the ship is comprehensively analyzed. The motion state of the ship target can include parameters such as its translational velocity, angular velocity, heading, and attitude changes in two-dimensional or three-dimensional space. These motion states can be estimated by fitting, averaging, or using more complex motion models (such as rigid body motion models) to the motion trajectories of multiple key feature points. For example, the least squares method or optimization algorithms can be used to solve for the overall ship motion parameters that best fit the observed trajectory.

[0116] Once the overall motion state of the ship target is obtained, and the pre-defined relative spatial geometric relationships between all key feature points (e.g., their relative positions on the ship's 3D model) are known, even if the recognition confidence of some key feature points is low, the expected positions of these low-confidence key feature points in the current image frame can be predicted or estimated using the overall motion state of the ship. For example, if it is known that the ship is moving at a specific speed and angular velocity, and a certain low-confidence key feature point has a fixed relative position with respect to a high-confidence key feature point, then the image position of the low-confidence key feature point can be estimated based on the current position of the high-confidence key feature point and the motion state of the ship.

[0117] This application overcomes the limitations of relying solely on static geometric relationships for estimation when the number of high-confidence key feature points is insufficient, by introducing the motion state information of the ship target. When direct geometric constraints are insufficient to accurately estimate the position of low-confidence key feature points, the dynamic motion information of the entire ship can provide additional and more robust constraints for the estimation of these points. The motion trajectories of initial key feature points with a confidence level higher than a preset threshold reflect the actual motion of the ship, and the calculated ship motion state can accurately describe the ship's attitude and displacement at the current moment. Combined with the preset spatial geometric relationships between key feature points, even if some key feature points are difficult to identify directly, their positions in the image can be reasonably predicted and corrected based on the overall motion trend of the ship, thereby improving the completeness and accuracy of key feature point identification. This application can effectively solve the problem of incomplete key feature point identification in complex environments or under partial occlusion conditions, significantly improving the robustness and accuracy of ship target identification. Even with limited initial identification data, it can ensure the comprehensive acquisition of key feature points, providing more reliable basic data for subsequent spatial distribution map construction and passage information acquisition, thereby improving the reliability of ship lock passage safety assessment.

[0118] Furthermore, the steps for conducting a gate security assessment based on passage information include:

[0119] The passage information is compared with the lock's rated passage size and preset safety margin;

[0120] The passage information includes passage dimensions, real-time attitude information, and location information.

[0121] Specifically, passage information includes the following aspects: Passage dimensions, which refers to the actual geometric dimensions of a vessel in a specific state, such as its length, width, draft, and height above the waterline. Real-time attitude information refers to the vessel's attitude relative to a reference coordinate system at a given moment, such as its roll, pitch, and bow angles. This information reflects the vessel's stability and tilt in the water, and is crucial for judging the vessel's dynamic behavior within the narrow lock passage. Position information refers to the vessel's real-time geographic coordinates or relative position within the lock area, which helps to accurately track the vessel's movement trajectory and determine whether it has deviated from its predetermined course or entered a danger zone. The real-time vessel passage information is compared with the maximum permissible passage dimensions determined during the lock design, as well as the additional space reserved to ensure safety. The rated passage dimensions are the maximum vessel size that the lock can accommodate, while the preset safety margin is an additional safety distance set based on factors such as vessel motion, water flow effects, and operational errors.

[0122] Therefore, by rigorously comparing this multi-dimensional, real-time passage information with the physical limitations and safety margins of the lock, the safety of vessel passage can be comprehensively and accurately assessed. By clearly defining the specific content of the passage information and comparing it with the lock's rated passage size and preset safety margins, misjudgments caused by incomplete information or ambiguous evaluation standards can be effectively avoided, significantly improving the safety of vessel passage and reducing the risk of accidents. This meticulous evaluation method helps lock managers make more informed decisions, ensuring the safe and efficient passage of vessels through the lock.

[0123] Secondly, see Figure 2 This application also provides a ship lock passage safety detection system for performing the above-mentioned ship lock passage safety detection method. The system includes:

[0124] Image analysis unit 210: Used to acquire and analyze ship image data in real time and separate ship targets;

[0125] Feature recognition unit 220: used to identify key feature points that characterize the core shape and structure of the ship target, and to construct a spatial distribution map containing the relative positional relationships of the key feature points;

[0126] Model matching unit 230: used to match the spatial distribution map with a pre-stored standard ship 3D model library, and obtain the passage information of the ship target based on the matched standard 3D model;

[0127] Safety assessment unit 240: Used to conduct gate passage safety assessment based on passage information.

[0128] The image analysis unit 210 receives continuous image frames from a camera or other image acquisition device and analyzes these image data using image processing algorithms. For example, it effectively separates the ship target from the background using techniques such as background subtraction and moving target detection. The feature recognition unit 220 further processes the ship target after it has been separated. For example, it applies corner detection, edge detection, or deep learning models to identify specific key points on the ship, such as the bow, stern, hull, and bridge. Subsequently, the relative positional relationships between these identified key points are calculated and organized into a spatial distribution map that reflects the ship's geometry and structural features.

[0129] Model matching unit 230 receives the spatial distribution map generated by the feature recognition unit and compares it with a database containing various standard ship 3D models. Using 3D model matching algorithms, such as the Iterative Closest Point (ICP) algorithm or feature-based matching methods, the system can determine the standard 3D model that best matches the current ship's spatial distribution map. Once a match is successful, the ship passage information contained in the standard 3D model, such as the ship's precise dimensions, draft, width, and height, is extracted. Safety assessment unit 240 receives the ship passage information provided by model matching unit 230 and compares it with the lock's rated passage size and preset safety margin. By comparing, it can determine whether the ship can safely pass through the lock, for example, whether there is a risk of contact with the lock walls, bottom, or top structures.

[0130] The core technical concept of this application lies in completely changing the reliance of traditional ship inspection methods on the complete hull outline. Instead, it is based on the spatial distribution matching of key feature points. By accurately identifying a few geometrically clear and relatively stable landmark points on the ship in the image, these two-dimensional points are used to form a spatial distribution map, which is then directly matched with a pre-stored three-dimensional standard model of the ship. This matching process can calculate the ship's precise three-dimensional attitude and real-time position in the lock in one go, and directly obtain the ship's true core passage dimensions from the successfully matched standard model. This cleverly avoids the difficulties faced by traditional contour extraction and attitude estimation in complex ship shapes, changing additional structures, optical interference, and multi-object scenarios, thereby achieving high-precision and high-reliability ship safety inspection in harsh real-world environments.

[0131] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A method for detecting the safety of ships passing through locks, characterized in that, include: Real-time acquisition and analysis of ship image data to separate ship targets; Identify key feature points that characterize the core shape and structure of the ship target, and construct a spatial distribution map containing the relative positional relationships of the key feature points; The spatial distribution map is matched with a pre-stored standard 3D ship model library, and the passage information of the ship target is obtained based on the matched standard 3D model. A gate passage safety assessment is conducted based on the aforementioned passage information; The method further includes: If the analysis reveals that the feature parameters of a ship in the ship image data exceed those of any single ship in the standard ship 3D model library, it is determined to be a composite ship target. Identify the key feature points of the composite ship target and construct a composite spatial distribution map; The composite spatial distribution map is matched with the standard ship 3D model library and the pre-stored formation connection rule library, and a virtual formation model is constructed based on the matched standard 3D model and the formation connection rules. Iteratively optimize the virtual formation model and obtain the passage information of each ship in the composite ship target; A gate safety assessment is conducted based on the aforementioned passage information.

2. The method for detecting ship safety during lock passage according to claim 1, characterized in that, The method further includes: The standard ship 3D model library includes a core hull model and additional component models, and the additional component models include adjustable geometric parameters and their constraint ranges. Based on the spatial distribution map or the composite spatial distribution map, a preliminary matching is performed with the core hull model and the additional component model. The matched core hull model and the additional component model are then virtually assembled, and the parameters of the core hull model and the additional component model are optimized and adjusted.

3. The method for detecting ship safety during lock passage according to claim 2, characterized in that, The method further includes: Obtain adjustable geometric parameters of multiple candidate additional component models for the spatial distribution map or the composite spatial distribution map; The identified key feature points are jointly matched and optimized with the adjustable geometric parameters of multiple candidate additional component models, and the reprojection error of the joint matching is recorded. By comparing the reprojection error with the constraint range of the adjustable geometric parameters of the additional component model, the joint matching result with the smallest reprojection error and the constraint range satisfying the physical constraints is selected.

4. The method for detecting ship safety during lock passage according to claim 1, characterized in that, The steps of acquiring and analyzing ship image data in real time to separate ship targets include: Obtain consecutive image frames corresponding to the ship image data, and identify a fixed reference region from the consecutive image frames; Track feature points within the fixed reference area, and estimate the overall background motion of the consecutive image frames based on the motion of the feature points; Background motion compensation is performed on the continuous image frames to eliminate the influence of the overall background motion. In the continuous image frames after background motion compensation, the motion vector of the moving foreground object is identified and extracted. Cluster analysis is performed on the motion vectors of the foreground moving object to identify and separate the ship target.

5. The method for detecting ship safety during lock passage according to claim 4, characterized in that, The step of performing cluster analysis on the motion vectors of the foreground moving object to identify and separate the ship target includes: Cluster analysis is performed on the motion vectors of the foreground moving object based on the spatial distribution similarity and motion feature similarity of the motion vectors to identify and separate the ship target.

6. The method for detecting ship safety during lock passage according to claim 1, characterized in that, The steps for identifying key feature points include: For the candidate key feature points for ship target identification, features are extracted from the candidate key feature points, and the features are matched with a preset key feature point template to determine initial key feature points, and the identification confidence of each initial key feature point is obtained. For the initial key feature points whose recognition confidence is lower than a preset threshold, based on the initial key feature points whose recognition confidence is higher than the preset threshold and the preset spatial geometric relationship between the predefined key feature points, the image position of the initial key feature points whose recognition confidence is lower than the preset threshold is calculated. The key feature points are obtained by merging the initial key feature points whose recognition confidence is higher than the preset threshold and the calculated image positions.

7. The method for detecting ship safety during lock passage according to claim 6, characterized in that, The method further includes: When the number of initial key feature points with recognition confidence higher than the preset threshold is insufficient, the motion trajectory of the initial key feature points with recognition confidence higher than the preset threshold is obtained; Calculate the motion state of the ship target based on the motion trajectory; Based on the motion state and the preset spatial geometric relationship between the key feature points, the image position of the initial key feature points whose recognition confidence is lower than the preset threshold is calculated.

8. The method for detecting ship safety during lock passage according to claim 1, characterized in that, The steps for conducting a gate passage safety assessment based on the passage information include: The passage information is compared with the lock's rated passage size and preset safety margin; The passage information includes passage size, real-time attitude information, and location information.

9. A ship lock passage safety detection system, used to perform the ship lock passage safety detection method as described in any one of claims 1-8, characterized in that, The system includes: Image analysis unit: used to acquire and analyze ship image data in real time and separate ship targets; Feature recognition unit: used to identify key feature points characterizing the core shape and structure of the ship target, and to construct a spatial distribution map containing the relative positional relationships of the key feature points; Model matching unit: used to match the spatial distribution map with a pre-stored standard ship 3D model library, and obtain the passage information of the ship target based on the matched standard 3D model; Safety assessment unit: used to perform gate passage safety assessment based on the passage information.

Images