Cargo identification method and system for a ship based on lock video monitoring

Abstract

The application discloses a kind of based on ship lock video monitoring of ship's cargo identification method and system, the present application relates to the technical field of cargo identification method, corresponding sea level is determined based on the detection of the cargo load distribution map of ship, and the compensation height of ship lock is determined according to the sea level, the current opening height of ship lock and the sailing speed of ship, improve the accuracy of the compensation height of ship lock.Therefore, the position deviation amount of cargo feature is determined based on the position data of multiple positions of cargo feature in sailing process, the stress distribution map of cargo feature is determined according to the position deviation amount of cargo feature, stress coefficient and corresponding cargo model;The position activity range of each cargo feature is determined based on the stress distribution map and current position of multiple cargo features, the risk coefficient of each cargo feature is predicted based on the position activity range of each cargo feature and the sailing speed of ship, improve the prediction accuracy of the risk coefficient of each cargo feature.

Description

Technical Field

[0001] This invention relates to the technical field of cargo identification methods, and more particularly to a cargo identification method and system for ships based on lock video monitoring. Background Technology

[0002] With the development of technology, ships navigate on the sea and gradually approach corresponding locks. Ships can only pass through when the locks are open. Multiple cameras in the locks face the ship from different directions and collect navigation videos. In the current technology, multiple navigation attitude features of the ship are determined based on the recognition of the ship's navigation videos, and the ship's navigation status is determined based on these multiple navigation attitude features. However, the characteristics of the cargo on the ship are not identified, resulting in low accuracy in predicting the risk coefficient of each cargo feature. Summary of the Invention

[0003] The purpose of this invention is to overcome the shortcomings of the prior art. This invention provides a method and system for identifying cargo on ships based on lock video monitoring.

[0004] This invention provides a method for cargo identification of ships based on lock video surveillance, comprising: acquiring monitoring video of the ship from the lock; determining the ship's current form and multiple cargo features based on the identification of the monitoring video; determining corresponding cargo models based on the identification of multiple cargo features; determining a cargo load distribution map of the ship based on the storage format of multiple cargo models and the ship's current navigation form; determining the corresponding sea level height based on the detection of the ship's cargo load distribution map; determining a compensation height of the lock based on the sea level height, the current opening height of the lock, and the ship's navigation speed to allow ship passage; determining the positional deviation of each cargo feature based on multiple positional data of the cargo feature during navigation; determining a force distribution map of the cargo feature based on the positional deviation, force coefficient, and corresponding cargo model; determining the positional activity range of each cargo feature based on the force distribution map of multiple cargo features and the current position; and predicting the risk coefficient of each cargo feature based on the positional activity range of each cargo feature and the ship's navigation speed to manage the risk of the cargo features.

[0005] This invention provides a cargo identification system for ships based on lock video surveillance, applied to the aforementioned cargo identification method. The system includes:

[0006] The cargo feature module is used to collect monitoring video of the ship from the lock, and to determine the current shape of the ship and multiple cargo features based on the identification of the monitoring video.

[0007] The cargo load distribution map module is used to determine the corresponding cargo model based on the identification of multiple cargo features, and to determine the cargo load distribution map of the ship based on the storage format of multiple cargo models and the current navigation status of the ship.

[0008] The compensation height module is used to determine the corresponding sea level height based on the detection of the ship's cargo load distribution map, and to determine the compensation height of the lock according to the sea level height, the current opening height of the lock and the ship's sailing speed, so as to allow the ship to pass.

[0009] The force distribution map module is used to determine the positional deviation of each cargo feature based on multiple positional data of the cargo feature during navigation, and to determine the force distribution map of the cargo feature based on the positional deviation, force coefficient and corresponding cargo model.

[0010] The cargo risk management module is used to determine the location and activity range of each cargo feature based on the force distribution map and current location of multiple cargo features, and to predict the risk coefficient of each cargo feature based on the location and activity range of each cargo feature and the ship's sailing speed, so as to manage the risk of cargo features.

[0011] Compared with the prior art, the beneficial effects of the present invention are:

[0012] In this embodiment of the invention, the method determines the corresponding sea level based on the detection of the ship's cargo load distribution map, and determines the compensation height of the lock based on the sea level, the current opening height of the lock, and the ship's speed to allow the ship to pass. A cargo model is introduced, incorporating the storage formats of multiple cargo models and considering the ship's current navigation status, thus improving the accuracy of the ship's cargo load distribution map. Furthermore, by comprehensively considering the sea level, the current opening height of the lock, and the ship's speed, the accuracy of the lock's compensation height is improved, achieving intelligent control of each ship by the lock.

[0013] Therefore, for each cargo feature, the positional deviation of the cargo feature is determined based on multiple positional data during navigation. The force distribution map of the cargo feature is determined based on the positional deviation, force coefficient, and corresponding cargo model. The positional activity range of each cargo feature is determined based on the force distribution maps of multiple cargo features and the current position. The risk coefficient of each cargo feature is predicted based on the positional activity range of each cargo feature and the ship's navigation speed. By introducing the force distribution map of the cargo feature, the positional activity range of each cargo feature and the ship's cargo load distribution map are considered as a whole, which improves the prediction accuracy of the risk coefficient of each cargo feature, so as to manage the risk of cargo features. Attached Figure Description

[0014] Figure 1 This is a flowchart illustrating the cargo identification method for ships based on lock video monitoring in an embodiment of the present invention.

[0015] Figure 2 This is a flowchart illustrating step S11 of the cargo identification method for ships based on lock video monitoring in an embodiment of the present invention.

[0016] Figure 3 This is a flowchart illustrating step S12 of the cargo identification method for ships based on lock video monitoring in an embodiment of the present invention.

[0017] Figure 4 This is a flowchart illustrating step S13 of the cargo identification method for ships based on lock video monitoring in an embodiment of the present invention.

[0018] Figure 5 This is a flowchart illustrating step S14 of the cargo identification method for ships based on lock video monitoring in an embodiment of the present invention.

[0019] Figure 6 This is a flowchart illustrating step S15 of the cargo identification method for ships based on lock video monitoring in an embodiment of the present invention.

[0020] Figure 7 This is a schematic diagram of the structural composition of a ship cargo identification system based on lock video monitoring in an embodiment of the present invention. Detailed Implementation

[0021] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention.

[0022] Please see Figures 1 to 7 A method for identifying ship cargo based on lock video surveillance is proposed and applied to ship cargo identification scenarios. The method includes:

[0023] Step S11: Collect the monitoring video of the ship from the lock, and determine the current form of the ship and multiple cargo features based on the identification of the monitoring video;

[0024] Step S12: Determine the corresponding cargo model based on the identification of multiple cargo features, and determine the cargo load distribution map of the ship based on the storage format of multiple cargo models and the current navigation format of the ship.

[0025] Step S13: Determine the corresponding sea level height based on the detection of the ship's cargo load distribution map, and determine the compensation height of the lock according to the sea level height, the current opening height of the lock, and the ship's sailing speed to allow the ship to pass.

[0026] Step S14: Among each cargo feature, determine the position deviation of the cargo feature based on multiple position data of the cargo feature during the navigation process, and determine the force distribution map of the cargo feature according to the position deviation, force coefficient and corresponding cargo model;

[0027] Step S15: Determine the location range of each cargo feature based on the force distribution map of multiple cargo features and the current location; predict the risk coefficient of each cargo feature based on the location range of each cargo feature and the ship's sailing speed, so as to manage the risk of the cargo features.

[0028] refer to Figure 2 In step S11, the specific steps are as follows:

[0029] S111: When a ship is sailing toward the lock, the video monitoring area is determined based on the ship's position and the lock's position. Multiple cameras located in the lock shoot toward the video monitoring area to determine the lock's monitoring video of the ship. Multiple sub-navigation videos are determined based on the monitoring video, time, and the ship's sailing distance.

[0030] S112: Based on the recognition of each sub-navigation video, determine multiple navigation images of the ship in different directions, determine the three-dimensional model of the ship based on the synthesis of multiple navigation images, and determine the current form of the ship based on the three-dimensional model of the ship and multiple attitude parameters of the ship at different times.

[0031] S113: Mark multiple cargo locations in each sub-navigation video, determine the corresponding cargo storage area based on the multiple cargo locations and the ship's 3D model, and determine multiple cargo features based on the identification of the cargo storage area, with each cargo feature corresponding to a type of cargo.

[0032] In the embodiments of this application, when a ship is sailing toward the lock, the position of the ship and the position of the lock are collected. The ship is equipped with AIS equipment and will broadcast its position, heading, speed and other information. Using the ship and lock position information obtained in the previous step, a reasonable video monitoring range that can cover the ship's journey from its current position to the lock entrance (or a key point, such as the middle of the lock chamber) is calculated. This range needs to take into account: the ship's sailing path and estimated arrival time, the coverage and viewing angle of the camera, and the lead time for monitoring. The monitoring area is usually a three-dimensional spatial area, and its boundary is defined by a geometric shape (such as a cuboid or cylinder), which includes position, direction, length, width and height.

[0033] Activate or adjust multiple cameras deployed on the lock so that their shooting direction is aimed at the identified monitoring area; the purpose of shooting is to obtain a continuous video stream of ships sailing within the monitoring area; the combination of multiple cameras covers a larger area.

[0034] The continuous surveillance video (from multiple cameras) obtained in the previous step is processed and divided into a series of smaller, more manageable and analyzable segments, namely "sub-navigation videos". The segmentation is based on: time and the ship's sailing distance. Each sub-navigation video should be accompanied by its corresponding timestamp and / or the ship's approximate sailing distance information at the beginning / end of the sub-video.

[0035] Further, clear and accurate ship images are extracted from each sub-navigation video (from step S111), and these images should represent the appearance of the ship from different perspectives (directions); representative video frames are selected from each sub-navigation video; keyframes are selected (such as I-frames in compressed videos) or extracted according to specific intervals (such as one frame per second); the ship's position is accurately located in the selected video frames using computer vision algorithms (such as deep learning-based object detection models YOLO, Faster R-CNN, etc.), and image regions containing only the ship and its background (such as the water surface and a small number of shipboard attachments) are extracted (i.e., navigation images). This step requires high precision to avoid background interference.

[0036] Reconstructing a 3D geometric model of a ship using 2D navigation images obtained from different directions typically involves 3D reconstruction techniques in computer vision. The main methods include: using at least two images taken from different perspectives, reconstructing a 3D point cloud by calculating camera pose (extrinsic parameters) and scene structure (intrinsic parameters), usually based on stereo vision principles and algorithms such as Structure from Motion (SfM) or Multi-View Stereo (MVS); simultaneously, inputting multiple navigation images, the pixel coordinates of the ship in each image (obtained through object detection and segmentation), the camera's intrinsic parameters (focal length, principal point, etc., which need to be pre-calibrated), and (if known or estimated) extrinsic parameters (camera pose); outputting a set of 3D point cloud data representing the geometry of the ship's surface, which is further processed to generate a triangular mesh model or a voxel model.

[0037] By combining the reconstructed 3D model with the ship's dynamic attitude information during navigation, a more complete and accurate description of the ship's current form is obtained. Attitude parameters typically include: heading: the direction the ship is moving, the angle relative to true north or magnetic north; pitch: the angle of rotation of the ship around the transverse axis, representing the up-and-down movement of the bow and stern; roll: the angle of rotation of the ship around the longitudinal axis, representing the heeling of the hull; bow / stern trim: the difference between the draft of the bow and stern, usually related to pitch; draft: the depth of the ship submerged in water, usually referring to the average draft or the bow / midship / stern draft.

[0038] Attitude parameters are obtained from AIS data, which typically provides heading information. These parameters are then applied to a 3D model and geometrically transformed (rotated and translated) to align with the ship's actual attitude and position at the current moment. Combined with draft data, the shape and submerged volume of the ship's underwater portion are determined, which is crucial for subsequent calculations of cargo load distribution and risk. The final output, the "current form," is a comprehensive description that includes the ship's precise 3D geometry, spatial position, orientation, and dynamic attitude (such as tilt and draft).

[0039] Therefore, multiple cargo locations are marked in each sub-navigation video, and the corresponding cargo storage area is determined based on the multiple cargo locations and the three-dimensional model of the ship. Multiple cargo features are determined based on the identification of the cargo storage area, and each cargo feature corresponds to a type of cargo. This approach takes into account the overall consideration of multiple cargo locations and the three-dimensional model of the ship, ensuring the accuracy of the corresponding cargo storage area.

[0040] At this point, the approximate location of the cargo loaded on the ship is identified and marked from the sub-navigation videos taken from different angles (from step S111); due to the cargo being stacked, obscured, or similar in color to the ship, this requires a combination of image processing and machine learning techniques.

[0041] Simultaneously, the video frames are processed to remove noise and enhance contrast, improving the accuracy of subsequent recognition. Object detection algorithms (such as Faster R-CNN, variants of YOLO, or models specifically designed for containers, bulk cargo, etc.) are used to find cargo regions in the image. The algorithm needs to be trained to recognize different types of cargo or at least the general category of "cargo". For each detected cargo, its position is marked on the image with a bounding box or a pixel-level mask. At the same time, the sub-voyage video, the frame, and the approximate relative position of the ship (such as bow, midship, stern, port, starboard) are recorded.

[0042] The cargo locations marked in the 2D image in the previous step are mapped onto the ship's 3D model (from step S112) to determine the actual physical area of ​​the ship where the cargo is located. Simultaneously, using the camera's position and angle information from step S111, and the 3D model and view projection relationship established in step S112, the cargo bounding box or mask marked in the image is transformed into the ship's 3D coordinate system. This involves a conversion from image pixel coordinates to world coordinates (ship coordinate system). On the ship's 3D model, the spatial range occupied by the cargo is determined based on the transformed 3D coordinate points. Combining the ship's structural information (such as deck divisions and cabin locations), the continuous spatial area where the cargo is located is defined as a "cargo storage area." This area can be a simple cuboid or a complex polyhedron, depending on the cargo stacking shape and the ship's structure. Each determined cargo storage area is assigned a unique identifier, and its 3D boundaries are recorded.

[0043] After identifying the cargo storage area, further analysis of the cargo characteristics within that area is needed to determine the type of cargo being loaded. This requires considering the shape, size, and location of the storage area, as well as extracting more details from video images. For each cargo storage area, the following features are extracted: Geometric features: volume, aspect ratio, and shape regularity (whether it is a standard rectangular container); Appearance features: images of the cargo in this area are collected from multiple sub-voyage videos, analyzing their color, texture, and whether there are standard markings (such as container numbers, dangerous goods signs, and the way bagged cargo is tied); image classification or feature matching techniques are used; Location features: the specific location of the cargo storage area on the ship (bow / midship / stern, port / midship / starboard) and stacking method (single layer, multi-layer); Dynamic features: if historical video is available, analyze whether the cargo in this area swayed or moved during the voyage.

[0044] The extracted features are combined into a feature vector. A classification model (such as SVM, CNN, or a rule-based system) is used to input the feature vector into the model and output the cargo category corresponding to the cargo storage area (such as "standard container", "bagged grain", "steel", "liquid tank" etc.). This classification model needs to be trained in advance with labeled cargo images and feature data. For each identified cargo category, a corresponding "cargo feature" is defined, which is a structured information containing category name, typical physical properties (such as density, center of gravity height), typical appearance description, etc.

[0045] refer to Figure 3 In step S12, the specific steps are as follows:

[0046] S121: Based on the identification of multiple cargo features, the corresponding cargo model information is determined. At the same time, the shipping database and the ship model information are collected. Based on the shipping database, the cargo model information and the ship model information, the corresponding cargo model is determined.

[0047] S122: Collect real-time images of multiple cargo features, determine the storage form of each cargo model based on the real-time images of multiple cargo features, the corresponding cargo models and the cargo storage area of ​​the ship, and determine the first load distribution map based on the storage form of each cargo model and the position of the cargo model relative to the ship.

[0048] S123: Collect the current navigation mode of the ship, determine multiple sub-navigation modes of the ship based on the decomposition of the current navigation mode of the ship, and determine the cargo load distribution map of the ship based on the multiple sub-navigation modes of the ship, the storage mode of each cargo model and the first load distribution map.

[0049] In the embodiments of this application, computer vision technologies (such as instance segmentation, 3D reconstruction, and texture analysis) are used to obtain more accurate dimensions, shapes, and surface features of the cargo. For example, for containers, it is necessary not only to identify whether they are "container type" but also to estimate their length, width, and height through algorithms such as corner detection and edge fitting, and to determine whether they are 20 feet, 40 feet, refrigerated containers, open-top containers, etc. Combining the known dimensions of the ship and the perspective relationship in the image, the actual physical dimensions of the cargo are estimated. More refined features are extracted, such as color (for bulk cargo), packaging method (for bagged cargo), and surface texture (for steel), which help to distinguish different models under the same category. The extracted refined features are initially matched with a predefined cargo model database to obtain one or more candidate models.

[0050] Information is obtained about the ship itself and the relevant shipping environment; the shipping database usually contains a lot of standard information about different types of cargo, the ship's own structure, load line, stability calculation rules, etc.; the ship's model information is specific to the ship currently being monitored, including its design parameters, structural details, and restrictions on the types of cargo that can be loaded.

[0051] The system needs to access one or more databases, which are stored on local servers, in the cloud, or obtained through API interfaces; the database contents include: standard cargo database, ship design database, and shipping regulations database.

[0052] Collect ship model information, which is usually directly related to the currently monitored ship and comes from the ship's electronic manual, registration information, or real-time sensor data; the content includes: ship type (e.g., 5,000-ton bulk carrier); specific structural information (e.g., deck strength, compartment division, ballast water tank location and capacity); design load capacity (gross deadweight tonnage, load limits for each cargo hold / deck area); stability parameters (e.g., lightship center of gravity, GM value range); and a list of special cargoes that can be loaded (e.g., dangerous goods class restrictions).

[0053] By incorporating a shipping database, cargo model information, and ship model information, a detailed, computationally usable "cargo model" is created for each identified cargo. This model includes not only the cargo's physical shape and weight but also information on how it is loaded and how it interacts with other cargoes and ship structures.

[0054] Simultaneously, combining cargo model information and databases, the precise dimensions, total weight, and center of gravity (X, Y, Z coordinates) of the cargo are determined; for example, the model of a "40-foot refrigerated container" includes its length, width, height, total weight (empty container weight + cargo weight), and the position of its center of gravity in the container coordinate system; based on cargo characteristics and ship information, the stacking restrictions, fastening requirements, and connection methods to the ship structure (such as whether to use lashing straps or clamps) are determined; for example, the model of "bagged corn" includes maximum stacking height limits and recommended fastening methods (such as covering with tarpaulins or installing baffles); the response of the cargo under ship motion is considered; for solid cargo (such as... For containers and steel, the model includes their moments of inertia; for bulk cargo (such as bagged corn), the model needs to consider its flowability and redistribution characteristics under acceleration; combining the cargo storage area determined in S113 and the ship solid model determined in S112, the cargo model is precisely placed at the designated position on the ship model; for example, the "40-foot refrigerated container" model is placed at the designated coordinates in area A of deck, taking into account its orientation; the contact relationship between the cargo and the ship structure (deck, bulkhead) or other cargo is determined; for example, the "square steel" model needs to know which deck area it is placed in and whether it is adjacent to other steel or containers.

[0055] Furthermore, real-time images of multiple cargo features are acquired, and instance segmentation or 3D point cloud processing techniques are used to extract the precise contours or 3D point clouds of the cargo from the real-time images. The extracted contours / point clouds are then matched and their poses estimated (e.g., using the ICP algorithm for point cloud registration) with the corresponding cargo model to calculate the specific pose (rotation matrix and translation vector) of the cargo relative to the storage area coordinate system. The calculated pose, size, stacking status, and other information are combined with the boundary information of the storage area to describe the "storage form." For example, whether a container is placed upright or tilted; whether a pile of bagged goods is neatly stacked or scattered; and how far they are from the edge of the deck.

[0056] At this point, based on the cargo model's center of gravity information and current storage form (attitude), the precise position of the cargo's actual center of gravity in the ship's coordinate system is calculated. The total weight of the cargo is distributed to the corresponding grid cells or structural nodes of the ship according to its contact area or length with the hull. For example, a uniformly distributed cargo weight will be evenly distributed across all grids covered by its bottom area; a concentrated load (such as large equipment) will have its weight primarily distributed to the grids under its support points. The weight distribution results of all cargoes are summed to obtain the total load of each part of the ship, and displayed in the form of a visualization (such as a heat map or contour map). This map is the "first load distribution map". The first load distribution map is usually displayed on the ship's two-dimensional plane (such as the deck plane) or three-dimensional space, showing the weight distribution per unit area or unit length in the form of grids or contour lines.

[0057] Therefore, the current navigation mode of the ship is collected, and multiple sub-navigation modes of the ship are determined based on the decomposition of the current navigation mode. The cargo load distribution map of the ship is determined based on the multiple sub-navigation modes of the ship, the storage mode of each cargo model, and the first load distribution map. This method takes into account the overall consideration of the multiple sub-navigation modes of the ship, the storage mode of each cargo model, and the first load distribution map, thus ensuring the accuracy of the cargo load distribution map of the ship.

[0058] At this point, the overall motion state of the ship at the current moment (or within a short time window) is obtained. This is usually achieved through the ship's own sensors (such as inertial navigation system INS, attitude reference system ARS, GPS, etc.) or by analyzing the ship's motion in video surveillance. Key parameters include: heading: the direction in which the ship is moving; speed: the speed at which the ship is moving; pitch: the angle at which the bow and stern swing up and down; roll: the angle at which the ship heels to the left and right; bow roll: the angle at which the ship rotates left and right around its vertical axis; heave: the vertical movement of the ship up and down.

[0059] Specifically, assuming the ship "Yuanyang Xing" is currently navigating inside the lock, we collected the following current navigation data by connecting to its INS system: heading: 90 degrees (straight upstream); speed: 0.5 knots; pitch: 1.5 degrees (bow slightly low); roll: 2 degrees (listing to starboard); bow roll: 0.5 degrees (slight yaw to port); heave: not obvious (the water flow inside the lock is relatively stable).

[0060] The overall motion of a ship (current navigation mode) is decomposed into several simpler motion patterns that are more basic and easier to analyze, namely sub-navigation modes. Common decomposition methods include: decomposition into translation and rotation: viewing the ship's motion as translation along three axes (longitudinal, transverse, and vertical) and rotation around three axes (transverse, longitudinal, and vertical axes).

[0061] The rolling, pitching, and yaw of a ship cause inertial forces (centrifugal force, Coriolis force, etc.) in the cargo, which alter the pressure distribution of the cargo on the hull. For example, when a ship rolls to starboard, the cargo moves to port, increasing the load on the port side and decreasing the load on the starboard side. Cargo with a high center of gravity experiences greater lateral inertial forces during rolling. Longitudinal and lateral acceleration / deceleration cause the cargo to move back and forth or left and right, and vertical undulation affects the contact pressure of the cargo. The storage form of different cargo models (such as rigid, flexible, bulk, and stacking methods) determines... The system describes how these cargoes respond to ship motion; rigid cargoes (such as containers) generate concentrated forces mainly at the contact points; bulk cargoes (such as grains) will flow internally, redistributing pressure; flexible cargoes (such as rolled materials) will deform; the first load distribution diagram represents the weight distribution of cargoes under static or quasi-static conditions. It introduces multiple sub-navigation modes of the ship, the storage modes of each cargo model, and the first load distribution diagram, and integrates these multiple sub-navigation modes of the ship, the storage modes of each cargo model, and the first load distribution diagram to output the ship's cargo load distribution diagram.

[0062] Specifically, considering the sub-voyage configuration of the "Ocean Star," the previously determined first load distribution map, and the cargo model:

[0063] A1 Container: Roll (±2 degrees) will cause periodically changing pressure at the contact point between the bottom of the container and the deck; when rolling to starboard, the force on the lower left corner increases and the force on the lower right corner decreases; pitch (±1.5 degrees) will also cause changes in the force on the fore and aft corners. These changes are superimposed on the first load distribution diagram, so that the load on the container is no longer a simple four fixed points, but a dynamic load that changes with time.

[0064] B2 Corn Pile: Corn is bulk cargo and will experience internal flow under roll (±2 degrees) and pitch (±1.5 degrees); when rolling to starboard, the corn will move to port, causing the corn pile on the port side to rise and increase pressure, while the pressure on the starboard side will decrease; pitching will cause changes in pressure fore and aft; the circular uniform load area in the first load distribution diagram will deform, the center will shift, and the internal pressure distribution will no longer be uniform; corn near the edge will spill due to shaking, resulting in a reduction in local load;

[0065] C3 square steel: Square steel has good rigidity and is mainly affected by the support points (staples); rolling and pitching will cause additional shear force and bending moment at the staple position; the concentrated load points in the first load distribution diagram will change dynamically due to the ship's movement. For example, during rolling, the force on the port side staple increases and the force on the starboard side staple decreases.

[0066] Taking all the dynamic changes mentioned above into account, a diagram is drawn that reflects the actual stress on the cargo under ship motion. This diagram is no longer a simple static point or area, but rather: For containers: load curves or cloud maps showing the changes over time at the four contact points; For corn stacks: an area that deforms with ship motion and has uneven internal pressure distribution, marked with edge spillage; For square steel: dynamic loads changing over time at the location of the dunnage, and marking areas of stress concentration caused by shear or bending moment. This final diagram is the "ship's cargo load distribution diagram," which comprehensively considers the static weight of the cargo, the inertial forces caused by dynamic motion, and the physical characteristics of the cargo itself, providing the most realistic input for subsequent steps (such as determining the stress distribution diagram and predicting risks).

[0067] refer to Figure 4 In step S13, the specific steps are as follows:

[0068] S131: Based on the detection of the cargo load distribution map of the ship, multiple load areas are determined, the height of the cargo relative to the sea surface is determined according to the spatial distribution of the multiple load areas, and the limit height of the ship relative to the sea surface is output.

[0069] S132: Collect the current image of the lock, and determine the current height of the lock based on the current image of the lock, the current image of the ship, and the cargo load distribution map. At this time, the ship's maximum height relative to the sea surface and the current height of the lock refer to the same sea surface.

[0070] S133: Determine the safe height of the ship based on the ship's maximum height relative to the sea surface and the ship's speed. Determine the compensation height of the lock based on the ship's safe height and the current height of the lock. Determine multiple sub-compensation height zones based on the compensation height of the lock and the distance of the ship relative to the lock. Mark the lock height control measures for each sub-compensation height zone to make step-by-step changes in the lifting and lowering speeds and allow ships to pass.

[0071] In the embodiments of this application, the cargo load distribution map of the ship is detected, and the cargo load distribution map is parsed and analyzed. The system needs to identify the regions in the map that represent different cargoes and their distribution. This involves: clustering closely adjacent spatial points or units that belong to the same whole cargo together; identifying which parts in the map correspond to which cargo based on the cargo model information determined in step S121; dividing the space occupied by the hull and all the cargo on it into several meaningful and relatively independent regions; for example, dividing by cargo type (such as container area, bulk cargo area, deck cargo area), or dividing by hull structure (such as bow, stern, midships, port side, starboard side), or dividing by the degree of load concentration; outputting multiple load regions; each region has a defined boundary, the cargo information it contains, and the total load characteristics (such as total weight, center of gravity position, etc.) within the region.

[0072] Multiple load areas are collected, and the spatial coordinates of each load area are transformed from the ship's coordinate system to an absolute coordinate system based on the sea surface. This requires considering the ship's current draft, heel, and trim. For example, a point located at (x, y, z) in the ship's coordinate system needs to have its height in the absolute coordinate system calculated based on the ship's heel angle and draft. For each load area, the height of its highest point in the absolute coordinate system is calculated, which requires knowing the specific stacking height or structural height of the cargo within that area. For example, for the container area, it is the height of the highest layer of containers stacked; for the bulk cargo area, it is the highest point of the cargo stack. The height of the highest point of each load area relative to the sea surface is then output.

[0073] Input the height of the highest point of each load area relative to the sea surface. Compare the calculated height values ​​of the highest points of all load areas and find the maximum value. This maximum value is the height of the highest point of the ship and all its cargo relative to the sea surface under the current attitude. This height takes into account the cargo distribution, the height of the cargo itself, and the ship's attitude (inclination and draft). Output the ship's limit height relative to the sea surface (a single value).

[0074] By analyzing the cargo load distribution map, the main load areas were identified. Combined with the real-time attitude of the ship, the height of the highest point of each area relative to the sea surface was accurately calculated. Finally, the overall height of the ship's highest point (limit height) was determined. This limit height is a key safety parameter for assessing whether the ship will collide with the top structure of the lock (such as the top of the gate, bridge, or tunnel). For example, if the clearance height at a certain point in the lock is only 20 meters, and the calculated limit height of the ship is 20.5 meters, it indicates a potential collision risk, and subsequent measures need to be taken (such as adjusting the lock water level or height, or requiring the ship to slow down / adjust its attitude).

[0075] Furthermore, current images of the lock are acquired, including images of the lock itself, the vessel, and a cargo load distribution map. Image processing techniques (such as edge detection, template matching, and optical character recognition (OCR) to read water level counts) are used to accurately identify the water surface position from the current lock image, providing the absolute water level height (relative to a fixed reference, such as the lock floor or national elevation datum). Using the vessel's current image, combined with its known position within the lock (determined via GPS, the lock's internal positioning system, or visual positioning), the vessel's image coordinates are aligned with the lock's physical coordinate system. This helps to more accurately understand the vessel's position relative to the lock structure. The height in the cargo load distribution map is usually calculated relative to the vessel's own reference (such as the keel); this reference needs to be converted. To reach an absolute reference level equal to the lock water level, this typically requires knowing the depth of the ship's keel above the water surface (draft). This is obtained through ship images (measuring the distance from the bottom of the ship to the water surface) or ship sensors (such as draft gauges), and also inferred from the cargo load distribution diagram (if the model includes information on the center of gravity and hull deformation). The calculation formula is roughly as follows: Current lock height (absolute) = Absolute water surface height = Absolute keel height + Draft; At the same time, the ship's maximum height (absolute) = Absolute keel height + Maximum height in the cargo load distribution diagram (relative to the keel); Output the absolute height value of the current lock water level relative to a uniform sea surface. This height value is consistent with the calculation reference of the ship's maximum height. At this point, the ship's maximum height relative to the sea surface and the current lock height refer to the same sea surface.

[0076] Specifically, suppose a clear electronic water level display and a scale are installed on the wall of the lock chamber; a monitoring camera is pointed at this area to capture images in real time; the images will show the water level reading (e.g., 5.2 meters) and the specific position of the water level on the scale (e.g., the water level line just passes through the 5.2-meter mark on the scale).

[0077] Assuming that image processing accurately identifies the absolute height of the water surface as 10.0 meters (relative to the lock floor); and that through ship image analysis or draft gauge data, the absolute height of the ship's keel from the lock floor is 7.5 meters (i.e., a draft of 2.5 meters); therefore, the current water level height (absolute) of the lock is 10.0 meters; our previously calculated maximum ship height of 20.5 meters is relative to the keel height; therefore, the absolute height of the ship's highest point from the lock floor is: 7.5 meters (keel height) + 20.5 meters (maximum height) = 28.0 meters; now, we have determined that the current water level height from the lock floor is 10.0 meters, which means that, from the water level, the absolute height of the ship's highest point from the water surface is 28.0 meters - 10.0 meters = 18.0 meters; (Note: the "current lock height" here actually refers to the absolute height of the water surface, or more accurately, the water surface height provides a benchmark for calculating the ship's height relative to the water surface).

[0078] The maximum height of a ship, 20.5 meters, is calculated relative to the keel, while the keel height of 7.5 meters is determined relative to the lock floor (or a fixed reference). The lock water level of 10.0 meters is also determined relative to the same lock floor (or fixed reference). Therefore, when we say the maximum height of a ship relative to the "sea surface" (referring to the plane where the current water level is located), we are referring to the height of the highest point of the ship above the water surface (18.0 meters). This calculation is based on a unified reference, which ensures that the comparison between the ship's height and the lock water level is valid.

[0079] By acquiring real-time images of the lock and the vessel, and combining them with the previously generated cargo load distribution map, the current water level of the lock was accurately determined and converted to an absolute height benchmark that is the same as the vessel's maximum height. This step is crucial because it links the vessel's status information (height) with the lock's environmental information (water level), making subsequent comparisons and decisions (such as determining whether there is a collision risk and calculating the safe height) more accurate. For example, through this step, the system confirmed that the current water level is 10.0 meters and the highest point of the vessel is 18.0 meters above the water surface.

[0080] Therefore, the safe height of a ship is determined based on its maximum height relative to the sea surface and its speed. The compensation height of the lock is then determined based on the ship's safe height and the current height of the lock. Multiple sub-compensation height zones are defined based on the compensation height and the ship's distance from the lock. Lock height control measures are then marked for each sub-compensation height zone to allow for step-by-step changes in lifting and lowering speeds while permitting ship passage. This approach considers both the lock's compensation height and the ship's distance from the lock, ensuring the accuracy of the multiple sub-compensation height zones. Furthermore, it considers the storage configuration of multiple cargo models and the ship's current navigation configuration, improving the accuracy of the ship's cargo load distribution map. By comprehensively considering the sea surface height, the lock's current opening height, and the ship's speed, the accuracy of the lock's compensation height is further enhanced, achieving intelligent control of each ship by the lock.

[0081] At this point, the maximum height of the ship relative to the sea surface is input (from step S131, e.g., 18.0 meters) and the ship's current speed (e.g., 5 knots, or approximately 2.57 meters per second). During navigation, the ship is affected by waves, currents, etc., causing changes in its attitude, particularly pitch (bow and stern swaying up and down) and heave (overall up and down movement). These effects are more pronounced at high speeds. The ship's speed is a crucial factor influencing these dynamic effects. The system needs to predict the maximum dynamic increase in height (e.g., bow lifting in waves) or decrease in height (e.g., bow submerging) that the ship will experience during navigation, based on the ship type, load distribution, current sea state (if available), and speed. This prediction is typically based on empirical models, simulation data, or real-time sensor data (such as from an inertial navigation system).

[0082] Based on the limit height, add the predicted maximum dynamic increase in height, plus a safety margin (e.g., 0.5 meters to 1.5 meters, depending on regulations and requirements) to obtain the ship's safe height. This safe height represents the maximum vertical position that the ship's highest point can reach during navigation, and sufficient space needs to be left. Output the ship's safe height (e.g., 18.0 meters + 1.0 meter dynamic increase + 1.0 meter safety margin = 20.0 meters).

[0083] At the same time, input the ship's safe height (e.g., 20.0 meters) and the current height of the lock (e.g., 10.0 meters, referring to the water level); the lock's required clearance height = the ship's safe height; the lock's compensation height refers to the height the lock needs to adjust (usually the water level) to achieve the required clearance height. This compensation height = required clearance height - current lock height + (the height from the ship's keel to the water surface, if precise calculation of the absolute height difference is required).

[0084] If the current water level of the lock is 10.0 meters and the keel of the ship is 7.5 meters above the water (from the example in step S131), then the highest point of the ship is 10.0 + 7.5 + 18.0 = 35.5 meters above the lock floor (or a fixed reference point). The height of the lock structure (such as the top of the gate) above the same reference point is fixed (assumed to be 40.0 meters). The current clearance is 40.0 - 35.5 = 4.5 meters. However, the required safety clearance is 20.0 meters, which means we need to increase... A clearance of 15.5 meters can be achieved by lowering the ship's height (unrealistic) or raising the lock's water level (if the structure allows and it is safe); assuming the lock design allows for raising the water level, and raising the water level will not cause other problems (such as flooding shore facilities), then the compensation height is the water level that needs to be raised from 10.0 meters to 10.0 + 15.5 = 25.5 meters; therefore, the compensation height is 15.5 meters (the amount by which the water level needs to be raised); the compensation height for the lock's output (e.g., the water level needs to be raised by 15.5 meters).

[0085] Specifically, the maximum height of the vessel is 18.0 meters (relative to the water surface); the system detects that the vessel is currently traveling at a speed of 5 knots; based on the vessel's load distribution (as mentioned earlier, the corn pile is high and close to the stern) and speed, the model predicts that under wave action, the bow will be raised by an additional 1.0 meter; plus a safety margin of 1.0 meter; therefore, the calculated safe height of the vessel is 18.0 + 1.0 + 1.0 = 20.0 meters. This means that in order to ensure that the vessel does not encounter any obstacles when navigating through the lock, the clearance height provided by the lock needs to be at least 20.0 meters.

[0086] The safety height requirement for ships is a clearance of 20.0 meters. Currently, the lock water level is 10.0 meters, the ship's keel is 7.5 meters below the waterline, and the highest point of the ship is 18.0 meters above the keel. Therefore, the distance from the highest point of the ship to a fixed reference point is 10.0 + 7.5 + 18.0 = 35.5 meters. Assuming the top structure of the lock is 40.0 meters from the same reference point, the current clearance is only 4.5 meters, far less than the safety requirement of 20.0 meters. The system calculates that an additional 15.5 meters of clearance is needed. If this is achieved by raising the water level, then the lock's compensation height needs to be increased by 15.5 meters (from 10.0 meters to 25.5 meters). If the lock structure does not allow such a large increase in water level, or if there are other limitations, the system needs to issue a warning indicating that safe passage is not possible, or take other measures (such as requiring the ship to slow down further or adjust its attitude).

[0087] Input the lock's compensation height (e.g., increase by 15.5 meters) and the vessel's distance relative to the lock (updated in real time, e.g., the vessel is 500 meters from the lock chamber entrance, then 300 meters, then 100 meters, and then enters the lock chamber); divide the process of the vessel approaching and passing through the lock into several stages or zones; for example: Zone A: The vessel is far from the lock (e.g., >500 meters); Zone B: The vessel is approaching the lock chamber entrance (e.g., 100-500 meters); Zone C: The vessel is about to enter the lock chamber (e.g., 0-100 meters); Zone D: The vessel is inside the lock chamber.

[0088] The total compensation height (15.5 meters) is allocated to various zones. The allocation principles typically consider: smooth transition: avoid sudden and significant adjustments to the water level when ships approach or enter critical zones to prevent violent waves or currents that could affect ship stability; control priority: ensure that the water level is adjusted to the correct level when ships are closest to or enter the lock chamber to meet safety clearance requirements; system capacity: consider the response speed and capacity of lock pumps, valves, and other systems; for example, it is decided to raise the water level by 5 meters in zone A, 5 meters in zone B, 3.5 meters in zone C, and 2 meters in zone D (ultimately reaching the target water level of 25.5 meters).

[0089] Define specific control measures for each sub-region, mainly the rate and / or magnitude of water level rise and fall; in regions A and B, begin adjusting the water level with a slower, step-like speed; in region C, the adjustment speed needs to be accelerated to ensure that the water level is close to the target when the vessel enters the lock chamber; in region D, based on the vessel's position and attitude within the lock chamber, precisely adjust to the final target water level and maintain stability. These speed changes are "tiered," meaning different preset speed levels are used in different regions. Link the above allocation and control strategies with the corresponding sub-regions to form an execution plan; output: multiple sub-compensation height regions and their corresponding tiered control measures (e.g., region A: 500-300 meters distance, rise 5 meters, speed 0.1 m / min; region B: 300-100 meters distance, rise 5 meters, speed 0.3 m / min; region C: 100-0 meters distance, rise 3.5 meters, speed 0.5 m / min; region D: after entering the lock chamber, rise 2 meters, speed 0.2 m / min, and stabilize).

[0090] Specifically, the safety height requirement for ships is a clearance of 20.0 meters. Currently, the lock water level is 10.0 meters, the ship's keel is 7.5 meters below the water surface, and the highest point of the ship is 18.0 meters above the keel. Therefore, the distance from the highest point of the ship to a fixed reference point is 10.0 + 7.5 + 18.0 = 35.5 meters. Assuming the distance from the top structure of the lock to the same reference point is 40.0 meters, the current clearance is only 4.5 meters, far less than the safety requirement of 20.0 meters. The system calculates that an additional 15.5 meters of clearance is needed. If this is achieved by raising the water level, then the lock's compensation height needs to be increased by 15.5 meters (from 10.0 meters to 25.5 meters). If the lock structure does not allow the water level to rise this much, or if there are other limitations, the system needs to issue a warning indicating that safe passage is not possible, or take other measures (such as requiring the ship to slow down further or adjust its attitude).

[0091] The water level needs to be raised by 15.5 meters. When the vessel is 500 meters from the lock, the system begins to execute the plan: Sub-zone 1 (500-300 meters): The water level begins to rise at a slower rate (e.g., 0.1 m / min), with a planned rise of 5 meters, providing time for system warm-up and initial adjustments; Sub-zone 2 (300-100 meters): The water level rise rate increases to 0.3 m / min, then rises another 5 meters; at this point, the vessel gradually approaches, requiring more aggressive adjustments; Sub-zone 3 (100-0 meters, i.e., approaching the lock chamber entrance): The water level rise rate further increases to 0.5 m / min, rising 3.5 meters. This is the most critical stage, ensuring that the water level is close to the target and the clearance meets the safety requirements (20.0 meters) before the vessel enters the lock chamber. Sub-zone 4 (after entering the lock chamber): As the vessel enters the lock chamber, the water level adjustment speed slows down to 0.2 meters / minute, and then rises by the remaining 2 meters to reach the target water level of 25.5 meters and remains stable, ensuring the vessel is safely moored in the lock chamber. The system monitors the vessel's position in real time and automatically switches to the corresponding control measures when the vessel enters the next sub-zone. In this way, the water level adjustment of the lock not only ensures the final safe clearance but also avoids violent fluctuations through step changes, allowing vessels to pass smoothly.

[0092] refer to Figure 5 In step S14, the specific steps are as follows:

[0093] S141: Real-time monitoring of each cargo feature. Each cargo feature has a position change signal as the ship sails. The position detection of each cargo feature is triggered based on the position change signal, and multiple position data of the cargo feature during the sailing process are collected. Based on the multiple position data and the corresponding cargo feature image, the position offset map of the cargo feature is determined, and the position deviation of the cargo feature is marked.

[0094] S142: Determine the force analysis method for the cargo feature based on its location, the ship's speed, and the ship's current navigation pattern, and trigger the force analysis of the cargo feature along this force analysis method to output the force coefficient of the cargo feature.

[0095] S143: Determine the first sub-force distribution map based on the positional deviation of the cargo feature and the force coefficient, determine the second sub-force distribution map based on the positional deviation of the cargo feature and the corresponding cargo model, and determine the force distribution map of the cargo feature based on the synthesis of the first and second sub-force distribution maps. The force distribution map of the cargo feature is dynamically updated as the cargo feature sails.

[0096] In the embodiments of this application, the system utilizes a lock video monitoring system to continuously observe the various cargo features that have been identified (e.g., containers, bulk cargo stacks, etc. previously identified by S123). As ships experience turbulence and rocking (pitching, rolling, heave, etc.) during navigation, the cargo will undergo slight displacement. The video monitoring system will capture these displacements and form a "position change signal". This signal is a pixel-level movement and is also obtained by comparing the center point or corner point position of the cargo features in consecutive frame images.

[0097] When a significant change in the position of a cargo feature is detected (exceeding a preset threshold), the system will trigger a more precise position detection of that cargo feature. This involves more complex image processing algorithms, such as feature point matching and contour extraction, to determine the current more precise boundary and posture of the cargo.

[0098] During a certain period of time during the ship's voyage (e.g., after a wave cycle or a specific distance), the system records multiple location data points of the cargo features. These data points include timestamps, the cargo's coordinates in the image, and the cargo's actual position and attitude information in the ship's coordinate system, calculated through image recognition and ship modeling.

[0099] The system compares the collected location data with the "reference position" of the cargo feature in a static or initial state; by plotting the changes of these location data over time or ship motion parameters (such as roll angle), a "position offset map" is generated, which visually shows the movement trajectory and range of the cargo during navigation; at the same time, the system calculates and marks the maximum deviation of the cargo feature from the reference position (e.g., the maximum displacement value in the ship's length direction, ship's width direction, and ship's height direction).

[0100] Specifically, suppose we are monitoring a container (cargo feature A) located on a ship's deck; the ship begins its voyage and encounters waves; the camera continuously captures images of container A; in several consecutive frames, we observe a slight change in the pixel coordinates of the bottom edge of container A in the image coordinate system (e.g., from (x1, y1) to (x2, y2)); the system detects that the difference between (x2, y2) and (x1, y1) exceeds a preset threshold for small changes (e.g., 5 pixels), generating a position change signal; the system then activates a more precise detection algorithm, matching the container's corners or edges to determine the precise quadrilateral outline of container A in the image, and converting it to the ship's coordinate system to obtain the actual coordinates of its four corners (X1', Y1', Z1'), (X2', Y2', Z1'), and (X2', Z2'). Z2'),...; In the next 10 seconds, whenever the container's position changes significantly, the system records its corner coordinates; for example, it records the coordinate data at t=1s, t=3s, t=5s, t=7s, and t=10s; the system uses the coordinates at t=0s as a reference; it plots the curves of the coordinates of a corner point (e.g., the lower left corner) of container A over 10 seconds in the ship's length direction (X-axis) and ship's width direction (Y-axis), which is part of the position offset diagram; the system finds that at t=5s, the corner point deviates from the reference position by 0.5 meters in the ship's length direction, and at t=7s, it deviates from the reference position by 0.3 meters in the ship's width direction; finally, it marks the maximum deviation of cargo feature A as: ±0.5 meters in the ship's length direction and ±0.3 meters in the ship's width direction.

[0101] Furthermore, the force analysis method for the cargo features is determined based on the location of the cargo features, the ship's speed, and the ship's current navigation pattern. The force analysis of the cargo features is then triggered along this method to output the force coefficients of the cargo features. This approach takes into account the overall consideration of the location of the cargo features, the ship's speed, and the ship's current navigation pattern, ensuring the accuracy of the force analysis method for the cargo features.

[0102] At this point, the system needs to select the appropriate mechanical model or analysis method based on the specific location of the cargo (e.g., on the deck, in the hold, near the ship's side, or in the middle of the ship), the ship's speed (which affects the magnitude of inertial forces), and the ship's current sailing pattern (e.g., facing the waves, with the waves, or against the waves; the current amplitude and frequency of rolling, pitching, and heave). For example, for containers on the deck, when the rolling is large, the main considerations are gravity, inertial forces (Coriolis force), and wind / wave impact forces, using a simplified six-degree-of-freedom model. For bulk cargo in the hold, the main considerations are gravity and inertial forces caused by the ship's motion (tangential, lateral, and vertical acceleration), and the friction and flow characteristics inside the bulk material need to be considered. When the ship is sailing at high speed, the proportion of inertial forces increases; when berthing at low speed, wind, wave, and current forces are more critical.

[0103] Once the analysis method is determined (for example, selecting a "six-DOF container model that considers roll and pitch"), the system uses current ship motion data (from ship sensors such as gyroscopes and accelerometers, or estimated through video analysis) and cargo position data to run the corresponding physical model or algorithm. This analysis process involves calculating the resultant force and resultant moment acting on the cargo.

[0104] The results of the stress analysis are quantified into one or more "stress coefficients," which represent the strength of the external forces currently acting on the cargo relative to its own weight or some benchmark value. For example, one coefficient represents the ratio of the total external force (or external torque) acting on the cargo to its weight; another coefficient represents the lateral, longitudinal, and vertical stress conditions; and a third coefficient represents the degree of unevenness in the pressure distribution on the bottom surface of the cargo. This coefficient is used to subsequently assess the stability of the cargo, and a larger value usually indicates that the cargo is more unstable.

[0105] Specifically, container A is located slightly to the right of the center of the deck; the ship is currently traveling at 10 knots, encountering moderate cross waves, and the ship's roll angle is periodically changing between ±5 degrees; based on container A's position on the deck (significantly affected by roll), the speed of 10 knots (inertial forces cannot be ignored), and the current roll pattern, the system determines to use a "six-degree-of-freedom model of the container considering the effects of roll and pitch" for analysis. This model calculates gravity and the inertial forces generated by the ship's roll and pitch (including forces caused by tangential and lateral acceleration); the system connects to the ship's sensors to obtain real-time data such as roll angle, pitch angle, roll angular velocity, and pitch angular velocity. The six-degree-of-freedom model is run, taking into account the dimensions and weight of container A (assuming they are known or estimated from images) and its position in the ship's coordinate system. The model calculates the resultant force (magnitude and direction) and resultant moment (about three axes) acting on container A at the current moment. The analysis results are converted into a "comprehensive force coefficient". It is assumed that the model calculation shows that the lateral inertial force currently acting on container A is approximately 0.15 times its weight, and the vertical force fluctuates between ±0.1 times its weight. The system outputs a comprehensive force coefficient, for example, 0.2, which indicates that the current external force (mainly inertial force) has reached a certain intensity relative to the weight of the cargo.

[0106] Therefore, a first sub-force distribution map is determined based on the positional deviation and force coefficient of the cargo feature, and a second sub-force distribution map is determined based on the positional deviation and corresponding cargo model of the cargo feature. The force distribution map of the cargo feature is determined by combining the first and second sub-force distribution maps. This force distribution map of the cargo feature is dynamically updated as the cargo feature sails, taking into account the overall consideration of combining the first and second sub-force distribution maps, thus ensuring the accuracy of the force distribution map of the cargo feature.

[0107] At this point, the first sub-force distribution diagram mainly reflects the "potential displacement distribution caused by external forces"; it combines the previously measured "position deviation" (the actual movement of the cargo) and "force coefficient" (the intensity of the current external force); it can be understood as: if the cargo is subjected to an external force of the current intensity and is allowed to move, where will it tend to move to? This distribution diagram represents the magnitude and direction of the tensile, compressive, or shear forces acting on different parts of the cargo, or represents the uncertainty of the overall movement trend of the cargo; for example, the larger the force coefficient, the larger the position deviation, and the higher the intensity of the potentially dangerous areas (such as edges and corners) on the first sub-force distribution diagram.

[0108] The second sub-force distribution diagram mainly reflects the "internal force distribution caused by the characteristics of the cargo itself"; it combines the "positional deviation" (the current posture and deformation of the cargo) and the "corresponding cargo model" (the physical properties of the cargo, such as material, shape, internal structure, loading method, etc.). This model helps us understand: under the current posture, what stress concentration points exist inside the cargo; for example, if a container is tilted (positional deviation), its bottom corners and side walls will bear greater pressure; if the top of a bulk cargo stack is flattened (positional deviation), the friction and pressure distribution between its internal particles will change; the second sub-force distribution diagram visualizes these internal stress / pressure distributions.

[0109] By superimposing or fusing the first sub-force distribution map (external force effect) and the second sub-force distribution map (internal force effect), the final "force distribution map of cargo characteristics" is obtained. This map integrates the effects of the external environment on the cargo and the influence of the cargo's own state on its internal force distribution, providing a more comprehensive depiction of the current mechanical state of the cargo. Since the ship's motion is dynamic, the cargo's positional deviation and force coefficient are also constantly changing. Therefore, this force distribution map needs to be updated in real time or near real time to reflect the latest mechanical state of the cargo.

[0110] Specifically, assuming that the positional deviation (maximum deviation of 0.5m in the ship's length direction and 0.3m in the ship's width direction) is obtained through S141, and the force coefficient of 0.2 is obtained through S142.

[0111] Determine the first sub-force distribution diagram: The system draws the first sub-force distribution diagram based on deviations of 0.5m and 0.3m, and a force coefficient of 0.2. This diagram shows that the right edge and front corner (along the ship's length) of container A are subjected to greater lateral tensile or shear forces because they deviate the most from the reference position, and the current force coefficient indicates significant lateral inertial forces; these areas in the diagram are indicated by darker colors or higher values ​​to represent the potential degree of danger.

[0112] Determine the second sub-stress distribution diagram: Based on the positional deviation (especially the 0.3m deviation in the ship's beam direction, which means a slight tilt) and the model of container A (assuming it is a standard 40-foot container with uniform weight distribution, but the bottom corners are the main load-bearing points), the system draws the second sub-stress distribution diagram. This diagram shows that the four bottom corners of container A, especially the two bottom corners on the right, bear greater pressure because the tilt increases the gravitational component at these points; the side walls of the container are also subjected to certain bending stress due to the tilt.

[0113] By combining the first and second sub-graphs, the final force distribution diagram will show that the right front corner of container A not only bears a large component of gravity due to its tilt (second sub-graph), but also bears a large shear force due to lateral inertial force (first sub-graph). Therefore, the overall force intensity at this corner point is the highest, represented by the darkest color or the highest value on the diagram. The right rear corner, left front corner, etc., are also marked with their corresponding intensities according to their deviation and internal force conditions. This composite diagram is a snapshot of the mechanical state of container A at the current moment. As the ship continues to sail, the roll angle changes, and the position deviation and force coefficient also change. The system will continuously repeat the process of S141-S143, constantly updating the force distribution diagram of container A and reflecting its stability state in real time.

[0114] refer to Figure 6 In step S15, the specific steps are as follows:

[0115] S151: Match each cargo feature with the corresponding force distribution map and mark the current position of the cargo feature. Determine multiple active position nodes of the cargo feature based on the current position of the cargo feature and the corresponding force distribution map. Determine the positional activity range of the cargo feature based on the multiple active position nodes of the cargo feature and the cargo load distribution map of the ship, so as to output the positional activity range of each cargo feature.

[0116] S152: Mark the relative position of each cargo feature, determine the first sub-risk coefficient based on the relative position and the range of movement of each cargo feature, and determine the second sub-risk coefficient based on the range of movement of each cargo feature and the ship's sailing speed.

[0117] S153: Based on the first sub-risk coefficient, the second sub-risk coefficient, and the risk coefficient mapping relationship, predict the risk coefficient of cargo features and mark the risk coefficient of each cargo feature. Determine the corresponding risk control area based on the risk coefficient of each cargo feature and the location of each cargo feature. Based on the risk control area, the cargo model of the cargo feature, and the ship's sailing speed, infer the corresponding risk control measures to manage the risk of cargo features.

[0118] In embodiments of this application, each cargo feature (such as container A, corn pile B) is associated with the latest force distribution map generated in step S143. This map shows the current hot spots and intensities of the forces acting on the cargo. The system records the precise location of the cargo feature on the ship (e.g., coordinates (X, Y, Z) or a location description based on the ship's structure). Combining the force distribution map and the current location, the system infers the key points or areas where the cargo is most likely to displace when subjected to current and future forces (such as ship swaying, acceleration, or deceleration). These points / areas are called "location activity nodes." For example, the corners or edges with the greatest pressure in the force map, or the points with the least internal friction.

[0119] Based on these active nodes, and combined with the overall cargo load distribution map of the ship (this map shows the overall distribution of cargo on the ship and the interactions that affect cargo movement, such as whether the movement of one cargo will squeeze the cargo next to it), the system simulates or calculates the maximum range of cargo movement under the forces at these nodes. This takes into account the physical characteristics of the cargo itself (such as shape and material), the lashing situation, and the constraints of surrounding cargo. This range is a predictive area, representing the area that the cargo will move to under normal navigation conditions, taking into account the influence of various forces. The calculated positional activity range of each cargo feature is output in the form of visualization (such as a semi-transparent area box) or data.

[0120] Specifically, for container A: the force distribution diagram shows that its right corner experiences the greatest force (dark red area); its current position is (X=10m, Y=5m, Z=15m); the system determines its active node to be the lower right corner; combined with the load distribution diagram (there are no particularly crowded goods nearby), the maximum range of its movement to the lower right is calculated, and a predicted activity range is output, centered on the current position and extending to the lower right by approximately 0.3 meters (X+0.2m, Y-0.1m, Z unchanged).

[0121] Corn pile B: The force distribution diagram shows that the contact area with the right side of the ship wall is under great force, and there is also pressure distribution inside; its current position is (X=20m, Y=3m, Z=8m); the system determines the active nodes as the contact point of the right side wall and the center of the pile; combined with the load distribution diagram (the ship wall is next to it, and the gap is on the other side), the system calculates the range of its movement to the right and slight sinking, and outputs a predicted activity range centered on the current position, extending to the right by about 0.5 meters and sinking slightly downward (Z axis) by 0.2 meters.

[0122] Corn pile C: The force distribution diagram shows that the force is concentrated in the contact area of ​​the right side of the ship's wall; its current position is (X=30m, Y=4m, Z=7m); the system determines the active node as the right side wall contact point; combined with the load distribution diagram (there are other goods on the left), the system calculates the range of its main rightward movement and outputs a predicted range of movement centered on the current position and extending to the right by about 0.4 meters.

[0123] Therefore, by marking the relative positions of each cargo feature, determining the first sub-risk coefficient based on the relative positions and the range of movement of each cargo feature, and determining the second sub-risk coefficient based on the range of movement of each cargo feature and the ship's speed, the overall consideration of the range of movement of each cargo feature and the ship's speed is taken into account, thus ensuring the accuracy of the second sub-risk coefficient.

[0124] At this point, the system calculates and records the relative positional relationships between each cargo feature; for example, the position of cargo A relative to cargo B, or the position of cargo C relative to the ship's wall; the first sub-risk coefficient (interaction risk) assesses the risk arising between cargoes due to their relative positions and ranges of movement; if the positions and ranges of movement of two cargo features overlap or are very close, when one cargo moves, it may collide with, squeeze, or interfere with the other cargo, resulting in risk; the magnitude of the coefficient depends on the degree of overlap / closeness, the size / weight of the cargo, and the direction and distance of their movement; the more overlap, the heavier the cargo, and the greater the distance of movement, the higher the coefficient.

[0125] The second sub-risk coefficient (dynamic environmental risk) assesses the impact of the ship's navigation state (speed, acceleration, rolling) on ​​the range of cargo movement, and thus the resulting risk. The faster the ship's speed, the greater the ship's acceleration or rolling amplitude, the larger the range of cargo movement, or the more sudden the movement, leading to an increase in risk. The coefficient depends on the ship's speed, the expected ship dynamics (such as maximum roll angle, pitch angle, acceleration), and the size of the cargo's own range of movement. The faster the speed, the more intense the dynamics, and the larger the range of movement, the higher the coefficient.

[0126] Specifically, the system records: Container A is approximately 5 meters to the left and forward of corn pile C; corn pile B is on the starboard side of the ship, and corn pile C is also on the starboard side, but about 10 meters further to the stern than B; Determine the first sub-risk factor: Check container A and corn pile C: their predicted activity ranges are far apart (A moves 0.3 meters to the right and down, C moves 0.4 meters to the right, initial distance 5 meters), with no overlap, and the first sub-risk factor is very low, for example, 0.1; Check corn pile B and corn pile C: they are both on the starboard side of the ship, B moves 0.5 meters to the right, and C moves 0.4 meters to the right; if their initial distance is close (for example, only 1 meter), their predicted activity ranges will partially overlap; assuming the overlapping area is small, but considering that they are both bulk cargo and there is mutual interference, the first sub-risk factor is set to medium, for example, 0.5.

[0127] Determine the second sub-risk coefficient: Assuming the current ship speed is 15 knots, a significant roll (maximum 15 degrees) is expected when passing through the lock; For container A: range of motion 0.3 meters, speed 15 knots, expected roll 15 degrees; Considering these factors, the second sub-risk coefficient is 0.4; For corn pile B: range of motion 0.5 meters (significant), speed 15 knots, expected roll 15 degrees; the second sub-risk coefficient is higher, for example, 0.7; For corn pile C: range of motion 0.4 meters, speed 15 knots, expected roll 15 degrees; the second sub-risk coefficient is 0.6.

[0128] Furthermore, based on the first sub-risk coefficient, the second sub-risk coefficient, and the risk coefficient mapping relationship, the risk coefficient of cargo features is predicted, and the risk coefficient of each cargo feature is marked. The corresponding risk control area is determined based on the risk coefficient and location of each cargo feature. Based on the risk control area, the cargo model of the cargo feature, and the ship's sailing speed, corresponding risk management measures are inferred to manage the risk of cargo features. This approach incorporates a holistic consideration of the risk coefficients and locations of each cargo feature, ensuring the accuracy of the corresponding risk control area. Simultaneously, a force distribution map of the cargo features is introduced to comprehensively consider the location and activity range of each cargo feature and the ship's cargo load distribution map, improving the prediction accuracy of the risk coefficients of each cargo feature for effective risk management.

[0129] At this point, the system uses a preset "risk coefficient mapping relationship" to combine the first sub-risk coefficient and the second sub-risk coefficient to calculate the comprehensive risk coefficient of each cargo feature. This mapping relationship defines how the two risks work together to generate the total risk. For example, the total risk coefficient = a × first sub-coefficient + b × second sub-coefficient, where a and b are weights. The calculated comprehensive risk coefficient is assigned to the corresponding cargo feature and marked (for example, low risk <0.3, medium risk 0.3-0.7, high risk >0.7).

[0130] Based on the risk coefficient and cargo characteristics, and the current location, the system delineates a "risk control zone" in the ship's virtual model or actual space. This zone is typically larger than the predicted activity range, or its extension range is set according to the risk level (high, medium, low). High risk corresponds to a larger control zone, indicating that special attention needs to be paid to the dynamics of this area and its vicinity. This zone is used for visual prompts or as a basis for triggering further measures. The system automatically recommends or triggers corresponding risk management measures based on the size of the risk control zone, the level of the risk coefficient, the model of cargo characteristics (e.g., containers need to be secured, bulk cargo needs anti-slip plates or ballast, liquids need ventilation / breathing control), and the current ship speed. For example, it may prompt the crew to check the securing status; suggest reducing the sailing speed; remind the crew to pay attention to the area and avoid personnel approaching; automatically adjust the ship's ballast water distribution to reduce rolling; if the risk is extremely high, it may even suggest suspending navigation or taking emergency reinforcement measures. The system communicates the recommended measures to the operators, or, if the system has the authority, automatically executes some measures (such as adjusting ballast water).

[0131] Specifically, assuming the mapping relationship is a simple weighted average with weights of 0.4 (first sub-item) and 0.6 (second sub-item); Container A: 0.4×0.1+0.6×0.4=0.04+0.24=0.28 (low risk); Corn pile B: 0.4×0.5+0.6×0.7=0.2+0.42=0.62 (medium risk); Corn pile C: 0.4×0.5+0.6×0.6=0.2+0.36=0.56 (medium risk).

[0132] Container A is marked as low risk (0.28), while corn piles B and C are marked as medium risk (0.62 and 0.56, respectively). In the virtual model, the system draws a small yellow warning box around container A (low risk) and a larger orange warning box around corn piles B and C (medium risk). These boxes are slightly larger than the predicted activity range. The system's risk management measures are as follows: For container A (low risk): the system only records the status or indicates "The lashing is good, continue monitoring"; For corn piles B and C (medium risk): the system will indicate "Pay attention to the bulk cargo area on the starboard side, there is a medium risk. It is recommended that the crew strengthen their observation and check for any obvious signs of movement; consider appropriately reducing the roll amplitude when passing through the lock (such as adjusting speed or ballast)." After seeing the medium risk warning, the operator will closely monitor these two corn pile areas and choose to pass through the lock at a slightly reduced speed to reduce the impact of the roll. If the risk coefficient further increases (e.g., exceeding 0.7), the system will issue a more urgent alarm and may even recommend temporary reinforcement.

[0133] Please see Figure 7 , Figure 7This is a schematic diagram of the structural composition of a ship cargo identification system based on lock video monitoring according to an embodiment of the present invention; the ship cargo identification system based on lock video monitoring is applied to the above-mentioned cargo identification method, and the system includes:

[0134] Cargo feature module 21 is used to collect monitoring video of the ship from the lock, and to determine the current form of the ship and multiple cargo features based on the recognition of the monitoring video.

[0135] Cargo load distribution map module 22 is used to determine the corresponding cargo model based on the identification of multiple cargo features, and to determine the cargo load distribution map of the ship based on the storage format of multiple cargo models and the current navigation status of the ship.

[0136] The compensation height module 23 is used to determine the corresponding sea level height based on the detection of the ship's cargo load distribution map, and to determine the compensation height of the lock according to the sea level height, the current opening height of the lock and the ship's sailing speed, so as to allow the ship to pass.

[0137] The force distribution diagram module 24 is used to determine the position deviation of each cargo feature based on multiple position data of the cargo feature during navigation, and to determine the force distribution diagram of the cargo feature based on the position deviation, force coefficient and corresponding cargo model.

[0138] The cargo risk management module 25 is used to determine the location range of each cargo feature based on the force distribution map and current location of multiple cargo features, and to predict the risk coefficient of each cargo feature based on the location range of each cargo feature and the ship's sailing speed, so as to manage the risk of cargo features.

[0139] The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity, not all combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

Claims

1. A method for identifying cargo on ships based on lock video surveillance, characterized in that, include: The system collects surveillance video of ships from the locks and identifies the current state of the ships and multiple cargo characteristics based on the recognition of this surveillance video. Based on the identification of multiple cargo features, the corresponding cargo model is determined, and the cargo load distribution map of the ship is determined based on the storage form of multiple cargo models and the current navigation form of the ship. The corresponding sea level is determined based on the detection of the ship's cargo load distribution map, and the compensation height of the lock is determined according to the sea level, the current opening height of the lock, and the ship's sailing speed to allow the ship to pass. In each cargo feature, the positional deviation of the cargo feature is determined based on multiple positional data of the cargo feature during the navigation process, and the force distribution map of the cargo feature is determined based on the positional deviation of the cargo feature, the force coefficient and the corresponding cargo model; Based on the force distribution diagram of multiple cargo features and their current location, the location and activity range of each cargo feature are determined. Based on the location and activity range of each cargo feature and the ship's sailing speed, the risk coefficient of each cargo feature is predicted in order to manage the risks of the cargo features.

2. The method for cargo identification of ships based on lock video monitoring according to claim 1, characterized in that, The system collects monitoring video of the ship from the lock, and determines the ship's current form and multiple cargo characteristics based on the identification of this monitoring video, including: When a ship is sailing toward the lock, the video monitoring area is determined based on the ship's position and the lock's position. Multiple cameras located in the lock shoot toward the video monitoring area to determine the lock's monitoring video of the ship. Based on this monitoring video, time, and the ship's sailing distance, multiple sub-sailing videos are determined. Based on the recognition of each sub-navigation video, multiple navigation images of the ship in different directions are determined. The three-dimensional model of the ship is determined by synthesizing the multiple navigation images. The current form of the ship is determined based on the three-dimensional model of the ship and multiple attitude parameters of the ship at different times. Multiple cargo locations are marked in each sub-navigation video. Based on the multiple cargo locations and the ship's 3D model, the corresponding cargo storage area is determined. Based on the identification of the cargo storage area, multiple cargo features are determined, and each cargo feature corresponds to a type of cargo.

3. The method for cargo identification of ships based on lock video monitoring according to claim 1, characterized in that, The process of determining the corresponding cargo model based on the identification of multiple cargo features, and determining the cargo load distribution map of the ship based on the storage format of the multiple cargo models and the current navigation status of the ship, includes: Based on the identification of multiple cargo features, the corresponding cargo model information is determined. At the same time, the shipping database and ship model information are collected. Based on the shipping database, the cargo model information and the ship model information, the corresponding cargo model is determined. Real-time images of multiple cargo features are acquired. Based on the real-time images of multiple cargo features, the corresponding cargo models, and the cargo storage area of ​​the ship, the storage form of each cargo model is determined. Based on the storage form of each cargo model and the position of the cargo model relative to the ship, the first load distribution map is determined. The ship's current navigation status is collected, and multiple sub-navigation statuses of the ship are determined based on the decomposition of the ship's current navigation status. Based on the multiple sub-navigation statuses of the ship, the storage status of each cargo model, and the first load distribution map, the cargo load distribution map of the ship is determined.

4. The method for cargo identification of ships based on lock video surveillance according to claim 1, characterized in that, The process of determining the corresponding sea level height based on the detection of the ship's cargo load distribution map, and determining the lock's compensation height based on this sea level height, the lock's current opening height, and the ship's speed to allow ship passage, includes: Multiple load zones are determined based on the detection of the ship's cargo load distribution map. The height of the cargo relative to the sea surface is determined according to the spatial distribution of the multiple load zones, and the limit height of the ship relative to the sea surface is output. The current image of the lock is acquired, and the current height of the lock is determined based on the current image of the lock, the current image of the ship, and the cargo load distribution map. At this time, the maximum height of the ship relative to the sea surface and the current height of the lock refer to the same sea surface.

5. The method for cargo identification of ships based on lock video monitoring according to claim 4, characterized in that, The method of determining the corresponding sea level height based on the detection of the ship's cargo load distribution map, and determining the compensation height of the lock based on the sea level height, the current opening height of the lock, and the ship's speed to allow ship passage, further includes: The safe height of the ship is determined based on the ship's maximum height relative to the sea surface and the ship's speed. The compensation height of the lock is determined based on the ship's safe height and the current height of the lock. Multiple sub-compensation height zones are determined based on the compensation height of the lock and the distance of the ship relative to the lock. Lock height control measures for each sub-compensation height zone are marked to allow for step-by-step changes in lifting and lowering speeds and to allow ships to pass.

6. The method for cargo identification of ships based on lock video surveillance according to claim 1, characterized in that, Among the various cargo features, the positional deviation of the cargo feature is determined based on multiple positional data during navigation. The force distribution diagram of the cargo feature is then determined based on the positional deviation, force coefficient, and corresponding cargo model, including: Real-time monitoring of each cargo feature. Each cargo feature has a position change signal as the ship sails. The position change signal triggers the position detection of each cargo feature and collects multiple position data of the cargo feature during the sailing process. Based on the multiple position data and the corresponding cargo feature image, the position offset map of the cargo feature is determined and the position deviation of the cargo feature is marked. The stress analysis method for the cargo feature is determined based on the location of the cargo feature, the ship's speed, and the ship's current navigation mode. The stress analysis of the cargo feature is then triggered along this stress analysis method to output the stress coefficient of the cargo feature.

7. The method for cargo identification of ships based on lock video surveillance according to claim 6, characterized in that, The method of determining the positional deviation of each cargo feature based on multiple positional data during navigation, and determining the force distribution diagram of the cargo feature based on the positional deviation, force coefficient, and corresponding cargo model, further includes: The first sub-force distribution map is determined based on the positional deviation of the cargo feature and the force coefficient. The second sub-force distribution map is determined based on the positional deviation of the cargo feature and the corresponding cargo model. The force distribution map of the cargo feature is determined by combining the first and second sub-force distribution maps. The force distribution map of the cargo feature is dynamically updated as the cargo feature sails.

8. The method for cargo identification of ships based on lock video surveillance according to claim 1, characterized in that, The method involves determining the positional range of each cargo feature based on the force distribution map and current position, and predicting the risk coefficient of each cargo feature based on its positional range and the ship's speed, in order to manage the risks of the cargo features. This includes: Each cargo feature is matched with a corresponding force distribution map, and the current position of the cargo feature is marked. Based on the current position of the cargo feature and the corresponding force distribution map, multiple positional activity nodes of the cargo feature are determined. Based on the multiple positional activity nodes of the cargo feature and the cargo load distribution map of the ship, the positional activity range of the cargo feature is determined, so as to output the positional activity range of each cargo feature.

9. The method for cargo identification of ships based on lock video monitoring according to claim 8, characterized in that, The method of determining the positional activity range of each cargo feature based on the force distribution map and current position of multiple cargo features, and predicting the risk coefficient of each cargo feature based on the positional activity range of each cargo feature and the ship's sailing speed, in order to manage the risk of cargo features, also includes: Mark the relative position of each cargo feature, determine the first sub-risk coefficient based on the relative position and the range of movement of each cargo feature, and determine the second sub-risk coefficient based on the range of movement of each cargo feature and the ship's sailing speed. The risk coefficients of cargo features are predicted based on the first sub-risk coefficient, the second sub-risk coefficient, and the risk coefficient mapping relationship. The risk coefficients of each cargo feature are then marked. The corresponding risk control areas are determined based on the risk coefficients of each cargo feature and the location of each cargo feature. Based on the risk control areas, the cargo model of the cargo features, and the ship's sailing speed, the corresponding risk control measures are inferred to manage the risks of the cargo features.

10. A cargo identification system for ships based on lock video surveillance, employing the cargo identification method as described in any one of claims 1-9, characterized in that, The system includes: The cargo feature module is used to collect monitoring video of the ship from the lock, and to determine the current shape of the ship and multiple cargo features based on the identification of the monitoring video. The cargo load distribution map module is used to determine the corresponding cargo model based on the identification of multiple cargo features, and to determine the cargo load distribution map of the ship based on the storage format of multiple cargo models and the current navigation status of the ship. The compensation height module is used to determine the corresponding sea level height based on the detection of the ship's cargo load distribution map, and to determine the compensation height of the lock according to the sea level height, the current opening height of the lock and the ship's sailing speed, so as to allow the ship to pass. The force distribution map module is used to determine the positional deviation of each cargo feature based on multiple positional data of the cargo feature during navigation, and to determine the force distribution map of the cargo feature based on the positional deviation, force coefficient and corresponding cargo model. The cargo risk management module is used to determine the location and activity range of each cargo feature based on the force distribution map and current location of multiple cargo features, and to predict the risk coefficient of each cargo feature based on the location and activity range of each cargo feature and the ship's sailing speed, so as to manage the risk of cargo features.

Images