Method for image recognition of ship behavior based on waterborne electronic socket

Abstract

The application relates to a method for identifying ship behaviors based on water electronic throat image recognition, and relates to the technical field of image recognition, and comprises the following steps: collecting multi-angle images of a target ship through a water electronic throat snapshot system, and constructing a three-dimensional model; matching a fixed early warning water level line by using a ship type database, combining real-time sailing interference information to correct and output an adaptive early warning water level line; analyzing ship three-dimensional model sequences in a preset time zone to obtain a real-time water level line and a driving posture; fusing ship sailing data and the ship driving posture to optimize the real-time water level line, and obtaining a corrected water level line; and if the corrected water level line exceeds the adaptive early warning water level line, overload early warning is performed. The application solves the problems that in a traditional system, a fixed water line is used, environmental factor changes are not considered, and the water line recognition is unstable and prone to false detection and missed detection due to influences such as light and shielding in a traditional image method, and the accuracy and reliability of cargo ship overload identification are improved.

Description

Technical Field

[0001] The present application relates to the field of image recognition, and in particular to a method for image recognition of ship behavior based on an on-water electronic bayonet. Background Art

[0002] With the development of intelligent water traffic monitoring technology, accurate identification of overloaded cargo ships has become a crucial foundation for improving water traffic safety and efficiency. Currently, traditional cargo ship overload detection relies primarily on fixed thresholds and single image recognition, requiring manual parameter calibration. This technology suffers from poor environmental adaptability, low recognition accuracy, and high false detection and missed detection rates. Furthermore, it struggles to meet the real-time monitoring needs of overloaded ships in complex environments.

[0003] The existing overload identification method relies solely on a fixed waterline and single image analysis, resulting in large deviations between the identification results and the actual loading status. This not only increases the risk of regulatory misjudgment and the cost of human calibration, but also makes it difficult to meet the requirements for accurate and reliable overload identification in the intelligent transformation of water transportation. Summary of the Invention

[0004] In order to solve the above technical problems, the present application provides a method for image recognition of ship behavior based on water electronic card slots, which improves the accuracy and reliability of cargo ship overload identification, solves the problem that traditional systems use fixed waterlines without considering environmental factors and are prone to false alarms, and traditional image methods are unstable in waterline recognition and prone to false detections and missed detections due to the influence of lighting, occlusion, etc.

[0005] The embodiments of this application disclose the following technical solutions:

[0006] The present application provides a method for image recognition of ship behavior based on an on-water electronic bayonet, the method comprising:

[0007] The multi-angle images of the target ship are collected through the water electronic bayonet capture system to perform 3D reconstruction of the hull and build a 3D model of the ship;

[0008] Using a ship type database, a fixed warning water level is obtained based on the three-dimensional model of the ship, and the fixed warning water level is corrected according to real-time navigation interference information to output an adapted warning water level;

[0009] According to the ship's 3D model sequence in the preset time zone, the real-time water level and ship's driving posture are analyzed;

[0010] Optimizing the real-time water level using the ship navigation data acquired by the ship wireless communication system in combination with the ship's driving posture to obtain a corrected water level;

[0011] If the correction water level exceeds the adaptation warning water level, an overload warning is issued to the target ship.

[0012] One or more technical solutions provided in this application have at least the following technical effects or advantages:

[0013] This application proposes a method for image recognition of ship behavior based on an on-water electronic card slot. Through multi-source data fusion and dynamic parameter optimization, it can achieve accurate identification of overloaded cargo ships. First, the on-water electronic card slot capture system is used to collect multi-angle images of the target ship, construct a three-dimensional model of the hull, and combine the ship type database with real-time navigation interference information to generate an adaptive warning water level. At the same time, based on the sequence of three-dimensional ship models in a preset time zone, the real-time water level and driving posture are analyzed, and the water level detection is optimized by integrating the ship's navigation data to form a corrected water level. If the corrected water level exceeds the adaptive warning water level, an overload warning will be automatically triggered and pushed to the supervision platform through the on-water electronic card slot system to ensure timely and reliable warnings. At the same time, if the abnormal behavior recognition model constructed based on multi-angle images detects abnormal behavior of the ship, it will also automatically trigger a ship behavior abnormality warning and push it to the supervision platform. This effectively solves the problems of low efficiency of manual interpretation and large impact of environmental interference, and provides an innovative solution for intelligent and precise water traffic supervision.

[0014] The technical solution of this application realizes accurate judgment of cargo ship overload based on water electronic card port image recognition by integrating multi-source data such as ship three-dimensional model parameters, navigation attitude parameters and environmental interference parameters, solving the problems of false detection and missed detection caused by traditional overload identification relying on fixed thresholds and single data, and effectively improving the accuracy and reliability of overload identification in complex water environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0016] Figure 1 A flowchart of a method for image recognition of ship behavior based on an on-water electronic card slot provided in an embodiment of the present application. DETAILED DESCRIPTION

[0017] The present application provides a method for image recognition of ship behavior based on an on-water electronic card slot, which is used to solve the technical problems in the prior art that a fixed waterline does not take into account changes in environmental factors and is prone to false alarms, and that image methods are affected by lighting, occlusion, etc., resulting in unstable waterline recognition and prone to false detections and missed detections.

[0018] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without making creative efforts are within the scope of protection of this application.

[0019] In the description of this application, the terms "first" and "second" are used for descriptive purposes only and should not be understood to indicate or imply relative importance or implicitly specify the number of the technical features indicated. Therefore, a feature specified as "first" or "second" may explicitly or implicitly include one or more of the described features. In the description of this application, "plurality" means two or more, unless otherwise specifically specified.

[0020] In the description of this application, the term "for example" is used to mean "used as an example, illustration or explanation". Any embodiment described as "for example" in this application is not necessarily to be construed as being more preferred or advantageous than other embodiments. The following description is given to enable any person skilled in the art to implement and use the present invention. In the following description, details are listed for the purpose of explanation. It should be understood that a person of ordinary skill in the art will recognize that the present invention can be implemented without using these specific details. In other examples, well-known structures and processes will not be elaborated in detail to avoid obscuring the description of the present invention with unnecessary details. Therefore, the present invention is not intended to be limited to the embodiments shown, but is consistent with the widest scope consistent with the principles and features disclosed in this application.

[0021] Example 1, as shown in the attached Figure 1 As shown, the present application provides a method for image recognition of ship behavior based on an on-water electronic bayonet, the method comprising the following steps:

[0022] S100: Uses an on-water electronic camera capture system to capture multi-angle images of the target ship, reconstructs the hull in 3D, and constructs a 3D model of the ship.

[0023] In the embodiment of the present application, in the scenario of identifying overloaded ships on the water, in order to accurately obtain the ship shape for water level analysis, it is necessary to use the water electronic card capture system to collect images from multiple perspectives and construct a three-dimensional model of the ship through three-dimensional reconstruction.

[0024] Specifically, when the target ship enters the coverage area of ​​the system, the first, second and third cameras collect multi-angle ship images of the side, front and bottom directions according to their layout angles and positions.

[0025] Further, the collected multi-angle ship images are processed and transmitted to an edge processing unit, a ship body three-dimensional reconstruction model based on a multi-view convolutional neural network is used to evaluate real-time image influencing factors such as light intensity and shielding ratio, determine an image interference coefficient, select a ship body three-dimensional reconstruction unit according to the coefficient, and perform three-dimensional reconstruction on the multi-angle images to obtain an initial ship three-dimensional model.

[0026] Further, a model fitting operation is performed on the initial ship three-dimensional model, model characteristics are fused, and an accurate ship three-dimensional model is output to completely present the ship shape and structure.

[0027] Finally, the ship three-dimensional model serves as basic data for subsequent analysis of real-time water level lines and ship driving postures, supports an overload identification process, and provides an accurate three-dimensional shape basis for ship behavior monitoring by using multi-view image acquisition and intelligent reconstruction logic.

[0028] The step S100 in the method provided in the embodiments of the application includes:

[0029] An overwater electronic lens snapshot system is constructed, wherein the overwater electronic lens snapshot system includes a first camera, a second camera and a third camera, the first camera is arranged on a fixed support on the left bank of the channel, is 3 meters away from the water surface, has a shooting direction that forms a 90° angle with the center line of the channel, and has a shooting angle that is a lateral horizontal view angle;

[0030] The second camera is arranged on a high-pole support that is 40 meters away from the first camera in the upstream direction of the channel, is 5 meters away from the water surface, has a shooting direction that is parallel to the center line of the channel, and has a shooting angle that is a forward horizontal view angle;

[0031] The third camera is arranged on a high-pole support above the midpoint of the line connecting the first camera and the second camera, is 8 meters away from the water surface, has a shooting direction that is downward looking, and has a shooting angle that is a 45° pitch angle.

[0032] When a target ship travels into the coverage range of the overwater electronic lens snapshot system, multi-angle images of the target ship are collected to obtain multi-angle ship images;

[0033] In the cloud server, a ship body three-dimensional reconstruction model is constructed based on a multi-view convolutional neural network and is downloaded to an edge processing unit of the overwater electronic lens snapshot system, wherein the ship body three-dimensional reconstruction model includes K ship body three-dimensional reconstruction units;

[0034] Real-time image influencing factors are obtained to perform image quality evaluation and determine an image interference coefficient, wherein the influencing factors at least include light intensity, shielding ratio and image blurriness;

[0035] A quantity P is selected according to the image interference coefficient setting unit, P hull three-dimensional reconstruction units are randomly selected from the K hull three-dimensional reconstruction units, three-dimensional reconstruction is performed on the multi-angle ship image to obtain P initial ship three-dimensional models, and model fitting is performed on the P initial ship three-dimensional models to obtain a ship three-dimensional model.

[0036] In the embodiment of the present application, in the scenario of intelligent monitoring of ship behavior, in order to accurately capture multi-angle images of the ship to support three-dimensional reconstruction, it is necessary to build a multi-perspective collaborative water electronic card capture system.

[0037] Specifically, first plan the camera deployment based on the waterway layout and monitoring needs.

[0038] Among them, the first camera is installed on a fixed bracket on the left bank of the channel, 3 meters from the water surface, with a lateral horizontal perspective (at a 90° angle to the center line of the channel), covering a longitudinal length of about 30 meters on the shore side of the hull, and collecting information such as the side waterline.

[0039] The second camera is placed on a high pole 40 meters upstream of the first camera, 5 meters from the water surface, with a forward horizontal viewing angle (parallel to the centerline of the channel), covering a 15-meter width of the hull, to assist in analyzing the symmetry characteristics of the hull structure.

[0040] The third camera is placed on a high pole above the midpoint of the line connecting the two cameras, 8 meters from the water surface, with a 45° pitch and downward viewing angle, covering a 40m×20m intersection area, supplementing the overall outline of the ship, deck cargo and navigation attitude information.

[0041] When a ship enters the coverage area of ​​the system, the three cameras are triggered synchronously to collect multi-angle ship images from the side, front and top, and summarize them in a unified format to form a ship image set.

[0042] Furthermore, a 3D reconstruction model of the hull is constructed on the cloud server based on a multi-view convolutional neural network. This model consists of K 3D reconstruction units. Once constructed, the model is transferred to the edge processing unit of the waterborne electronic bayonet capture system.

[0043] The step of “constructing a 3D reconstruction model of a hull based on a multi-view convolutional neural network” in the method provided in the embodiment of the present application includes:

[0044] According to the ship monitoring log of the water electronic bayonet capture system, a sample multi-angle ship image set is collected, and the sample multi-angle ship images are reconstructed using a three-dimensional reconstruction algorithm to obtain a sample ship three-dimensional model set;

[0045] The sample multi-angle ship image set and the sample three-dimensional ship model set are used as training data, divided into K equal parts, and selected with replacement K times to obtain a first training set, and the selection is iterated K times to obtain K training sets, where K is an integer greater than 5;

[0046] The K training sets are used to train the multi-view convolutional neural network until convergence, to obtain K hull three-dimensional reconstruction units, and to construct a hull three-dimensional reconstruction model.

[0047] In an embodiment of the present application, in order to improve the efficiency of ship behavior analysis, a three-dimensional reconstruction model of the hull needs to be constructed in the cloud server to achieve efficient three-dimensional ship modeling through the lightweight computing logic of the multi-view convolutional neural network.

[0048] Specifically, we collected a set of sample multi-angle ship images covering different ship types (e.g., cargo and passenger ships), navigation states (empty / loaded), and environmental conditions (sunny / cloudy / different lighting conditions). These images were processed using a 3D reconstruction algorithm to generate a set of sample ship 3D models containing the ship's 3D geometry and dimensional parameters, ensuring both diversity and representativeness of the sample data.

[0049] Among them, unlike directly using the three-dimensional reconstruction algorithm, the amount of redundant data can be reduced by pre-screening typical samples (such as hull feature images under different load conditions), and the sample images can be preliminarily reconstructed using a lightweight three-dimensional reconstruction algorithm to generate a sample ship three-dimensional model set containing key three-dimensional structures (such as hull contours and waterline datums), thereby reducing the model complexity while retaining the core features.

[0050] Furthermore, the sample multi-angle ship image set and the sample 3D model set are used as training data and divided into K equal parts (K is an integer greater than 5, such as K=10). Sampling with replacement is performed iteratively K times, generating a different training set combination each time, thereby enhancing the model's adaptability to different data distributions.

[0051] For example, during the first selection, some samples are randomly selected to form the first training set, and during the second sampling, repeated selection of samples is allowed to construct a new training set, ensuring that the K training sets all contain differentiated sample features.

[0052] Furthermore, the multi-view convolutional neural network is trained independently using K training sets. That is, each training set is input into the corresponding multi-view convolutional neural network, and the difference between the 3D ship model output by the network and the sample 3D ship model set is calculated through forward propagation to obtain the loss function value.

[0053] Further, the network weights and bias parameters are optimized by a backpropagation algorithm to minimize the loss function, and the network parameters are iteratively optimized to enable the network to extract key features such as ship hull contours and waterlines from multi-angle images and map them to three-dimensional space.

[0054] During the training process, the convergence of the loss function is monitored. When the loss function of each network no longer significantly decreases or reaches a preset convergence threshold (e.g., mean square error less than 0.01), the network training is considered complete, and a ship three-dimensional reconstruction unit is obtained.

[0055] Further, the above process is repeated to train K training sets, and finally K ship three-dimensional reconstruction units with independent reconstruction capabilities are obtained, which are finally combined to form a complete ship three-dimensional reconstruction model.

[0056] Through this independent training method, each reconstruction unit can be optimized for different sample features, thereby improving the overall reconstruction capability and generalization of the model, and laying a foundation for subsequent accurate construction of the ship three-dimensional model.

[0057] After the ship three-dimensional reconstruction model is constructed, the light intensity data is further collected in real time by the image sensor to evaluate the influence of lighting conditions on image quality.

[0058] Specifically, a histogram analysis and threshold comparison method is used to determine whether the current lighting is in the overexposure, underexposure, or normal range.

[0059] For example, when the proportion of pixel values exceeding 200 is greater than 30%, it is determined that there is strong light interference, and the lighting intensity influence coefficient is set to 0.8; when the proportion of pixel values less than 50 exceeds 40%, it is determined that there is weak light interference, and the influence coefficient is set to 0.7.

[0060] At the same time, a target detection algorithm is used to identify the occlusions (such as other ships, floating objects, bridge piers, and shore trees) in the ship body area, calculate the proportion of the occluded area to the total ship body area, and if the occlusion proportion exceeds 20%, the occlusion proportion coefficient is set to 0.6, otherwise it is set to 0.3.

[0061] For image blurriness, the mean and variance of the image gradient amplitude are calculated. When the gradient mean is less than 15 and the variance is less than 50, it is determined that the image is severely blurred, and the blurriness coefficient is set to 0.9, otherwise it is set to 0.2.

[0062] The specific values of the influence coefficient (e.g., 0.8), the occlusion proportion coefficient (e.g., 0.6), and the blurriness coefficient (e.g., 0.9) are determined based on a large amount of historical sample data and experimental verification. These values are continuously optimized through analysis of historical data and actual scene testing to ensure that the coefficient settings accurately reflect the degree of image interference.

[0063] Furthermore, the coefficients corresponding to the illumination intensity, occlusion ratio, and image blur are added and divided by 3 to obtain the image interference coefficient in the range of 0-1.

[0064] For example, if the illumination intensity coefficient is 0.8, the occlusion ratio coefficient is 0.6, and the blur coefficient is 0.9, the image interference coefficient is (0.8+0.6+0.9) / 3=0.77.

[0065] This coefficient is positively correlated with the complexity of hull reconstruction. The larger the coefficient, the more serious the interference of factors such as lighting, occlusion, and blur on the image, the more difficult the hull reconstruction is, and more hull three-dimensional reconstruction units need to be called to ensure reconstruction accuracy; conversely, the smaller the coefficient, the better the image quality, the lower the complexity of hull reconstruction, and the fewer reconstruction units need to be called.

[0066] After calculating the image interference coefficient, the number of unit selections P needs to be set according to the coefficient size to ensure that the 3D reconstruction task can be completed efficiently and accurately under different image quality conditions, thereby performing high-quality 3D reconstruction processing on multi-angle ship images.

[0067] In the method provided in the embodiment of the present application, the step of “setting the number P of unit selections according to the image interference coefficient” includes:

[0068] Acquiring ship navigation data using a ship wireless communication system, wherein the ship navigation data includes a navigation route and a navigation speed;

[0069] The route deviation, the mean sailing speed and the sailing speed fluctuation coefficient of the preset historical time zone are calculated based on the sailing route and the sailing speed;

[0070] Determining the complexity of hull reconstruction based on the route deviation, the mean sailing speed, the sailing speed fluctuation coefficient, and the image interference coefficient, wherein the complexity of hull reconstruction is positively correlated with the route deviation, the mean sailing speed, the sailing speed fluctuation coefficient, and the image interference coefficient;

[0071] The ratio of the hull reconstruction complexity to the historical maximum hull reconstruction complexity is multiplied by K and rounded to the integer to obtain the number P of unit selections.

[0072] In the embodiment of the present application, when the number P of units is selected according to the image interference coefficient setting, it is necessary to first use the ship wireless communication system to obtain the ship navigation data (including the navigation route and navigation speed). That is, through the wireless communication system, the AIS (automatic identification system) signal sent by the ship can be received in real time, and the navigation trajectory and speed information can be analyzed to provide a data basis for subsequent analysis.

[0073] Specifically, based on the obtained navigation route and navigation speed, the route deviation, navigation speed mean and navigation speed fluctuation coefficient (the fluctuation coefficient is the ratio of the standard deviation to the mean) of the preset historical time zone (such as within the last 10 minutes) are calculated.

[0074] For example, the preset time zone is the last 10 minutes, the speed is monitored 30 times to obtain a speed sequence, the standard deviation and mean of the sequence are calculated, and the ratio of the standard deviation to the mean is used as the speed fluctuation coefficient to reflect the stability of the ship's sailing speed.

[0075] Furthermore, the actual navigation route of the ship during the period is compared with the centerline of the channel, and the ratio of the maximum lateral distance to the channel width is calculated as the route deviation.

[0076] For example, if the channel width is 100 meters and the maximum lateral distance is 5 meters, the course deviation is 5 / 100 = 0.05 (i.e., 5%). Simultaneously, the arithmetic mean of the speed data is taken to obtain the mean sailing speed. For example, if the mean of 30 speed data points within 10 minutes is 15 knots (1 knot = 1.852 km / h), and the calculated standard deviation is 1.2 knots, then the speed fluctuation coefficient is 1.2 / 15 = 0.08. A smaller value indicates a more stable speed.

[0077] Furthermore, the route deviation, mean sailing speed, sailing speed fluctuation coefficient and image interference coefficient are combined to determine the hull reconstruction complexity through comprehensive analysis, which is calculated by the formula "hull reconstruction complexity = combined route deviation + mean sailing speed + sailing speed fluctuation coefficient + image interference coefficient".

[0078] Among them, the complexity of hull reconstruction is positively correlated with four parameters: route deviation, mean sailing speed, sailing speed fluctuation coefficient and image interference coefficient. That is, an increase in any parameter value will lead to an increase in reconstruction complexity.

[0079] For example, within a preset historical time zone, a cargo ship's route deviation is calculated to be 0.05 (5%), its average speed is 15 knots, its speed fluctuation coefficient is 0.08, and the image interference coefficient, which is affected by lighting, occlusion, and blur, is 0.77. Therefore, the ship reconstruction complexity is calculated as 0.05 + 15 + 0.08 + 0.77 = 15.9.

[0080] If the ship's speed suddenly increases to 20 knots and other parameters remain unchanged, the complexity = 0.05 + 20 + 0.08 + 0.77 = 20.9, which is significantly higher than the previous one, reflecting the positive correlation between the increase in the mean speed and the increase in reconstruction complexity.

[0081] Finally, the calculated hull reconstruction complexity is ratioed to the historical maximum hull reconstruction complexity, and the ratio is multiplied by K and rounded to the integer to obtain the number of unit selection P. P units are randomly selected from the K hull 3D reconstruction units for 3D reconstruction.

[0082] For example, if K = 10, and the ratio of the current hull reconstruction complexity (such as 15.9) to the historical maximum hull reconstruction complexity value (such as 26.5) is 0.6, then P = 0.6 × 10 = 6, that is, 6 of the K hull 3D reconstruction units are selected for 3D reconstruction to ensure the reconstruction accuracy in complex scenes.

[0083] Among them, each reconstruction unit is an independent model formed by multi-view convolutional neural network training, with differentiated feature extraction capabilities. Random selection can utilize model diversity to reduce the error of a single model.

[0084] Furthermore, the multi-angle ship images are fed into the six selected reconstruction units, each of which independently processes the image data to generate a corresponding initial 3D model. For example, different units may have higher reconstruction accuracy for the bow, midship, and stern of the hull, respectively, resulting in six initial models with different details.

[0085] Furthermore, the six initial models are fitted. First, model registration is performed. This involves extracting key geometric features (such as hull contour points and deck edge lines) from each initial model using a feature point matching algorithm (such as the ICP algorithm). The six models are then unified into the same coordinate system to eliminate spatial position deviations.

[0086] Secondly, model fusion fitting is performed, that is, the statistical distribution of the feature overlapping area of ​​the aligned model is calculated and processed using smoothing, averaging and completion strategies.

[0087] Specifically, for repeated features (such as the mid-hull structure shared by multiple models), the average is taken to optimize the surface accuracy; for missing features (such as the stern details that are not fully reconstructed in some models), they are supplemented based on the ship's geometric priors (symmetry, ship type specifications), and finally a three-dimensional ship model with complete structure and consistent accuracy is formed.

[0088] The final fitted three-dimensional ship model integrates the advantages and features of multiple initial models, which can more accurately reflect the actual shape of the ship and provide reliable data support for subsequent water level analysis and overload warning. It can effectively improve the reconstruction accuracy, especially in complex scenarios (such as strong light and high-speed fluctuations).

[0089] S200: using a ship type database, obtaining a fixed warning water level based on the three-dimensional ship model, and correcting the fixed warning water level according to real-time navigation interference information to output an adapted warning water level;

[0090] In the embodiment of the present application, in the scenario of identifying overloaded ships on water, in order to accurately determine the overload status of the ship, it is necessary to correct the warning water level line in combination with the ship's own shape and the real-time navigation environment.

[0091] Specifically, relying on the ship type database, the constructed three-dimensional ship model is compared with the ship type features in the database to match the fixed warning water level line corresponding to the target ship type. This water level line is the benchmark for determining overload of the ship type under standard conditions.

[0092] Furthermore, real-time navigation interference information collection is carried out. Based on preset navigation interference indicators, including water temperature, water level, water flow velocity and wind speed, real-time data is obtained through sensors and other equipment. Among them, the water temperature factor needs to refer to the water density characteristics to interfere with the water level determination.

[0093] Furthermore, in order to quantify the impact of interference, it is necessary to use the ship type database, with the target ship type and the three-dimensional model of the ship as constraints, to collect a sample navigation interference information set (covering different water temperature, water level and other combination scenarios), and to count the historical water level deviation ratio under the corresponding scenario as the sample deviation ratio to form a sample deviation ratio set.

[0094] Furthermore, a sample navigation interference information set and a sample deviation ratio set were used as training data, and a water level deviation predictor was constructed based on machine learning. The navigation interference information collected in real time was input into the predictor, and the predicted water level deviation ratio was obtained through analysis.

[0095] Finally, the fixed warning water level is corrected according to the predicted water level deviation ratio, and the warning water level adapted to the current navigation environment is output to ensure that the water level for overload identification is more accurate under interferences such as different water temperatures and water flows, providing a reliable basis for ship overload identification. By using the water level correction logic adapted to the environment, the accuracy of overload identification in complex scenarios is improved.

[0096] Step S200 in the method provided in the embodiment of the present application includes:

[0097] Collecting interference information based on preset navigation interference indicators to obtain real-time navigation interference information, wherein the preset navigation interference indicators include water temperature, water level, water flow velocity, and wind speed;

[0098] Using a ship type database, the target ship type is obtained based on the three-dimensional ship model. With the target ship type and the three-dimensional ship model as constraints, a sample navigation interference information set is collected, and the historical water level deviation ratio under different sample navigation interference information is obtained, which is set as the sample deviation ratio to obtain a sample deviation ratio set;

[0099] Using the sample navigation interference information set and the sample deviation ratio set as training data, a water level deviation predictor is constructed based on machine learning, and a predicted water level deviation ratio is obtained according to the real-time navigation interference information analysis;

[0100] The fixed warning water level is corrected according to the predicted water level deviation ratio, and an adapted warning water level is output.

[0101] In the embodiment of the present application, during the ship overload identification process, in order to eliminate the interference of real-time environmental factors on the water level determination, it is necessary to dynamically correct the warning water level in combination with the navigation interference information to improve the accuracy of overload identification.

[0102] First, environmental data such as water temperature, water level, water flow velocity and wind speed are collected in real time through the sensor network.

[0103] Specifically, water temperature sensors are deployed at waterway monitoring points to transmit real-time water temperature data. For example, when the water temperature changes from 4°C (peak water density, approximately 1000.00 kg / m³) to 25°C, the sensors quickly transmit the temperature value to the system backend.

[0104] Water density varies nonlinearly with temperature. Below 4°C, water's density decreases due to its loose structure (for example, at 0°C, the density is 999.84 kg / m³). Above 4°C, the density decreases continuously with increasing temperature (for example, at 25°C, the density is approximately 997.0 kg / m³, a decrease of approximately 3.0 kg / m³ compared to 4°C). This temperature-induced density change directly affects a ship's buoyancy and draft, thereby interfering with waterline determination and requiring dynamic correction via a subsequent waterline identification error predictor.

[0105] The water level is measured using a radar gauge, which accurately records the current water level in centimeters. For example, if the water level rises from 1.5 meters to 1.8 meters within 10 minutes, the device can quickly capture and report the specific value with an accuracy of ±1 centimeter.

[0106] Water velocity is monitored using a Doppler flowmeter, which is accurate to 0.1 m / s and can track subtle changes in water velocity in real time. For example, if the water velocity suddenly increases from 1.2 m / s to 1.8 m / s, the device can capture this change within 1 second and transmit it to the system.

[0107] A wind speed sensor, mounted on a high mast, acquires wind speed and direction information on the channel surface. For example, if the wind speed increases from level 3 to level 5 and the wind direction changes from southeast to northeast, the sensor simultaneously records the wind speed (e.g., from 6.7 m / s to 10.2 m / s) and wind direction (e.g., from 135° to 45°), providing data support for analyzing the impact of wind on ship navigation.

[0108] The water temperature, water level, water flow velocity, wind speed and other data collected by four types of sensors together constitute real-time navigation interference information, providing key environmental parameter basis for subsequent water level deviation analysis and early warning water level correction.

[0109] Furthermore, the ship type database is used to match the target ship type.

[0110] Specifically, based on the constructed three-dimensional ship model, the ship type (such as bulk carrier, container ship) is determined by extracting the main dimensions of the hull (such as length, breadth, and depth) and structural features, and comparing them with the standard ship templates in the database.

[0111] Furthermore, with the target ship type and three-dimensional model as constraints, a set of sample navigation interference information of similar ship types under different environmental conditions in historical navigation data was collected (covering scenarios such as water temperature 5℃~35℃, water level fluctuation of ±3 meters, water flow velocity 0~5 meters / second, wind speed 0~6 levels, etc.), and the historical water level deviation ratio under each scenario (that is, the difference between the actual water level line and the standard water level line as a percentage of the ship's depth) was counted to form a sample deviation ratio set.

[0112] For example, in the scenario of water temperature of 25℃, water level height of +2 meters, current speed of 3 meters / second and wind speed of level 5, the actual water level of a bulk carrier is 15 centimeters higher than the standard water level. If the ship is 10 meters deep, the historical water level deviation ratio is (15 centimeters / 10 meters) × 100% = 1.5%; in the scenario of water temperature of 10℃, water level height of -1.5 meters, current speed of 1 meter / second and wind speed of level 2, the actual water level is 8 centimeters lower than the standard water level, and the deviation ratio is (-8 centimeters / 10 meters) × 100% = -0.8%.

[0113] By collecting a large number of deviation data of similar ship types under different environmental combinations, a sample deviation ratio set covering multiple scenarios is formed, providing rich labeled data for the subsequent training of the water level deviation predictor.

[0114] Furthermore, a machine learning algorithm (such as random forest) is used to construct a water level deviation predictor. The sample navigation interference information set is used as input features, and the sample deviation ratio is used as the output label. Through training, the model learns the mapping relationship between environmental parameters and water level deviation.

[0115] First, the collected navigation interference information and the corresponding deviation samples were divided into training and validation sets at a ratio of 8:2. For example, 800 sets of historical data were selected as training sets and 200 sets as validation sets from 1000 sets of historical data to ensure that the data distribution reflects the water level deviation characteristics in different environmental scenarios.

[0116] The training set was then used to train the random forest model. The original environmental parameters were fed into the model, and the decision tree's automatic splitting mechanism was used to learn the relationship between these parameters and the deviation. During training, the model automatically adjusted the splitting conditions for each decision tree, for example, using water temperature as the primary splitting node for a particular tree, and gradually fitting the nonlinear relationship between parameters like water temperature and water velocity and the water level deviation.

[0117] The initial number of decision trees was set at 50 during training, and performance was evaluated on the validation set every time 10 trees were added. After the first round of training, the average error between the predicted and actual deviations on the validation set was 0.12. When the number of trees was increased to 80, this error gradually decreased to 0.06, indicating that the model's ability to fit environmental parameters improved with the increase in the number of decision trees.

[0118] An early stopping mechanism is implemented during training. This mechanism stops training if the validation set error does not decrease for five consecutive epochs (e.g., the error fluctuation is less than 0.01) to prevent overfitting. For example, if the validation set error stabilizes at 0.05 and does not decrease for five consecutive epochs after 30 epochs, early stopping is triggered, indicating that the model has converged.

[0119] Ultimately, the trained predictor can directly receive real-time navigation disturbance information (such as water temperature of 25°C, water level +1.5 meters, current velocity of 2.3 meters per second, and wind speed level 4) and output the corresponding predicted water level deviation. For example, after inputting these parameters, the model outputs a deviation of +2.8%. This result is used to adjust the fixed warning water level to adapt to real-time environmental changes.

[0120] After obtaining the trained water level deviation predictor, the real-time navigation interference information can be input into the predictor to obtain the predicted water level deviation ratio.

[0121] Furthermore, the fixed warning water level is corrected according to the predicted water level deviation ratio, that is, the fixed warning water level is multiplied by the predicted deviation ratio and then added to obtain a warning water level that is adapted to the current navigation environment and output.

[0122] For example, if the fixed warning water level is 80% of the ship's draft and the predicted deviation ratio is +2.8%, the corrected adaptive warning water level is 82.8% of the draft. This ensures that the warning water level can adapt to real-time environmental changes and improves the accuracy and reliability of cargo ship overload identification.

[0123] S300: Analyze and obtain the real-time water level and ship's driving posture according to the ship's three-dimensional model sequence in the preset time zone;

[0124] In the embodiment of the present application, in the scenario of identifying overloaded ships on water, in order to accurately obtain the real-time operating status of the ship, it is necessary to analyze the dynamic changes of the water level and the driving posture based on the three-dimensional model of the time series.

[0125] Specifically, multiple 3D ship models are first acquired at multiple monitoring time points within a preset time zone, forming a 3D ship model sequence. This sequence collects ship morphological data at fixed intervals (e.g., once per second), fully recording the ship's navigation trajectory and morphological changes within the preset time zone (e.g., the last 5 minutes), providing a time series data foundation for subsequent analysis.

[0126] Furthermore, water level detection is performed based on multiple 3D ship models. The intersection of the hull outline and the water surface in the 3D model is used to identify the waterline at consecutive time points. After removing wave fluctuations and environmental noise interference through a filtering algorithm, a sliding average method is used to construct a water level height variation curve. The curve is then averaged to obtain the real-time water level.

[0127] Furthermore, based on the ship's 3D model sequence, the geometric comparison of adjacent models is performed in the order of monitoring time nodes. By calculating parameters such as the main axis vector angle and center of gravity displacement, the rotation and offset between 3D models are identified, and attitude change parameters such as pitch, roll, and yaw angles are extracted.

[0128] Furthermore, the attitude parameter sequence is smoothed using B-spline curve fitting to construct a continuous attitude parameter curve, ultimately fitting the ship's navigation attitude. This attitude data can be used to assist in determining whether the ship suffers from load offset, navigation anomalies, or attitude instability, providing a basis for attitude compensation for waterline analysis.

[0129] Finally, by integrating real-time water level data and ship's driving posture data, a multi-dimensional judgment basis is provided for overload identification, ensuring that water level changes are accurately captured during the ship's dynamic navigation process, and improving the reliability and accuracy of overload identification in complex scenarios.

[0130] Step S300 in the method provided in the embodiment of the present application includes:

[0131] Acquire a three-dimensional ship model sequence, wherein the three-dimensional ship model sequence includes multiple three-dimensional ship models at multiple monitoring time nodes in a preset time zone;

[0132] Performing water level detection based on the multiple three-dimensional ship models, constructing a water level height change curve, and calculating the average to obtain a real-time water level;

[0133] Based on the three-dimensional ship model sequence, geometric comparison of adjacent models is performed in the order of monitoring time nodes, attitude change parameters are extracted, an attitude parameter sequence is constructed, and the ship's driving attitude is obtained by fitting, wherein the attitude parameters include pitch angle, roll angle and yaw angle.

[0134] In an embodiment of the present application, in the process of analyzing the real-time operating status of the ship, in order to accurately obtain the dynamic changes of the water level line and the ship's driving posture, it is necessary to process the ship's three-dimensional model sequence based on the preset time zone to obtain the real-time water level line and the ship's driving posture, providing multi-dimensional data support for cargo ship overload identification.

[0135] First, the ship images at each monitoring time node in the preset time zone are obtained and processed to generate corresponding ship three-dimensional models. These models are then integrated in chronological order to obtain a ship three-dimensional model sequence containing multiple ship three-dimensional models at multiple monitoring time nodes in the preset time zone.

[0136] Furthermore, water level detection is performed on the obtained ship 3D model sequence of multiple ship 3D models, and the real-time water level is calculated through the water level height change curve (constructed by the sliding average method).

[0137] Specifically, when the sliding average method is used to construct the water level height change curve, the water level is first detected based on multiple three-dimensional ship models. The intersection of the hull contour and the water surface of the three-dimensional model is used to identify the waterline within continuous time nodes. The filtering algorithm is used to remove environmental interference such as wave fluctuations and light reflections to obtain the original water level height data at each time node.

[0138] Then, the sliding window size is set (such as selecting the data of the first 5 time nodes), and the mean of the water level height data at each time node within the sliding window is calculated. By moving the sliding window backward one by one, the original data is smoothed to eliminate the influence of random noise, and finally a continuous water level height change curve is constructed.

[0139] Furthermore, the arithmetic mean of all data points in the water level height change curve after smoothing by the sliding average method is taken to eliminate fluctuations caused by accidental factors, and obtain a real-time water level line that can reflect the average draft status of the ship in the preset time zone.

[0140] Furthermore, based on the acquired three-dimensional ship model sequence, the geometric comparison of adjacent models is performed in the order of monitoring time nodes. Specifically, by calculating the main axis vector angle, center of gravity displacement and other parameters of adjacent three-dimensional models, the rotation and offset between models are identified, and the attitude change parameters are extracted.

[0141] Specifically, when calculating the angle between the principal axis vectors of adjacent three-dimensional models, the principal axis vectors of the two adjacent three-dimensional models (such as the longitudinal symmetry axis of the hull) are first extracted respectively, and the cosine value of the angle between the two vectors is calculated using the vector dot product formula. The cosine value is then converted into an angle value to obtain the change in pitch angle or roll angle.

[0142] When calculating the center of gravity displacement, the coordinates of the center of gravity of each three-dimensional model are first determined, and then the displacement size and direction of the center of gravity of adjacent models in three-dimensional space are calculated using the Euclidean distance formula, thereby reflecting the translation state of the ship.

[0143] Through the parameter calculation in the above steps, the rotation and offset between models can be accurately identified, and then the attitude change parameters such as pitch angle, roll angle and yaw angle can be extracted.

[0144] For example, in a sequence of three-dimensional ship models in a preset time zone, for three-dimensional models 1 and 2 in adjacent time nodes, the longitudinal symmetry axis vector V1 of model 1 is first extracted as (0, 0, 1), and the longitudinal symmetry axis vector V2 of model 2 changes to (0, 0.2, 0.98) due to rolling. The cosine value of the angle is calculated to be 0.98 through the vector dot product formula, which is converted to an angle of approximately 11.54° (arccos(0.98)≈11.54°), that is, the roll angle change is 11.54°.

[0145] Assuming the reference direction is the positive x-axis direction (1, 0, 0), the projection of V1 on the horizontal plane (XY plane) is (0, 0), and the projection of V2 on the horizontal plane is (0.1, 0.2). The cosine of the angle between V2's horizontal projection vector (0.1, 0.2) and the reference direction (1, 0) is calculated to be cosθ≈0.447, which converts to an angle of 63.43°, indicating a yaw angle change of 63.43°.

[0146] Assume that the transverse symmetry axis vector of model 1 is (1, 0, 0), and the pitch motion of model 2 causes the transverse symmetry axis vector to become (0.95, 0, 0.3). The cosine value of the angle with the vertical direction (Z axis) is 0.308, which is converted to an angle value of 72.02°, that is, the pitch angle change is 72.02°.

[0147] At the same time, the coordinates of the center of gravity of model 1 are (10, 5, 2), and the coordinates of the center of gravity of model 2 are (12, 5.2, 1.8). The Euclidean distance formula is used to calculate the displacement of approximately 2.02 meters, and the displacement direction is from (10, 5, 2) to (12, 5.2, 1.8). This indicates that the ship moves forward about 2 meters in the X-axis direction, deflects 0.2 meters laterally on the Y-axis, and sinks 0.2 meters on the Z-axis. Combined with the change in the main axis vector angle, it is determined that the ship has attitude changes in roll (11.54°), yaw (63.43°), and pitch (72.02°) during the translation process.

[0148] Furthermore, the roll angle (11.54°), yaw angle (63.43°), and pitch angle (72.02°) are used to construct an attitude parameter sequence in chronological order. The control points and node vectors are set using the B-spline curve fitting algorithm, and the sequence is smoothed. Finally, the ship's driving attitude trajectory in adjacent time periods is obtained by fitting.

[0149] Specifically, the extracted attitude parameters, such as pitch, roll, and yaw, are first arranged in order of monitoring time nodes to form attitude parameter sequence data. Then, using a B-spline curve fitting algorithm, the discrete attitude parameters are smoothed by setting appropriate control points and node vectors. This method preserves the attitude variation trend while filtering out random noise, thereby constructing a continuous attitude parameter curve. Ultimately, the ship's driving attitude within a preset time zone is fitted.

[0150] This attitude data can be used to analyze whether the ship has problems such as load offset, navigation abnormalities or unstable attitude, provide attitude compensation basis for water level detection, and improve the accuracy of overload identification.

[0151] S400: optimizing the real-time water level using the ship navigation data acquired by the ship wireless communication system in combination with the ship's driving posture to obtain a corrected water level;

[0152] In the embodiment of the present application, in the scenario of identifying overloaded ships on water, in order to eliminate the influence of the ship's navigation status and environmental interference on the water level detection, it is necessary to integrate multi-source data to dynamically optimize the real-time water level.

[0153] Specifically, the ship's navigation data, including key parameters such as navigation route and navigation speed, is first integrated and combined with the acquired ship's navigation posture (including pitch angle, roll angle, and yaw angle) to provide multi-dimensional input for water level optimization.

[0154] Furthermore, a waterline recognition error predictor was pre-trained. Based on historical ship monitoring logs, the predictor uses the target ship type and three-dimensional model as constraints, collects sample state data, and associates the corresponding waterline detection errors. This predictor is trained to convergence using a machine learning model to construct a waterline recognition error predictor that can accurately predict errors.

[0155] Furthermore, the trained water level recognition error predictor is used to input parameters such as the current image interference coefficient, route deviation, mean navigation speed, navigation speed fluctuation coefficient and ship driving posture to predict the water level detection error and obtain a quantitative predicted water level detection error value.

[0156] Finally, the real-time water level is compensated based on the predicted water level detection error. By adding the real-time water level value to the predicted error value, the detection deviation introduced by the ship's dynamic operation and environmental interference is eliminated, and the final corrected water level is obtained, providing a more accurate water level determination basis for overload warning.

[0157] Step S400 in the method provided in the embodiment of the present application includes:

[0158] Pre-trained water level recognition error predictor;

[0159] The water level detection error is predicted according to the image interference coefficient, the route deviation degree, the average sailing speed, the sailing speed fluctuation coefficient and the ship driving posture by using the water level line identification error predictor, and a predicted water level detection error is obtained.

[0160] The real-time water level is compensated according to the predicted water level detection error, and a corrected water level is obtained.

[0161] In the embodiment of the application, in the process of optimizing the real-time water level, in order to eliminate the influence of the ship sailing state and environmental interference on the water level detection, precise compensation of the water level is realized through multi-source data fusion and intelligent prediction.

[0162] The step of "pre-training a water level line identification error predictor" in the method provided by the embodiment of the application includes:

[0163] According to the ship monitoring log, the sample state data of the historical ship is collected with the target ship type and the three-dimensional model of the ship as constraints, a sample state data set is constructed, and the historical water level detection error corresponding to different sample state data is obtained, and a sample water level error set is obtained, wherein the sample state data includes historical image interference coefficient, historical route deviation degree, historical average sailing speed, historical sailing speed fluctuation coefficient and historical ship driving posture.

[0164] The sample state data set and the sample water level error set are used as training data, and a machine learning model is trained to convergence, and a water level line identification error predictor is obtained.

[0165] In the embodiment of the application, in order to realize precise prediction of the water level detection error and provide a reliable basis for real-time water level compensation, a water level line identification error predictor needs to be constructed.

[0166] Specifically, first, the target ship type and the three-dimensional model are taken as the screening conditions, and the sample state data such as historical image interference coefficient, historical route deviation degree, historical average sailing speed, historical sailing speed fluctuation coefficient and historical ship driving posture is extracted from the ship monitoring log, and a structured sample state data set is formed.

[0167] At the same time, through the timestamp and ship identification recorded in the ship monitoring log, each sample state data is matched and associated with the corresponding historical water level detection error one by one, and a sample water level error set is constructed, providing labeled data for model training, so that the machine learning model trained subsequently can learn the mapping relationship between the influencing factors and the error.

[0168] Further, the two types of data sets are used as training inputs, and a machine learning model (such as a neural network) is selected for iterative training.

[0169] Specifically, first, the sample state data set and the sample water line error set are divided into a training set and a test set in a ratio of 8:2, for example, 800 groups are extracted from 1000 groups of historical data as the training set, and 200 groups are extracted as the test set, to ensure that the data distribution can fully reflect the water line detection error characteristics under different ship states.

[0170] Secondly, a multi-layer neural network model is constructed, the number of nodes in the input layer corresponds to the feature dimension of the sample state data (such as image interference coefficient, route deviation degree, etc. 5 features), the hidden layer adopts a 2-layer structure (the number of nodes is 10 and 5 respectively), and the number of nodes in the output layer is 1 (corresponding to the water line detection error). During the training process, the Adam optimizer is used to minimize the mean square error loss function, and the initial learning rate is set to 0.001, and the learning rate is attenuated by 0.95 times after every 50 rounds of training.

[0171] At the same time, the early stopping mechanism is implemented during the training phase, that is, the model performance is evaluated every 10 rounds with the test set. At the 10th round, the mean square error on the test set is 0.03; when training to the 60th round, the error is reduced to 0.008, and the error fluctuation of the subsequent 5 consecutive rounds is less than 0.001, then the early stopping mechanism is triggered, and the model is determined to be converged.

[0172] Exemplarily, in a certain group of training data, when the input image interference coefficient is 0.2, the route deviation degree is 10%, the average sailing speed is 8 knots, the speed fluctuation coefficient is 0.15, and the roll angle is 3°, the model predicts that the water line detection error is +0.12 meters, which is within the acceptable range of deviation from the actual error +0.11 meters.

[0173] Finally, the trained water line recognition error predictor can receive the image interference coefficient, route deviation degree, average sailing speed, speed fluctuation coefficient and driving attitude of the current ship, and output the quantized predicted water line detection error. For example, after inputting the real-time parameters, the model outputs an error of +0.15 meters, which will be used for real-time water line compensation correction to improve the accuracy of overload identification.

[0174] After obtaining the trained water line recognition error predictor, input the image interference coefficient, route deviation degree, average sailing speed, sailing speed fluctuation coefficient and ship driving attitude of the current ship into the error predictor, and the error predictor learns the mapping relationship based on historical training data to extract and analyze the features of the input parameters, and outputs the predicted water line detection error.

[0175] Further, according to the output predicted water line detection error, the error value is added to the real-time water line, that is, through the formula "corrected water line = actual water line + predicted error value", to realize the compensation of the real-time water line.

[0176] For example, when the predictor outputs an error of +0.12 meters, the real-time water level is compensated based on the predicted error. If the real-time water level detection value is 3.2 meters, the corrected water level is 3.2 meters + 0.12 meters = 3.32 meters. This eliminates the influence of the ship's driving state and environmental interference on the water level detection, providing an accurate water level determination basis for overload warning.

[0177] S500: If the corrected water level exceeds the adapted warning water level, an overload warning is issued to the target ship.

[0178] In the embodiment of the present application, in the scenario of identifying overloaded ships on water, a reliable early warning trigger mechanism needs to be established to accurately determine whether the cargo ship is overloaded.

[0179] Specifically, the corrected water level line obtained through multi-source data fusion optimization is first numerically compared with the adaptive warning water level line, where the adaptive warning water level line is a dynamic threshold that is corrected based on the ship type database matching and combined with real-time navigation interference information.

[0180] Furthermore, if the corrected water level exceeds the adapted warning water level, an overload warning process is automatically triggered. For example, if the adapted warning water level is 85% of the ship's draft (assuming a draft of 4 meters, the warning threshold is 3.4 meters), and the corrected water level is 3.5 meters, exceeding the threshold by 0.1 meters, the vessel is deemed to be at risk of overloading and the information is sent to the monitoring platform in real time via the waterborne electronic checkpoint system.

[0181] At the same time, the early warning information will include key information such as ship identification, real-time water level data and time, ensuring that the supervision platform can obtain the ship's overload status in a timely manner.

[0182] In addition, for ships that trigger an early warning for the first time, a secondary data verification process will be automatically initiated to ensure the reliability of the early warning results by reviewing the three-dimensional model reconstruction accuracy, water level detection algorithm and error compensation process, thereby avoiding false warnings caused by data fluctuations.

[0183] Ultimately, this mechanism will enable accurate identification and timely warning of overloaded cargo ships, improving the intelligence level of water traffic supervision.

[0184] The “method for image recognition of ship behavior based on water electronic card slots” provided in the embodiment of the present application also includes:

[0185] Multi-angle images of the target ship are acquired through the on-water electronic bayonet capture system;

[0186] According to preset anomaly detection indicators, sample data is collected to train a convolutional neural network and build an abnormal behavior recognition model, where the preset anomaly detection indicators include the ship's unsealed cargo hold, the crew not wearing life jackets, the ship's name being blocked, and the ship not flying the national flag;

[0187] The abnormal behavior recognition model is used to detect abnormal behavior of the multi-angle ship images, and if abnormal behavior exists, an abnormal ship behavior warning is issued.

[0188] In the embodiment of the present application, in the scenario of water vessel safety supervision, in order to comprehensively identify ship violations, it is also necessary to combine the acquisition of multi-angle images with the construction of recognition models to achieve accurate detection of abnormal behavior.

[0189] Specifically, first, based on the three cameras of the water electronic bayonet capture system, multi-angle images of the target ship in the side, front and top directions are synchronously collected, covering key areas such as the hull deck, hatches, bow and stern, to ensure that abnormal behavior characteristics are captured without blind spots.

[0190] For example, the side image can clearly show the ship's name and identification, and the overhead image can fully record the status of deck personnel and hatch closure status.

[0191] Furthermore, based on the sample image data corresponding to the preset abnormality detection indicators, an abnormal behavior recognition model is constructed to achieve accurate identification of four types of abnormal behaviors: the ship's cargo hold is not sealed, the crew members are not wearing life jackets, the ship's name is blocked, and the ship is not flying the national flag.

[0192] Among them, the collection of sample image data is through the historical monitoring records of the water electronic card capture system, and the preset abnormal detection indicators are used as screening conditions to extract ship images covering different ship types, navigation scenarios and environmental conditions.

[0193] For example, for the case of "the ship is not sealed", side and top view images of hatches that are open, unlocked or missing are collected in different weather conditions (sunny, rainy) and time periods (daytime, nighttime); for the case of "crew members not wearing life jackets", multi-angle images of the crew members' wearing status during deck operations are included, including samples of different numbers of people, standing positions and lighting conditions.

[0194] In addition, to address the "ship name obscuration" situation, images of ship name areas obscured by stains, coverings, or bad weather are selected, covering samples of different ship name positions (bow, side); to address the "ship not flying a national flag" situation, forward and side images of ships with no national flag on the bow mast, damaged national flag, or obscured national flag are collected.

[0195] At the same time, in order to ensure sample diversity, images of ships in normal conditions are also included as negative samples to form a data set containing both positive and negative samples, providing a comprehensive feature learning foundation for abnormal behavior recognition model training.

[0196] Furthermore, an abnormal behavior recognition model is constructed with convolutional neural network as the core framework.

[0197] Specifically, the collected sample image data is first divided into a training set and a validation set in an 8:2 ratio. The training set contains a total of 5,000 abnormal samples of various types (1,250 for each type of abnormality), and the validation set contains 1,250.

[0198] Furthermore, the model input layer receives 640×640 pixel image data, extracts basic features through the backbone network, and realizes multi-scale feature fusion through the neck network. That is, shallow features capture small target details such as crew members and national flags, and deep features identify large area anomalies such as hatches and ship names.

[0199] At the same time, a coordinate attention mechanism is introduced during the feature fusion stage to enhance the model's attention to abnormal areas by assigning weights to channels and spatial dimensions. For example, the weight of the area obscured by the ship name is increased by 20%.

[0200] During training, the CIoU loss function was used to calculate the deviation between predictions and annotations. The Adam optimizer was used for parameter updates, with an initial learning rate of 0.001 and a 0.9x decay every 30 epochs. The model performance was evaluated on the validation set every 10 epochs, and the detection accuracy and recall of the four types of abnormal behaviors were recorded.

[0201] For example, after 50 rounds of training, the detection accuracy of "crew members not wearing life jackets" reached 96.3% and the recall rate was 95.7%, the accuracy of "ship name occlusion" reached 94.8% and the recall rate was 93.2%, and the average accuracy of the overall model was 95.1%.

[0202] In addition, an early stopping mechanism is set. That is, when the average accuracy of the validation set increases by less than 0.2% for five consecutive rounds, the early stopping mechanism is triggered to stop training to avoid overfitting. At the same time, it is determined that the trained abnormal behavior recognition model has converged.

[0203] The abnormal behavior recognition model constructed through the above steps can perform real-time analysis on the input multi-angle ship images to accurately output the category of each abnormal behavior.

[0204] For example, after the abnormal behavior recognition model performs reasoning analysis on the overhead image of a ship, the model outputs the analysis results of "the ship is not sealed" and "two crew members are not wearing life jackets", providing an accurate judgment basis for abnormal warning.

[0205] Ultimately, when the abnormal behavior recognition model detects abnormal behavior of a ship and outputs the analysis results, it will automatically trigger an abnormal ship behavior warning to ensure that the abnormal situation is known and handled by the relevant regulators in a timely manner.

[0206] Similarly, the warning information includes basic information such as ship identification number, ship name, location coordinates, abnormality type and location in the image, collection and warning timestamps, as well as abnormal image thumbnails and storage paths. It is pushed to the relevant supervision platform through the water electronic card system to achieve rapid response and tracking and disposal of abnormal ship behavior.

[0207] For example, when a bulk carrier enters the monitoring range of the water electronic checkpoint, the image collected by the side camera is analyzed by the abnormal behavior recognition model, and the output is "the ship name is blocked by the blue waterproof cloth". The overhead image detects the abnormal behavior analysis result of "3 deck crew members are not wearing life jackets". The system then generates an early warning information, including the ship's identification number, ship name, real-time location coordinates, marking the area where the ship name is blocked and the position where the national flag is not hoisted, and attached with the collection time, warning time and abnormal image thumbnail.

[0208] At the same time, the generated warning information will be immediately pushed to the relevant supervision platform through the water electronic checkpoint system. The platform's electronic nautical chart will simultaneously mark the ship's position and issue an audible and visual alarm to ensure that the supervisory party is aware of the abnormal situation in a timely manner.

[0209] The embodiments of the present application achieve the following technical effects through the above specific implementation methods:

[0210] This application proposes a method for image recognition of ship behavior based on an on-water electronic card slot, which is used to identify overloaded cargo ships. This method constructs a full-process system from data acquisition to early warning through multi-source data fusion and dynamic parameter optimization. First, the three cameras of the on-water electronic card slot capture system are used to collect multi-angle images of the ship. After processing through a multi-view convolutional neural network, a three-dimensional model of the ship is constructed. Secondly, an adaptive early warning water level is generated by combining a ship type database with real-time navigation interference information. At the same time, the sequence of three-dimensional ship models in a preset time zone is analyzed to obtain the real-time water level and driving posture. The water level is then optimized by integrating the ship's navigation data to form a corrected water level. If the corrected water level exceeds the adaptive early warning water level, the system automatically triggers an overload early warning and pushes it to the supervision platform. The initial early warning will also initiate a secondary data verification. If the abnormal behavior recognition model detects an anomaly, it will also automatically trigger a ship behavior abnormality early warning, ensuring that all abnormal situations are promptly known and handled by the supervisory authority.

[0211] The method provided in the embodiment of the present application solves the problems of single data and fixed thresholds in traditional cargo ship overload identification. Through the steps of "image acquisition-3D reconstruction-dynamic correction-intelligent early warning", it improves the adaptability and accuracy of overload identification in complex water environments, avoids the risks of false detection and missed detection due to environmental interference and changes in ship posture, and provides an innovative solution for the intelligent and comprehensive supervision of water traffic.

[0212] It should be noted that the order in which the embodiments of the present application are presented is for illustrative purposes only and does not necessarily represent the superiority or inferiority of the embodiments. Furthermore, the foregoing descriptions of specific embodiments of this specification are provided. Furthermore, the processes depicted in the accompanying drawings do not necessarily require the specific order or sequential sequence shown to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0213] The above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application should be included in the scope of protection of the present application.

[0214] This specification and drawings are merely illustrative of the present application and are intended to cover any and all modifications, variations, combinations, or equivalents within the scope of this application. Obviously, those skilled in the art may make various modifications and variations to this application without departing from the scope of this application. Thus, this application is intended to include such modifications and variations as fall within the scope of this application and its equivalents.

Claims

1. A method for image recognition of ship behavior based on an electronic bayonet on water, characterized in that: Methods include: The multi-angle images of the target ship are collected through the water electronic bayonet capture system to perform 3D reconstruction of the hull and build a 3D model of the ship; Using a ship type database, a fixed warning water level is obtained based on the three-dimensional model of the ship, and the fixed warning water level is corrected according to real-time navigation interference information to output an adapted warning water level; According to the ship's 3D model sequence in the preset time zone, the real-time water level and ship's driving posture are analyzed; Optimizing the real-time water level using the ship navigation data acquired by the ship wireless communication system in combination with the ship's driving posture to obtain a corrected water level; If the correction water level exceeds the adaptation warning water level, an overload warning is issued to the target ship; The fixed warning water level is corrected according to the real-time navigation interference information, and an adapted warning water level is output, including: Collecting interference information based on preset navigation interference indicators to obtain real-time navigation interference information, wherein the preset navigation interference indicators include water temperature, water level, water flow velocity, and wind speed; Using a ship type database, the target ship type is obtained based on the three-dimensional ship model. With the target ship type and the three-dimensional ship model as constraints, a sample navigation interference information set is collected, and the historical water level deviation ratio under different sample navigation interference information is obtained, which is set as the sample deviation ratio to obtain a sample deviation ratio set; Using the sample navigation interference information set and the sample deviation ratio set as training data, a water level deviation predictor is constructed based on machine learning, and a predicted water level deviation ratio is obtained according to the real-time navigation interference information analysis; Correcting the fixed warning water level according to the predicted water level deviation ratio and outputting an adapted warning water level; The ship navigation data obtained by the ship wireless communication system is used to optimize the real-time water level line in combination with the ship's driving posture to obtain a corrected water level line, including: Pre-trained water level recognition error predictor; Using the water level recognition error predictor, the water level detection error is predicted according to the image interference coefficient, the route deviation, the mean sailing speed, the sailing speed fluctuation coefficient and the ship's driving posture to obtain a predicted water level detection error; The real-time water level is compensated according to the predicted water level detection error to obtain a corrected water level.

2. The method for image recognition of ship behavior based on water electronic card slots according to claim 1 is characterized in that: Construct an electronic bayonet capture system on water, wherein the electronic bayonet capture system on water includes a first camera, a second camera, and a third camera. The first camera is set on a fixed bracket on the left bank of the waterway, 3 meters above the water surface, and the shooting direction is at a 90° angle to the centerline of the waterway, and the shooting angle is a lateral horizontal perspective. The second camera is installed on a high pole bracket 40 meters upstream of the first camera along the channel, 5 meters above the water surface, with a shooting direction parallel to the centerline of the channel and a shooting angle of a positive forward horizontal perspective; The third camera is set on a high pole bracket above the midpoint of the line connecting the first camera and the second camera, at a height of 8 meters from the water surface, with a shooting direction looking downward and a shooting angle of 45°.

3. The method for image recognition of ship behavior based on water electronic bayonet according to claim 1, characterized in that: Collect multi-angle images of the target ship, perform 3D reconstruction of the hull, and build a 3D model of the ship, including: When the target ship enters the coverage area of ​​the water electronic bayonet capture system, multi-angle images of the target ship are collected to obtain a multi-angle ship image; On the cloud server, a 3D reconstruction model of the hull is constructed based on a multi-view convolutional neural network and transferred to the edge processing unit of the on-water electronic bayonet capture system. The 3D reconstruction model of the hull includes K 3D reconstruction units of the hull. Obtaining real-time image influencing factors, performing image quality assessment, and determining an image interference coefficient, wherein the influencing factors include at least illumination intensity, occlusion ratio, and image blur; A quantity P is selected according to the image interference coefficient setting unit, P hull three-dimensional reconstruction units are randomly selected from the K hull three-dimensional reconstruction units, three-dimensional reconstruction is performed on the multi-angle ship image to obtain P initial ship three-dimensional models, and model fitting is performed on the P initial ship three-dimensional models to obtain a ship three-dimensional model.

4. The method for image recognition of ship behavior based on water electronic bayonet according to claim 3 is characterized in that: The 3D reconstruction model of the ship hull is constructed based on a multi-view convolutional neural network, including: According to the ship monitoring log of the water electronic bayonet capture system, a sample multi-angle ship image set is collected, and the sample multi-angle ship images are reconstructed using a three-dimensional reconstruction algorithm to obtain a sample ship three-dimensional model set; The sample multi-angle ship image set and the sample three-dimensional ship model set are used as training data, divided into K equal parts, and selected with replacement K times to obtain a first training set, and the selection is iterated K times to obtain K training sets, where K is an integer greater than 5; The K training sets are used to train the multi-view convolutional neural network until convergence, to obtain K hull three-dimensional reconstruction units, and to construct a hull three-dimensional reconstruction model.

5. The method for image recognition of ship behavior based on water electronic bayonet according to claim 3 is characterized in that: Selecting the quantity P according to the image interference coefficient setting unit includes: Acquiring ship navigation data using a ship wireless communication system, wherein the ship navigation data includes a navigation route and a navigation speed; The route deviation, the mean sailing speed and the sailing speed fluctuation coefficient of the preset historical time zone are calculated based on the sailing route and the sailing speed; Determining the complexity of hull reconstruction based on the route deviation, the mean sailing speed, the sailing speed fluctuation coefficient, and the image interference coefficient, wherein the complexity of hull reconstruction is positively correlated with the route deviation, the mean sailing speed, the sailing speed fluctuation coefficient, and the image interference coefficient; The ratio of the hull reconstruction complexity to the historical maximum hull reconstruction complexity is multiplied by K and rounded to the integer to obtain the number P of unit selections.

6. The method for image recognition of ship behavior based on water electronic bayonet according to claim 1, characterized in that: According to the ship's 3D model sequence in the preset time zone, the real-time water level and ship's driving posture are analyzed, including: Acquire a three-dimensional ship model sequence, wherein the three-dimensional ship model sequence includes multiple three-dimensional ship models at multiple monitoring time nodes in a preset time zone; Performing water level detection based on the multiple three-dimensional ship models, constructing a water level height change curve, and calculating the average to obtain a real-time water level; Based on the three-dimensional ship model sequence, geometric comparison of adjacent models is performed in the order of monitoring time nodes, attitude change parameters are extracted, an attitude parameter sequence is constructed, and the ship's driving attitude is obtained by fitting, wherein the attitude parameters include pitch angle, roll angle and yaw angle.

7. The method for image recognition of ship behavior based on water electronic bayonet according to claim 1, characterized in that: Pre-trained waterline recognition error predictor, including: Based on the ship monitoring logs, with the target ship type and the ship's three-dimensional model as constraints, we collected sample state data of historical ships, constructed a sample state dataset, and obtained the historical water level detection errors corresponding to different sample state data to obtain a sample water level error set. The sample state data includes the historical image interference coefficient, historical route deviation, historical navigation speed mean, historical navigation speed fluctuation coefficient, and historical ship driving posture. The sample state data set and the sample water level error set are used as training data, and a machine learning model is trained until convergence to obtain a water level recognition error predictor.

8. The method for image recognition of ship behavior based on water electronic bayonet according to claim 1, characterized in that: The method also includes: Multi-angle images of the target ship are acquired through the on-water electronic bayonet capture system; According to preset anomaly detection indicators, sample data is collected to train a convolutional neural network and build an abnormal behavior recognition model, where the preset anomaly detection indicators include the ship's unsealed cargo hold, the crew not wearing life jackets, the ship's name being blocked, and the ship not flying the national flag; The abnormal behavior recognition model is used to detect abnormal behavior of the multi-angle ship images, and if abnormal behavior exists, an abnormal ship behavior warning is issued.

Images