A ship water gauge recognition method, system and device based on weakly supervised deep learning

Abstract

The present invention proposes a ship draft gauge recognition method, system and device based on weakly supervised deep learning, aiming to solve the accuracy problems in the process of automatic identification of ship draft gauges under complex environmental conditions, the dependence on a large amount of accurately labeled data and the high consumption of computing resources. By adopting a weakly supervised learning strategy, the demand for accurately labeled data is significantly reduced, while the generalization ability and recognition accuracy of the model are improved by using unlabeled data. The specially designed deep learning model can adapt to various complex environmental conditions, such as different light intensities and water surface reflections, ensuring high-accuracy recognition of ship draft gauge readings. In addition, through efficient management of model optimization and training processes, the consumption of computing resources is significantly reduced, making the technology more suitable for deployment on devices with limited computing power, providing an efficient and low-cost solution for the automated detection of ship draft gauge readings.

Description

Technical field:

[0001] The present invention relates to the field of ship monitoring technology, and in particular to a ship water gauge recognition method, system and device based on weakly supervised deep learning. Background technology:

[0002] Ship draft gauge recognition is an important requirement in the shipping industry, which is related to ship safety, transportation efficiency, cargo management and other aspects. The traditional draft gauge reading method mainly relies on artificial vision for recognition, which often encounters accuracy problems in bad weather conditions or poor lighting. In order to improve recognition accuracy and reduce labor costs, computer vision, machine learning and deep learning technologies have been gradually introduced into the draft gauge recognition process in recent years.

[0003] Early automation attempts were mainly based on image processing techniques, using methods such as edge detection and threshold segmentation to identify water gauge marks. However, these methods are particularly sensitive to environmental conditions and require manual adjustment of parameters, limiting their wide application and accuracy. Subsequently, machine learning techniques such as support vector machines (SVMs) and decision trees were introduced to try to improve recognition accuracy through manual feature engineering, but these methods are limited by the ability of the model to handle complexity and require a lot of manual intervention in feature extraction.

[0004] The emergence of deep learning technology, especially convolutional neural networks (CNN), has significantly improved the accuracy and environmental adaptability of recognition through automatic feature extraction and training with large amounts of data. This marks an important advancement in the automatic recognition of ship water gauges. However, deep learning methods require a large amount of annotated data and expensive computing resources, which poses a substantial challenge for resource-constrained application scenarios.

[0005] In view of the above situation, there is an urgent need for a solution that can effectively improve the accuracy of ship water gauge reading detection and reduce the dependence on a large amount of labeled data and computing resources. The present invention introduces a method based on weakly supervised deep learning to solve the key challenges faced by the existing technology in the process of automated water gauge recognition, including improving the recognition accuracy under complex environmental conditions, reducing the demand for manually labeled data, enhancing the environmental adaptability of the model, and reducing the consumption of high-performance computing resources. Summary of the invention:

[0006] The purpose of the present invention is to address the defects of the prior art and provide a ship draft gauge recognition method, system and device based on weakly supervised deep learning, aiming to solve the key challenges in the automatic recognition of ship draft gauges, such as accuracy issues under environmental conditions, dependence on a large amount of precisely labeled data, and high computing resource consumption.

[0007] To achieve the above purpose, the present invention provides the following technical solution: a ship water gauge recognition method based on weakly supervised deep learning, the specific method steps are as follows:

[0008] S1: Acquisition of water level image, collecting ship image datasets from multiple sources, including but not limited to ship water level images under different weather conditions, at different time periods and at different angles;

[0009] S2: Image preprocessing, preprocessing the acquired water gauge image, including image scaling, rotation, and lighting adjustment operations, to enhance image quality and prepare for subsequent recognition steps;

[0010] S3: Data annotation, including the position and reading of the water gauge numbers and the character "M", using a weakly supervised learning strategy and using unlabeled images to assist training through a semi-supervised learning method;

[0011] S4: Build a deep learning model based on the YOLOv8 and SAM deep learning frameworks, and train it for the ship water gauge recognition task. During the model training process, the cross-validation method is used to optimize the model parameters to achieve accurate recognition of the ship water gauge numbers.

[0012] S5: Draft reading recognition: Use the trained deep learning model to process the new ship draft image, automatically recognize the draft reading in the image, and calculate the actual draft value;

[0013] S6: Ship status output, based on the water gauge reading and the known draft of the ship, the current load status of the ship is comprehensively calculated.

[0014] As a preferred technical solution of the present invention, the specific operation of acquiring the water level image described in S1 is: collecting a diverse ship image data set, including images at different angles, lighting conditions and backgrounds, as well as clear and partially blocked or difficult to identify water level mark images. The data set is composed of ship water level data collected in real business scenarios. After preprocessing, 3,000 images are selected for model training and testing.

[0015] As a preferred technical solution of the present invention, the specific operation of data annotation described in S3 is: using a weakly supervised learning method, only a subset of the data set is subjected to detailed water level marking and reading annotation, while the remaining data is not labeled in detail or is only simply annotated. The operation of simple annotation is to annotate the image with ships but not to specifically indicate the water level marking.

[0016] As a preferred technical solution of the present invention, the deep learning model described in S4 designs a set of anchor boxes for the shape and size of the water gauge mark, and uses the k-means clustering algorithm on the training data set to determine the optimal anchor box size and ratio. The distance metric used in the clustering process is the IoU (Intersection over Union) distance, and the calculation formula is as follows:

[0017] D IoU (box,centroid)=1-IoU(box,centroid)

[0018] Among them, D IoU represents the IoU distance, box is a bounding box in the dataset, and centroid is the cluster center (i.e. the selected anchor box).

[0019] As a preferred technical solution of the present invention, the training of the deep learning model described in S4 adopts weakly supervised learning, and the specific training process is:

[0020] T1: Pseudo label generation: For each sample x in the unlabeled dataset u , use the trained model f to generate pseudo labels

[0021]

[0022] Among them, f(x u ) represents the model for unlabeled samples x u The predicted distribution of , the argmax operation selects the category with the highest probability as the pseudo label;

[0023] T2: Pseudo label training: The generated pseudo labels Compared with the original labeled data y l They are used together for training and updating the model. The loss function s during training consists of two parts. One part is the loss L of labeled data. l , and the other part is the loss of pseudo-label data L u :

[0024]

[0025] Among them, x l and l are labeled samples and their labels, L l and L u Respectively represent the loss functions of labeled data and pseudo-labeled data, and λ is a weight parameter that adjusts the importance of both;

[0026] T3: Confidence threshold: In order to improve the quality of pseudo-labels, usually only unlabeled samples whose model prediction confidence is higher than a certain threshold τ are selected for training:

[0027]

[0028] Among them, s represents the maximum prediction probability of the model for the selected category. Only when this probability is greater than the threshold τ, the corresponding unlabeled samples will be used for training.

[0029] As a preferred technical solution of the present invention, the training of the deep learning model described in S4 adopts a weighted loss function, which is as follows:

[0030] Classification loss function: Use multivariate cross entropy to determine whether a character belongs to this category. The specific calculation formula is:

[0031]

[0032] Where b is the number of categories; y ic is a sign function (0 or 1); if the true category of sample i is equal to c, it takes 1, otherwise it takes 0; p ic is the predicted probability that the observed sample i belongs to category c;

[0033] Positioning loss: Use the squared error loss function L loc , calculate the difference between the predicted bounding box and the true bounding box;

[0034] Classification loss: Use the cross entropy loss function L cls , measures the difference between the predicted class and the actual class;

[0035] Confidence loss: Use confidence loss L conf , a measure of the model’s confidence in the correctness of its predictions;

[0036] The total loss function is the weighted sum of these three parts:

[0037] L total =λ loc L loc +λ cls L cls +λ conf L conf

[0038] Among them, λ loc , cls and λ conf It is a weight parameter used to balance different loss parts.

[0039] As a preferred technical solution of the present invention, the calculation method of the water gauge reading described in S5 is:

[0040] S5-1: By extracting the recognition box with the largest coordinates among all recognition boxes, obtain the coordinates of its upper left corner (x1, y1) and lower right corner (x2, y2), thereby calculating the center coordinates (x0, y0); secondly, obtain all characters "M" according to the category index and sum their coordinates, compare and record the index with the largest coordinate sum, and according to a similar method, select the character with a smaller horizontal coordinate, compare their coordinate sums, obtain the maximum value, and obtain the integer part parameter N;

[0041] S5-2: In the water gauge character recognition, the upper character closest to the waterline is recorded as F, and the decimal part of the reading can be calculated based on F;

[0042] S5-3: Eliminate the data of all characters to the left of "M" through clustering method, sort the remaining data in ascending order according to the y-axis coordinates and perform list matching with the template [10, 8, 6, 4, 2, 10, 8, 6, 4, 2, ...]. A successful match indicates that the recognition is correct. At this time, subtract the y2 values ​​of the water ruler characters in pairs, and calculate the average value as M, which is the pixel length corresponding to 0.2m;

[0043] S5-4: Finally, the water gauge reading R can be calculated:

[0044]

[0045] Wherein, H is the pixel length of the line segment obtained by drawing a perpendicular line from the bottom midpoint coordinate of the identification box with the largest coordinate in step S5-1 to the waterline segmentation area.

[0046] A ship draft gauge recognition system based on weakly supervised deep learning is used to complete the above-mentioned recognition method. The system includes: a draft gauge image acquisition module, an image preprocessing module, a draft gauge character recognition module, a draft gauge waterline segmentation module and a draft gauge reading recognition module.

[0047] A ship water gauge identification device based on weakly supervised deep learning, the device comprising: at least one processor, a display device and a memory connected to the at least one processor in communication, the memory storing instructions executed by the at least one processor, the instructions being executed by the at least one processor so that the at least one processor executes the above-mentioned identification method.

[0048] Compared with the related prior art, the present invention has the following technical advantages and the beneficial effects of the present invention are:

[0049] Higher recognition accuracy: The present invention can significantly improve the recognition accuracy of ship draft readings through the application of deep learning models, especially in the context of using a large amount of unlabeled data, through the optimization of weakly supervised learning strategies. Compared with existing supervised deep learning methods, the recognition accuracy on unknown samples is significantly improved.

[0050] Enhanced adaptability: Using deep learning technology, the present invention can adapt to various complex environmental conditions, including different light intensities, weather conditions, and water surface reflections.

[0051] Reduce the need for labeled data: Through weakly supervised learning methods, the present invention significantly reduces the reliance on accurately labeled data. Compared with traditional deep learning methods, it reduces the need for labeled data by about 70%, greatly reducing data preparation costs and time.

[0052] Reduced computing resource consumption: This invention reduces the reliance on high-performance computing resources through efficient management of model optimization and training processes; compared with traditional deep learning methods, computing resource consumption is reduced by about 40%, making this technology more suitable for deployment on devices with limited computing power. Description of the drawings:

[0053] Figure 1 is a flow chart of the method provided by the present invention;

[0054] Figure 2 It is a structural diagram of the device provided by the present invention;

[0055] Figure 3 It is a structural diagram of the device provided by the present invention;

[0056] Figure 4 It is an implementation flow chart of a specific embodiment provided by the present invention;

[0057] Figure 5 It is a YOLOv8 water ruler character recognition image of a specific embodiment provided by the present invention;

[0058] Figure 6 It is an image before and after SAM waterline segmentation of a specific embodiment provided by the present invention;

[0059] Figure 7 is a final ship water gauge recognition image of a specific embodiment provided by the present invention;

[0060] Figure 8 It is a sample image of a water gauge data set of a specific embodiment provided by the present invention. Specific implementation method:

[0061] The present invention is further described below in conjunction with the accompanying drawings and examples. However, the present invention can be implemented in many different ways and should not be construed as being limited to the embodiments shown; on the contrary, these embodiments provide those skilled in the art with implementation methods that meet applicable legal requirements.

[0062] Embodiment 1: According to Figure 1As shown, the present invention provides a ship water gauge recognition method based on weakly supervised deep learning, and the specific steps are as follows:

[0063] S1: Acquisition of water level image, collecting ship image datasets from multiple sources, including but not limited to ship water level images under different weather conditions, at different time periods and at different angles;

[0064] S2: Image preprocessing, preprocessing the acquired water gauge image, including image scaling, rotation, and lighting adjustment operations, to enhance image quality and prepare for subsequent recognition steps;

[0065] S3: Data annotation, including the position and reading of the water gauge numbers and the character "M", using a weakly supervised learning strategy and using unlabeled images to assist training through a semi-supervised learning method;

[0066] S4: Build a deep learning model based on the YOLOv8 and SAM deep learning frameworks, and train it for the ship water gauge recognition task. During the model training process, the cross-validation method is used to optimize the model parameters to achieve accurate recognition of the ship water gauge numbers.

[0067] S5: Draft reading recognition: Use the trained deep learning model to process the new ship draft image, automatically recognize the draft reading in the image, and calculate the actual draft value;

[0068] S6: Ship status output, based on the water gauge reading and the known draft of the ship, the current load status of the ship is comprehensively calculated.

[0069] In order to implement the above method, Figure 2 As shown, the present invention provides a ship draft gauge recognition system based on weakly supervised deep learning. The system of the present invention includes: a draft gauge image acquisition module, an image preprocessing module, a draft gauge character recognition module, a draft gauge waterline segmentation module and a draft gauge reading recognition module. These modules cooperate with each other to automatically complete the entire process from image acquisition to draft depth calculation, effectively improving the automation level and accuracy of ship draft depth detection, and reducing labor costs and error rates.

[0070] In order to implement the above method, Figure 3 As shown, the present invention provides a ship water level identification device based on weakly supervised deep learning. The device of the present invention includes: at least one processor, a display device and a memory communicatively connected to the at least one processor, the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor.

[0071] Embodiment 2: According to Figure 4As shown, the following will use a specific real case to demonstrate how to use the ship water gauge recognition method based on weakly supervised deep learning provided by the present invention:

[0072] like Figure 4 As shown in FIG. 1 , the present invention collects a diverse ship image dataset, such as Figure 5 As shown, it specifically includes images at different angles, lighting conditions and backgrounds, as well as clear and partially blocked or difficult to identify water gauge mark images. The dataset consists of ship water gauge data collected in real business scenarios. After preprocessing, 3000 images are selected for model training and testing. In terms of annotation strategy, a weakly supervised learning method is adopted, and only a subset (1000 images) of the dataset is annotated with detailed water gauge marks and readings, while the remaining data is not annotated in detail or only simply annotated (the ship exists in the annotated image but the water gauge mark is not specifically specified). This strategy aims to reduce the workload of large-scale data annotation, while using unlabeled data to improve the generalization ability of the model. In order to enhance the adaptability and robustness of the model to different environmental changes, the present invention implements a series of data enhancement techniques. This includes geometric transformation of the image (such as rotation, scaling), adjusting the lighting conditions of the image, and adding synthetic occlusion of the water gauge mark. These data enhancement techniques help the model learn to accurately identify water gauge marks in various complex environments, and improve recognition accuracy and stability.

[0073] like Figure 4 -B, in the present invention, an efficient ship water gauge character recognition system is proposed, which adopts the YOLOv8 algorithm framework and combines deep learning technology to realize real-time and accurate recognition of ship water gauge characters. The system can recognize water gauge characters composed of numbers 0-9 and the character "M". Through a specially constructed training data set, the system achieves high-precision recognition. During the recognition process, the system first determines the "M" character closest to the water surface, then identifies the water gauge digits on its left, and accurately converts them into actual water gauge readings. In addition, the system can also automatically generate a mask frame of a specified size based on the water gauge character closest to the water surface, so as to further analyze or process the image information in the mask frame.

[0074] A deep convolutional neural network is used. This network architecture deeply mines image features, including multiple convolutional layers, residual connections and attention mechanisms, which greatly enhances the model's ability to recognize ship water gauge details. At the same time, by implementing multi-scale detection at different levels, the system can adapt to the size changes of water gauge marks at different distances and angles, thereby improving the accuracy and stability of recognition. In order to further improve the detection accuracy, the present invention also specially designs anchor boxes for the shape and size of water gauge marks to optimize the target detection process.

[0075] The size and proportion of the anchor box are critical to improving the accuracy of object detection. The optimization of the anchor box is performed on the training data set through the k-means clustering algorithm to determine the optimal anchor box size and proportion. The distance metric used in the clustering process is the IoU (Intersection over Union) distance, which is calculated as follows:

[0076] D IoU (box,centroid)=1-IoU(box,centroid).

[0077] Among them, D IoU represents the IoU distance, box is a bounding box in the dataset, and centroid is the cluster center (i.e. the selected anchor box).

[0078] In response to the challenges of weakly supervised learning environments, a model is built to implement weakly supervised learning. The model uses labeled data for initial training and unlabeled data for further learning and optimization. Pseudo-labeling technology is used to use its own predictions to guide the learning process of unlabeled data.

[0079] Pseudo label generation: For each sample x in the unlabeled dataset u , use the trained model f to generate pseudo labels

[0080]

[0081] Here, f(x u ) represents the model for unlabeled samples x u The argmax operation selects the category with the highest probability as the pseudo label.

[0082] Pseudo label training: Generate pseudo labels Compared with the original labeled data y l They are used together for training and updating the model. The loss function L in the training process consists of two parts. One part is the loss L of the labeled data. l , and the other part is the loss of pseudo-label data L u :

[0083]

[0084] Among them, x l and l are labeled samples and their labels, L l and L u They represent the loss functions for labeled data and pseudo-labeled data respectively, and λ is a weight parameter that adjusts the importance of both.

[0085] Confidence threshold: In order to improve the quality of pseudo-labels, usually only unlabeled samples whose model prediction confidence is higher than a certain threshold τ are selected for training:

[0086]

[0087] Among them, max(f(x u )) represents the maximum predicted probability of the model for the selected category. Only when this probability is greater than the threshold τ, the corresponding unlabeled samples will be used for training.

[0088] In the waterline segmentation task, the SAM model demonstrates its ability to accurately analyze complex water scenes through its highly general and flexible framework. The model uses an advanced convolutional neural network architecture, combined with deep learning technology, to efficiently extract features and classify pixels on images containing water surfaces. By training on a large amount of image data of water areas and their boundary areas, the SAM model can accurately identify and segment the boundaries between water surfaces and other objects. In addition, the training process of the model involves sophisticated loss function design, as well as subsequent optimization strategies, such as model fusion and conditional random field applications, which further improve the segmentation accuracy. In order to improve the reasoning efficiency of the SAM model, the present invention performs SAM waterline segmentation on the image within the mask frame obtained after YOLOv8 processing. The effects before and after segmentation are shown as follows. Figure 6 shown.

[0089] Due to the interference of water gauge characters and water surface reflection, the SAM waterline segmentation results have errors and need to be corrected. Then, the waterline is sampled from head to tail to fit into a straight line and the midpoint of the waterline is taken out. At this time, the vertical line segment between the water gauge character at the top of the waterline and the waterline is drawn and its pixel distance H is calculated. The final ship water gauge recognition effect is as follows: Figure 7 shown.

[0090] It is known that the height of the ship's water gauge character is specified to be 0.1m, and the distance between the upper and lower characters is also 0.1m. After the above steps, the water gauge character recognition and waterline segmentation tasks are realized. The water gauge reading calculation method is as follows:

[0091] (1) First, extract the recognition box with the largest coordinates among all the recognition boxes, obtain its upper left corner (x1, y1) and lower right corner coordinates (x2, y2), and thus calculate the center coordinates (x0, y0); secondly, obtain all the characters "M" according to the category index and sum their coordinates, compare and record the index with the largest coordinate sum, and according to a similar method, select the character with a smaller horizontal coordinate, compare their coordinate sums, obtain the maximum value, and obtain the integer part of the parameter N.

[0092] (2) In water gauge character recognition, the upper character closest to the waterline is denoted as F, and the decimal part of the reading can be calculated based on F.

[0093] (3) The data of all characters to the left of "M" are eliminated through the clustering method, and the remaining data are sorted in ascending order according to the y-axis coordinates and matched with the template [10, 8, 6, 4, 2, 10, 8, 6, 4, 2, ...]. A successful match indicates that the recognition is correct. At this time, the y2 values ​​of the water ruler characters are subtracted in pairs, and the average value is counted as M, where M is the pixel length corresponding to 0.2m.

[0094] (4) Finally, the water gauge reading R (unit: m) can be calculated:

[0095]

[0096] like Figure 4 -C, in the present invention, in terms of the division of the data set, the data set is divided according to a ratio of 4:1, as shown in 8, of which 800 pictures are used as training sets for model training and parameter optimization, and the other 200 pictures are used as validation sets to evaluate the performance and generalization ability of the model. The experimental operating system is Windows 11, and the software environment configuration is: Python 3.10, CUDA 11.7, cuDNN 8.5, OpenCV 4.6.0, Pytorch1.13.0. The hardware configuration environment is: graphics card GeForce RTX 309024GB, CPU AMD Ryzen 75700X 8-Core Processor, memory 32GB DDR4 2400MHz.

[0097] For the YOLOv8 water ruler character recognition model training, the Adam optimizer is used, the initial learning rate is 0.01, the weight decay factor is 0.0005, and the dropout regularization is used. The training is 100 cycles in total. The training uses multivariate cross entropy as the classification loss function to determine whether a character belongs to this category. The formula is:

[0098]

[0099] Where b is the number of categories; y ic is a sign function (0 or 1); if the true category of sample i is equal to c, it takes 1, otherwise it takes 0; p ic is the predicted probability that the observed sample i belongs to category c. At the same time, the present invention designs a loss function that can adapt to the weak label nature of the data set, including the unsupervised loss of unlabeled data, which can effectively balance the contribution of fully labeled data and unlabeled data.

[0100] Positioning loss: usually using the square error loss function L loc , computing the difference between the predicted bounding box and the true bounding box.

[0101] Classification loss: Use the cross entropy loss function L cls , which measures the difference between the predicted class and the actual class.

[0102] Confidence loss: measures the confidence of the model in the correctness of its predictions, usually using confidence loss L conf .

[0103] The total loss function is the weighted sum of these three parts:

[0104] L total =λ loc L loc +λ cls L cls +λ conf L conf

[0105] Among them, λ loc , cls and λ conf It is a weight parameter used to balance different loss parts.

[0106] Training strategy: Use a training strategy that gradually introduces unlabeled data, starting with a model trained on the labeled subset. Techniques such as pseudo-labeling or self-training can be used to gradually refine the model with unlabeled data.

[0107] Regularization and Optimization: Implement regularization techniques (such as dropout, L2 regularization) to prevent overfitting. Select the Adam optimization algorithm and fine-tune the learning rate and other hyperparameters based on the validation performance.

[0108] like Figure 4 -D shows that the accuracy is used to evaluate the YOLOv8 water gauge character recognition model, the mean intersection over union (MIoU) and pixel accuracy (PA) are used to evaluate the SAM waterline segmentation model, and finally the MAE (Mean Absolute Error) is used to evaluate the overall model performance. MAE can better reflect the actual situation of the error between the predicted value and the true value of the water gauge reading. The formula is as follows:

[0109]

[0110] Where n is the number of samples, y i is the true value, is the predicted value.

[0111] Through the detailed description of the above embodiments, the present invention successfully demonstrates the effectiveness and practicality of the ship water gauge recognition method based on weakly supervised deep learning. By combining the YOLOv8 algorithm framework and deep learning technology, the present invention not only significantly improves the recognition accuracy of ship water gauge readings, but also optimizes the weakly supervised learning strategy by utilizing a large amount of unlabeled data, significantly reducing the dependence on accurately labeled data. In addition, by adopting preprocessing and data enhancement technology, the adaptability and robustness of the model to various complex environmental conditions are enhanced, so that the technology can work stably in variable environments such as different light intensities, weather conditions, and water surface reflections.

[0112] In particular, the present invention effectively solves the problem of size variation of water gauge marks at different distances and angles through a carefully designed deep learning model structure and training strategy, ensuring high-accuracy recognition results. The optimization of the anchor box further improves the accuracy of target detection, while the pseudo-labeling technology in the weakly supervised learning strategy effectively utilizes unlabeled data and reduces the model's need for a large amount of labeled data.

[0113] In addition, the embodiments of the present invention have successfully demonstrated how to significantly reduce the consumption of high-performance computing resources through automatic feature extraction and learning of deep learning technology, as well as optimization of the model and efficient management of the training process. This makes the present invention suitable not only for high-performance computing environments, but also for deployment on devices with limited computing power, providing an efficient and economical solution for the automated detection of ship water gauge readings.

[0114] In summary, the ship draft gauge recognition method based on weakly supervised deep learning provided by the present invention has significant advantages such as higher recognition accuracy, enhanced environmental adaptability, significantly reduced annotation data requirements and computing resource consumption, etc., which brings innovative technical improvements and practical application value to the field of ship draft gauge reading detection.

[0115] The above embodiments only express the implementation methods of the present invention, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention. It should be pointed out that, for those of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present invention, and these all belong to the protection scope of the present invention.

Claims

1. A ship water gauge recognition method based on weakly supervised deep learning, characterized by: The method includes: S1: Acquisition of water level image, collecting ship image datasets from multiple sources, including but not limited to ship water level images under different weather conditions, at different time periods and at different angles; The specific operation of obtaining water gauge images is as follows: a diverse ship image dataset is collected, including images at different angles, lighting conditions and backgrounds, as well as clear and partially blocked or difficult to identify water gauge mark images. The dataset consists of ship water gauge data collected in real business scenarios. After preprocessing, 3,000 images are selected for model training and testing; S2: Image preprocessing, preprocessing the acquired water gauge image, including image scaling, rotation, and lighting adjustment operations, to enhance image quality and prepare for subsequent recognition steps; S3: Data annotation, including the position and reading of the water gauge numbers and the character "M", using a weakly supervised learning strategy and using unlabeled images to assist training through a semi-supervised learning method; The specific operation of data annotation is as follows: using weakly supervised learning method, only a subset of the data set is annotated with detailed water gauge marks and readings, while the rest of the data is not annotated in detail or is only annotated simply. The simple annotation operation is to annotate the image with ships but not to specify the water gauge marks. S4: Build a deep learning model based on the YOLOv8 and SAM deep learning frameworks, and train it for the ship water gauge recognition task. During the model training process, the cross-validation method is used to optimize the model parameters to achieve accurate recognition of the ship water gauge numbers. The deep learning model designs a set of anchor boxes for the shape and size of the water gauge mark. The k-means clustering algorithm is used on the training data set to determine the optimal anchor box size and ratio. The distance metric used in the clustering process is the IoU distance, which is calculated as follows: D IoU (box,centroid)=1-IoU(box,centroid) Among them, D IoU represents the IoU distance, box is a bounding box in the dataset, and centroid is the cluster center; The training of the deep learning model uses weakly supervised learning. The specific training process is as follows: T1: Pseudo label generation: For each sample x in the unlabeled dataset u , use the trained model f to generate pseudo labels Among them, f(x u ) represents the model for unlabeled samples x u The predicted distribution of , the argmax operation selects the category with the highest probability as the pseudo label; T2: Pseudo label training: The generated pseudo labels Compared with the original labeled data y l They are used together for training and updating the model. The loss function s during training consists of two parts. One part is the loss L of labeled data. l , and the other part is the loss of pseudo-label data L u : Among them, x l and l are labeled samples and their labels, L l and L u Respectively represent the loss functions of labeled data and pseudo-labeled data, and λ is a weight parameter that adjusts the importance of both; T3: Confidence threshold: In order to improve the quality of pseudo-labels, usually only unlabeled samples whose model prediction confidence is higher than a certain threshold τ are selected for training: Among them, s represents the maximum prediction probability of the model for the selected category. Only when this probability is greater than the threshold τ, the corresponding unlabeled sample will be used for training; The deep learning model is trained using a weighted loss function, as follows: Classification loss function: Use multivariate cross entropy to determine whether a character belongs to this category. The specific calculation formula is: Where b is the number of categories; y ic is a sign function; if the true category of sample i is equal to c, it takes 1, otherwise it takes 0; p ic is the predicted probability that the observed sample i belongs to category c; Positioning loss: Use the squared error loss function L loc , calculate the difference between the predicted bounding box and the true bounding box; Classification loss: Use the cross entropy loss function L cls , measures the difference between the predicted class and the actual class; Confidence loss: Use confidence loss L conf , a measure of the model’s confidence in the correctness of its predictions; The total loss function is the weighted sum of these three parts: L total =λ loc L loc +λ cls L cls +λ conf L conf Among them, λ loc , cls and λ conf It is a weight parameter used to balance different loss parts; S5: Draft reading recognition: Use the trained deep learning model to process the new ship draft image, automatically recognize the draft reading in the image, and calculate the actual draft value; The calculation method of water gauge reading is: S5-1: By extracting the recognition box with the largest coordinate among all recognition boxes, obtaining its upper left corner and lower right corner coordinates, and thus calculating the center coordinates; secondly, obtaining all characters "M" according to the category index and summing their coordinates, comparing and recording the index with the largest coordinate sum, and according to a similar method, selecting the character with the smaller horizontal coordinate, comparing their coordinate sums, obtaining the maximum value, and obtaining the integer part parameter N; S5-2: In the water gauge character recognition, the upper character closest to the waterline is recorded as F, and the decimal part of the reading can be calculated based on F; S5-3: Eliminate the data of all characters to the left of "M" by clustering method, sort the remaining data in ascending order according to the y-axis coordinates and perform list matching with the template [10, 8, 6, 4, 2, 10, 8, 6, 4, 2, ...]. A successful match indicates that the recognition is correct. At this time, subtract the y2 values ​​of the water ruler characters in pairs, and calculate the average value as M, where M is the pixel length corresponding to 0.2m, where y2 represents the y-axis coordinate of the lower right corner; S5-4: Finally, the water gauge reading R can be calculated: Wherein, H is the pixel length of the line segment obtained by drawing a perpendicular line from the bottom midpoint coordinate of the identification box with the largest coordinate in step S5-1 to the waterline segmentation area; S6: Ship status output, based on the water gauge reading and the known draft of the ship, the current load status of the ship is comprehensively calculated.

2. A ship water gauge recognition system based on weakly supervised deep learning, characterized by: Used to complete the recognition method described in claim 1, the system includes: a water gauge image acquisition module, an image preprocessing module, a water gauge character recognition module, a water gauge waterline segmentation module and a water gauge reading recognition module.

3. A ship water gauge recognition device based on weakly supervised deep learning, characterized in that: The device includes: at least one processor, a display device and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor executes the method according to claim 1.

Images