Ship water gauge card identification method and device

Abstract

The present invention provides a ship draft gauge identification method and device, comprising: obtaining a ship draft gauge image captured by a shore-based camera; gridding the draft gauge image to obtain multiple grid-processed draft gauge images; inputting the multiple grid-processed draft gauge images in parallel into a target detection model to output multiple target detection results; inputting the multiple target detection results into a digital classification model to output multiple classification results; inputting the draft gauge image into a seawater region segmentation model to output a seawater segmentation result for the draft gauge image; and determining the ship draft gauge card reading based on the multiple classification results and the seawater segmentation results. This method utilizes an algorithm for automatically detecting a ship's waterline using image processing technology to automatically detect the ship's draft gauge scale value. It also overcomes a series of issues associated with manual visual inspection, completely records the waterline position throughout the entire observation phase, and enables subsequent data processing. This method reduces system complexity while improving efficiency and accuracy.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a method and device for identifying a ship draft gauge card. Background Art

[0002] With the development of shipping technology, ship transportation has attracted increasing attention. As a scientific weighing method with a certain degree of accuracy, ship draft weighing has been widely used internationally. Its calculation results can be used as the basis for commodity delivery settlement, claims settlement, freight calculation, and customs clearance tax calculation. It is suitable for calculating the weight of bulk solid commodities with low value and difficult to weigh, such as coal, iron ore, cement, and grain. The advantages of draft weighing are that it saves time, effort, and money, and can quickly calculate the weight of the entire ship's cargo. However, the calculation process is relatively complex, and there are many objective factors that affect the weighing results. In particular, the ship's scale observation level is the most important factor affecting the draft gauge's accuracy.

[0003] Currently, waterline measurements rely primarily on trained observers visually observing the ship's draft markings to determine the vessel's actual draft. However, to obtain a more accurate value, the observer must perform multiple checks to obtain an average. This method of visually measuring the waterline is often subjective and has significant limitations, such as inconvenience, significant environmental influences, and potential risks. Summary of the Invention

[0004] In light of this, the present invention aims to provide a method and device for identifying a ship's draft gauge. This method utilizes image processing technology to automatically detect the ship's draft gauge scale value using an algorithm that automatically detects the ship's waterline. This method also overcomes a series of problems caused by manual visual inspection, completely records the waterline position throughout the entire observation phase, and enables subsequent data processing. This method reduces system complexity while improving efficiency and accuracy.

[0005] In a first aspect, an embodiment of the present invention provides a method for identifying a ship's draft gauge card, the method comprising: obtaining a draft gauge image of a ship taken by a shore-based camera; performing grid processing on the draft gauge image to obtain multiple grid-processed draft gauge images; inputting the multiple grid-processed draft gauge images into a target detection model in parallel, and outputting multiple target detection results; wherein the target object detected by each target detection result is a number 0-9, the letter m, or the letter M; inputting the multiple target detection results into a digital classification model, and outputting multiple classification results; inputting the draft gauge image into a seawater area segmentation model, and outputting a seawater segmentation result of the draft gauge image; wherein the pixel value of the seawater area position in the seawater segmentation result is 1; and determining the reading of the ship's draft gauge card based on the multiple classification results and the seawater segmentation results.

[0006] In an optional embodiment of the present application, after the above-mentioned step of outputting multiple target detection results, the method further includes: eliminating non-target objects from the multiple target detection results based on digital fusion non-maximum suppression.

[0007] In an optional embodiment of the present application, the above method also includes: training a digital classification model based on a data augmentation balancing operation.

[0008] In an optional embodiment of the present application, the above-mentioned steps of inputting the water gauge image into the seawater region segmentation model and outputting the seawater segmentation result of the water gauge image include: inputting the water gauge image into the seawater region segmentation model; the seawater region segmentation model sequentially performs grayscale processing and binarization processing on the water gauge image; and the seawater region segmentation model detects the seawater segmentation result of the water gauge image after the binarization processing.

[0009] In an optional embodiment of the present application, the above-mentioned step of determining the reading of the ship's draft gauge card based on multiple classification results and seawater segmentation results includes: determining multiple classification results outside the seawater area based on multiple classification results and seawater segmentation results; dividing the classification results outside the multiple seawater areas into at least one letter classification result in which the target object is the letter m or the letter M and multiple digital classification results in which the target objects are numbers 0-9; based on multiple digital classification results, determining the meter reading at a preset threshold to the left of the letter m or the letter M in the letter classification result; wherein the meter reading includes: numbers 0-9; taking the meter reading with the highest confidence as the root node, taking the numbers in the multiple digital classification results greater than the meter reading with the highest confidence as the left subtree, and taking the numbers in the multiple digital classification results less than the meter reading with the highest confidence as the right subtree; determining the reading of the ship's draft gauge card based on the root node, the left subtree and the right subtree.

[0010] In an optional embodiment of the present application, the above method also includes: forming a ray emitted from a larger value to a smaller value based on the numbers in the smallest 2-digit classification results, and determining the equation of the ray; when the ray touches the seawater area, determining the meter reading of the seawater area based on the equation of the ray.

[0011] In an optional embodiment of the present application, after the above-mentioned step of obtaining the water gauge image of the ship taken by the shore-based camera, the method further includes: if the water gauge card of the ship is tilted, performing image correction on the water gauge image of the ship.

[0012] In an optional embodiment of the present application, the above-mentioned step of performing image correction on the ship's draft gauge image if the ship's draft gauge card is tilted includes: obtaining the standard PTZ value of the shore-based camera position facing the sea surface, and the real-time PTZ value after the shore-based camera position is rotated; performing homography matrix conversion based on the standard PTZ value and the real-time PTZ value, and performing image correction on the ship's draft gauge image based on the result of the homography matrix conversion.

[0013] In an optional embodiment of the present application, the above method further includes: performing data cleaning on the reading data of the water gauge card using a normal distribution method of the data.

[0014] In a second aspect, an embodiment of the present invention further provides a device for identifying a ship draft gauge card, the device comprising: a draft gauge image acquisition module for acquiring a draft gauge image of a ship taken by a shore-based camera; a target detection module for performing grid processing on the draft gauge image to obtain a plurality of grid-processed draft gauge images; the plurality of grid-processed draft gauge images are input in parallel into a target detection model to output a plurality of target detection results; wherein the target objects detected by each target detection result are the numbers 0-9, the letter m or the letter M; a digital classification module for inputting the plurality of target detection results into a digital classification model to output a plurality of classification results; a seawater area segmentation module for inputting the draft gauge image into the seawater area segmentation model to output a seawater segmentation result of the draft gauge image; wherein the pixel value of the seawater area position in the seawater segmentation result is 1; a draft gauge card reading determination module for determining the reading of the ship's draft gauge card based on the plurality of classification results and the seawater segmentation results.

[0015] The embodiments of the present invention bring the following beneficial effects:

[0016] The present invention provides a method and device for identifying a ship's draft gauge. The method comprises obtaining a ship's draft gauge image captured by a shore-based camera; performing grid processing on the draft gauge image to obtain multiple grid-processed draft gauge images; inputting the multiple grid-processed draft gauge images into a target detection model in parallel to output multiple target detection results, wherein each target detection result detects a target object that is a number 0-9, the letter m, or the letter M; inputting the multiple target detection results into a digital classification model to output multiple classification results; inputting the draft gauge image into a seawater region segmentation model to output a seawater segmentation result for the draft gauge image; wherein the pixel value at the seawater region location in the seawater segmentation result is 1; and determining the ship's draft gauge card reading based on the multiple classification results and the seawater segmentation results. This method utilizes an algorithm for automatically detecting a ship's waterline using image processing technology to automatically detect the ship's draft gauge scale value. It also overcomes a series of issues caused by manual visual inspection, completely records the waterline position throughout the entire observation period, and enables subsequent data processing. This method reduces system complexity while improving efficiency and accuracy.

[0017] Other features and advantages of the present disclosure will be set forth in the following description, or some features and advantages may be inferred or unambiguously determined from the description, or may be learned by practicing the above-mentioned technology of the present disclosure.

[0018] In order to make the above-mentioned objectives, features and advantages of the present disclosure more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the specific embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0020] Figure 1 A flow chart of a method for identifying a ship draft gauge card provided by an embodiment of the present invention;

[0021] Figure 2 A schematic diagram of a method for identifying a ship draft gauge card provided by an embodiment of the present invention;

[0022] Figure 3 A schematic diagram of a water gauge image provided by an embodiment of the present invention;

[0023] Figure 4 A schematic diagram of another water gauge image provided by an embodiment of the present invention;

[0024] Figure 5 A binary sea surface recognition processing image provided by an embodiment of the present invention;

[0025] Figure 6 A schematic diagram of image correction of a water gauge image provided by an embodiment of the present invention;

[0026] Figure 7 A schematic diagram of unprocessed data provided by an embodiment of the present invention;

[0027] Figure 8 A schematic diagram of processed data provided by an embodiment of the present invention;

[0028] Figure 9 A schematic structural diagram of a ship water gauge card identification device provided by an embodiment of the present invention;

[0029] Figure 10 A schematic structural diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0030] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of them. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0031] Currently, waterline detection relies primarily on trained observers visually observing the ship's draft markings to determine the vessel's actual draft. However, to obtain a more accurate value, the observer must perform multiple checks to obtain an average. This method of visually measuring the waterline is often subjective and has significant limitations, such as inconvenience, significant environmental influences, and potential risks.

[0032] Based on this, embodiments of the present invention provide a method and device for identifying a ship draft gauge, specifically providing an artificial intelligence-based method for reading and identifying a ship draft gauge. In recent years, the continuous improvement of imaging equipment has enabled highly accurate resolution and high-quality image clarity. Simultaneously, the rapid development of computer architecture and algorithms has significantly improved image processing performance and computational speed, making it possible to determine the waterline. In a computer, an image can be defined as a two-dimensional function f(x, y), where x and y are spatial (plane) coordinates, and the amplitude f at any pair of spatial coordinates (x, y) is called the grayscale at that point. When x, y, and the grayscale value f are finite, discrete values, the image is called a numerical image. Digital image processing involves processing these digital images with the help of a computer. A digital image is composed of a finite number of elements, each with a unique position and value. Converting a secondary image into digital form requires digitizing x, y, and the grayscale value f. Using these numerical values, a corresponding algorithm can be designed to obtain the precise scale values of a ship draft gauge.

[0033] To facilitate understanding of this embodiment, a method for identifying a ship draft gauge card disclosed in an embodiment of the present invention is first introduced in detail.

[0034] Example 1:

[0035] The embodiment of the present invention provides a method for identifying a ship water gauge card. Figure 1 The flowchart of a method for identifying a ship draft gauge card is shown, and the method for identifying a ship draft gauge card includes the following steps:

[0036] Step S102: Acquire a water level image of the ship taken by a shore-based camera.

[0037] See also Figure 2 The schematic diagram of a ship draft gauge identification method is shown. The main processes in this embodiment can be divided into seven, namely, actual usage scenario survey, draft gauge image material annotation based on deep learning, draft gauge card digital (small target) detection module based on deep learning, digital classification module based on deep learning, seawater area segmentation module based on deep learning, and draft gauge card reading calculation and inference module.

[0038] like Figure 2 As shown, the input image in this embodiment is a water level image of a ship taken by a shore-based camera.

[0039] Shore-based cameras are installed at fixed locations near the coastline to monitor coastline and offshore waters for safety, the marine environment, and the entry and exit of ships. Shore-based cameras are primarily used to monitor coastlines and offshore waters to ensure safety. They can also monitor the marine environment, such as water quality and waves, providing data support for environmental protection. Furthermore, shore-based cameras can monitor the entry and exit of ships at ports to ensure normal port operations and safety.

[0040] In this embodiment, scene survey can be actually used. The scenario where intelligent water gauge recognition is needed is the identification of the waterline of ships docked at the dock. The maximum load of ships at the bulk cargo terminal is about 400,000 tons, and the ship's draft scale is usually in the range of about 25M. Considering the empty draft of the ship itself, the algorithm recognition range is limited to 5-30 to improve the efficiency and accuracy of recognition.

[0041] In the recognition scenario, there are situations where the ship scales are rusted, tilted, bent, there are interferences in the picture, the interference of the ship hull reflection under strong light and the sea surface reflection, and the water surface is light-transmitting. It is necessary to collect corresponding materials for these situations, and perform labeling and training.

[0042] This embodiment can perform deep learning to annotate water gauge image materials. The original materials in the field are collected and used. In order to ensure the recognition accuracy under the interference of rust and tilt, it is necessary to ensure that there are two large scale marks in the picture. Figure 3 A schematic diagram of a water gauge image is shown in FIG. Figure 3 The water ruler scale size in the water ruler image is in an appropriate state.

[0043] See also Figure 4 The schematic diagram of another water ruler image is shown in FIG. Figure 4 In the case of obstruction caused by a single piece of rust or water surface fluctuations in the picture, the water gauge can be judged associatively based on the previous scale value, which is used to solve the recognition anomaly caused by the obstruction of the scale in the case of a single piece of rust or water surface fluctuations.

[0044] In step S104, the water gauge image is grid-processed to obtain a plurality of grid-processed water gauge images; the plurality of grid-processed water gauge images are input into the target detection model in parallel to output a plurality of target detection results; wherein the target object detected by each target detection result is a number 0-9, the letter m or the letter M.

[0045] like Figure 2 As shown, this embodiment can perform water gauge card digital (small target) detection based on deep learning.

[0046] The input in this embodiment is a ship's water gauge image captured by a shore-based camera. First, the digital detection image pre-processing module is entered. Because the actual digital size of the business is relatively small compared to the entire image, the commonly used deep learning-based template detection method is ineffective. Small target detection is also a challenging aspect of deep learning-based target detection. To address this issue, this embodiment uses an image segmentation method based on 3×3 grid processing. The grid-processed image is fed to the graphics card and processed in parallel by setting the batch_size of the digital target detection model to ensure processing efficiency.

[0047] Batch_size refers to the number of samples used in each iteration during deep learning model training. It determines the amount of data used by the model when updating weights.

[0048] In some embodiments, non-target objects may be eliminated from multiple target detection results based on digital fusion non-maximum suppression.

[0049] In this embodiment, the parallel processing results can be digitally fused using the non-maximum suppression (NMS) method to remove redundant target detections. Finally, a digital detection of the ship water gauge card for the entire image is obtained.

[0050] Digital fusion non-maximum suppression is a post-processing technique commonly used in object detection tasks. It is primarily used to select the most appropriate bounding box from multiple candidate boxes and suppress those boxes that have high overlap with the optimal candidate box and low classification confidence.

[0051] In this embodiment, the target objects detected by each target detection result are all numbers 0-9, letters m or M, that is, there is only one type of target object currently detected, including all the numbers and letters m / M that appear: 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, m, M.

[0052] Step S106: input the multiple target detection results into a digital classification model and output multiple classification results.

[0053] like Figure 2 As shown, this embodiment can perform digital classification based on deep learning. The digital image is extracted using the bounding_box area of the digital target detection of the ship water gauge card and sent to the digital classification model based on deep learning for classification.

[0054] The bounding_box is a rectangular box used to describe the location and extent of an object in an image. A bounding box typically consists of a set of coordinates that indicate the position and size of the rectangle. Specifically, a bounding box can be defined by the x and y coordinates of the upper left corner and the x and y coordinates of the lower right corner, or by the coordinates of the center of the bounding box, along with its width and height. In object detection tasks, the model achieves object detection and localization by predicting the bounding box of the target object.

[0055] In some embodiments, a digit classification model may also be trained based on a data augmentation balancing operation.

[0056] When training data for classifying ship gauge digits, the numbers 2, 4, 6, 8, m, and M appear frequently. Odd numbers 1, 3, 5, 7, and 9 only appear in meter readings. To address this data imbalance, data augmentation and balancing were performed during the model training phase to optimize classification for each category. Because of data association, the deep learning-based digit classification module runs sequentially after the deep learning-based gauge digit (small object) detection module.

[0057] Step S108 , inputting the water gauge image into the seawater region segmentation model, and outputting the seawater segmentation result of the water gauge image; wherein the pixel value of the seawater region position in the seawater segmentation result is 1.

[0058] like Figure 2 As shown, this embodiment can also perform seawater region segmentation based on deep learning. The deep learning-based seawater region segmentation module and the deep learning-based water gauge card digital (small target) detection module are parallel modules. For the same input image, seawater region segmentation is performed simultaneously with digital detection and classification. The pixel values of the segmented seawater region are set to 1.

[0059] In some examples, the water gauge image can be input into the seawater region segmentation model; the seawater region segmentation model sequentially performs grayscale processing and binarization processing on the water gauge image; and the seawater region segmentation model detects the seawater segmentation result of the binarized water gauge image.

[0060] The specific processing method of this embodiment is to grayscale and binarize the image. Compared with color images, edge detection of grayscale images is more convenient and faster, and the amount of calculation is small, so the acquired color image is first converted into a grayscale image. The binarization of the image is to set the grayscale value of the pixel on the image to 0 to 255. The effect can be seen in Figure 5 The binary sea surface recognition processing diagram shown in FIG. represents the grayscale value of each pixel obtained after sampling using a matrix, that is, the quantization of the grayscale image.

[0061] like Figure 5As shown, the image in the middle section represents the foreground, values greater than the middle section represent noise, and values less than the middle section represent the background. A more complex situation occurs when values less than the threshold represent the foreground, while values greater than the threshold represent the background. For example, in the morning when sunlight is weak, the water gauge area appears dark, while the surrounding area appears bright. By projecting the grayscale values onto the curve, we can obtain the grayscale distribution characteristics.

[0062] Step S110 , determining the reading of the ship's water gauge card based on the multiple classification results and the seawater segmentation results.

[0063] After determining the classification results and the seawater segmentation results, this embodiment can determine the reading of the ship's draft gauge based on the multiple classification results and the seawater segmentation results, thereby performing calculation of the reading of the ship's draft gauge.

[0064] This example uses appropriate image segmentation and coordinate encoding techniques to magnify the water gauge numerals on a new water gauge image. A deep learning-based object detection model then extracts numerals from each of these image segments. The extracted numerals are then coordinate-reversed and transferred back to the original image. For overlapping numerals extracted from the segmented images, weighted non-maximum suppression is employed to improve the accuracy of numeral location. Once a numeral is located, the image data within the extracted location frame is fed into a deep learning classification network for inference. Based on the reading recognition function, the numeral is appropriately classified.

[0065] In actual ship draft gauges, the number of odd numbers is significantly less than the number of even numbers. To address this, this embodiment introduces a small sample training technique to maintain a certain accuracy rate in the final classification results of odd and even numbers.

[0066] Ships often navigate in seawater, and draft gauges are subject to various interferences, such as varying degrees of rust and obstruction. To address this, this embodiment has developed a powerful post-processing reading inference algorithm. When individual draft gauge numbers are unclear or missing, appropriate reading inference can be performed based on some of the recognized numbers. The final reading is the intersection of the draft gauge and the sea surface. For the sea surface contour, a deep learning segmentation algorithm is used. At the same time, when the seawater is relatively transparent or the draft gauge reading is reflected in the seawater, a contour definition method is used to exclude readings within the seawater contour, thereby obtaining a more accurate reading.

[0067] An embodiment of the present invention provides a method for identifying a ship's draft gauge. The method comprises obtaining a ship's draft gauge image captured by a shore-based camera; performing grid processing on the draft gauge image to obtain multiple grid-processed draft gauge images; inputting the multiple grid-processed draft gauge images into a target detection model in parallel to output multiple target detection results, wherein each target detection result detects a target object that is a number 0-9, the letter m, or the letter M; inputting the multiple target detection results into a digital classification model to output multiple classification results; inputting the draft gauge image into a seawater region segmentation model to output a seawater segmentation result for the draft gauge image; wherein the pixel value at the seawater region position in the seawater segmentation result is 1; and determining the ship's draft gauge card reading based on the multiple classification results and the seawater segmentation results. This method utilizes an algorithm for automatically detecting a ship's waterline using image processing technology to automatically detect the ship's draft gauge scale value. It also overcomes a series of problems caused by manual visual inspection, completely records the waterline position throughout the entire observation period, and enables subsequent data processing. This method reduces system complexity while improving efficiency and accuracy.

[0068] Example 2:

[0069] This embodiment provides another method for identifying a ship's draft gauge, which is implemented on the basis of the above embodiment. This embodiment focuses on describing the specific steps of determining the reading of a ship's draft gauge based on multiple classification results and seawater segmentation results.

[0070] In some embodiments, classification results outside multiple seawater areas can be determined based on multiple classification results and seawater segmentation results; the classification results outside multiple seawater areas are divided into at least one letter classification result in which the target object is the letter m or the letter M and multiple digital classification results in which the target objects are numbers 0-9; based on multiple digital classification results, the meter reading is determined at a preset threshold to the left of the letter m or the letter M in the letter classification result; wherein the meter reading includes: numbers 0-9; the meter reading with the highest confidence is used as the root node of the tree, the numbers in the multiple digital classification results that are greater than the meter reading with the highest confidence are used as the left subtree, and the numbers in the multiple digital classification results that are less than the meter reading with the highest confidence are used as the right subtree; based on the root node, the left subtree and the right subtree, the reading of the ship's water gauge card is determined.

[0071] like Figure 2 As shown, this embodiment can also calculate and infer draft gauge readings. After successful inference, the ship draft gauge reading can be calculated. First, the algorithm finds the letter classification results for the m / M classification result among all detected numbers. It then searches for the meter reading within a certain threshold space to the left of the m / M. Since the draft gauge is arranged vertically, after successfully finding the meter reading, the remaining number classification results can be divided into different intervals based on the m / M image vertical coordinate value.

[0072] A tree-like structure is designed to store water gauge readings. The highest-confidence number (i.e., the meter reading with the highest confidence) is selected from all m / M values as the root node. Numbers greater than the root node (i.e., numbers classified as greater than the highest-confidence meter reading) are stored in the left subtree, while numbers less than the root node (i.e., numbers classified as less than the highest-confidence meter reading) are stored in the right subtree. After looping through all the numbers, the entire water gauge reading is generated.

[0073] In some embodiments, a ray emitted from a larger value to a smaller value can be formed based on the numbers in the smallest 2-digit classification results to determine the equation of the ray; when the ray touches the seawater area, the meter reading of the seawater area is determined based on the equation of the ray.

[0074] In actual business use, some numbers on ship draft gauge cards are often blurred or rusted. Therefore, this embodiment also designs a reading inference algorithm. When some numbers are not fully recognized, reading jump inference can be performed based on the Euclidean distance relationship between the numbers. The introduction of the reading inference algorithm can improve the final reading accuracy.

[0075] When the ship's water gauge readings fill the tree structure, the algorithm takes the two smallest readings and forms a ray from the larger reading to the smaller reading. When the ray hits the seawater area, the reading algorithm module determines that the reading can be output according to the straight line equation.

[0076] In some embodiments, if the draft gauge of a ship is tilted, image correction may be performed on the draft gauge image of the ship.

[0077] Due to the limited field of view of shore-based cameras, when a ship is docked too far forward or too far back, the water gauge in the captured image is often severely tilted, affecting the water gauge digital detection based on deep learning. For camera positions that produce severely tilted images, this embodiment can use a method of first correcting the image and then processing the image.

[0078] In some embodiments, the standard PTZ value of the shore-based camera facing the sea surface and the real-time PTZ value of the shore-based camera after rotation can be obtained; homography matrix conversion is performed based on the standard PTZ value and the real-time PTZ value, and image correction is performed on the water level image of the ship based on the result of the homography matrix conversion.

[0079] PTZ values refer to the parameters of the camera's pan / tilt control function, including Pan (horizontal rotation), Tilt (vertical pitch), and Zoom. Image correction first determines the PTZ value of the camera facing the sea surface. After the camera rotates, a homography matrix is calculated based on the newly acquired real-time PTZ and the PTZ facing the sea surface. After correction, the position of the ship's water gauge stuck in the image, which was originally tilted, becomes vertical. For more information, see [Note: The following text appears to be corrupted and cannot be accurately translated]. Figure 6 A schematic diagram of image correction for a water gauge image is shown.

[0080] To correct shore camera footage, you need to determine whether the camera supports PTZ data acquisition. If it does, adjust the horizontal and vertical matrix conversions to correct tilted or curved scales to near-vertical angles. If the camera does not support PTZ acquisition, you can set the initial data to 0,0 before performing the matrix conversion.

[0081] In addition, during the identification process, it is necessary to ensure that the corrected water level scale retains two large mark values on the water surface to meet the identification requirements under various abnormal situations.

[0082] Since image correction will produce more dark areas, the digital detection model and seawater area segmentation model originally used under normal conditions will not be effective. Therefore, for severely tilted images, it is necessary to additionally collect ship water gauge card data from different camera positions and correct it before training.

[0083] Furthermore, this embodiment can also perform data cleaning for outliers during the recognition process. Environmental factors such as wind, waves, lighting, obstructions, and rain can dynamically change during the recognition process, leading to the identification of abnormal image data. Among hundreds of data items, a certain amount of outliers can affect the accuracy of the final data, necessitating data cleaning. Typically, outliers can be identified using graphical methods (such as boxplots and normal distribution plots) and modeling techniques (such as linear regression, clustering algorithms, and K-nearest neighbor algorithms).

[0084] In some embodiments, the reading data of the water gauge card may be cleaned using a normal distribution of data.

[0085] During the water gauge identification process, the readings are consistent with the frequency of wind and wave surges on the sea surface. Correctly identified data should present a wave-like curve graph, with the highest and lowest values falling at the peaks and troughs of a certain period of data. This indicates that the data is regular. Data cleaning can be performed using a normal distribution approach to eliminate obvious identification errors and remove invalid data from periods of excessive wind and waves, ensuring that the data curve approaches a static horizontal line.

[0086] When using normal data distribution to clean outliers, the effect can be seen in Figure 7 A schematic diagram of unprocessed data and Figure 8 The diagram shows a schematic diagram of processed data. It is obvious that the anomaly identification data has been cleaned, and the retained data presents a wave-like shape with ups and downs, indicating that the cleaned data is already real data. In the case of long-term continuous data, a single point error cannot have a significant impact on a large data set.

[0087] Example 3:

[0088] Corresponding to the above method embodiment, the embodiment of the present invention provides a ship water gauge card identification device. Figure 9 A structural diagram of a ship water gauge card identification device, the ship water gauge card identification device includes:

[0089] The water gauge image acquisition module 91 is used to acquire the water gauge image of the ship taken by the shore-based camera;

[0090] The target detection module 92 is configured to perform grid processing on the water gauge image to obtain a plurality of grid-processed water gauge images; the plurality of grid-processed water gauge images are input into the target detection model in parallel, and a plurality of target detection results are output; wherein the target object detected in each target detection result is a number 0-9, the letter m, or the letter M;

[0091] A digital classification module 93 is used to input multiple target detection results into a digital classification model and output multiple classification results;

[0092] The seawater region segmentation module 94 is used to input the water gauge image into the seawater region segmentation model and output the seawater segmentation result of the water gauge image; wherein the pixel value of the seawater region position in the seawater segmentation result is 1;

[0093] The water gauge reading determination module 95 is used to determine the reading of the water gauge of the ship based on the multiple classification results and the seawater segmentation results.

[0094] An embodiment of the present invention provides a ship draft gauge identification device. The device captures a ship draft gauge image captured by a shore-based camera; performs grid processing on the draft gauge image to obtain multiple grid-processed draft gauge images; inputs the multiple grid-processed draft gauge images into a target detection model in parallel, outputting multiple target detection results; wherein each target detection result detects a target object that is a number 0-9, the letter m, or the letter M; inputs the multiple target detection results into a digital classification model, outputting multiple classification results; inputs the draft gauge image into a seawater region segmentation model, outputting a seawater segmentation result for the draft gauge image; wherein the pixel value at the seawater region location in the seawater segmentation result is 1; and determines the ship draft gauge reading based on the multiple classification results and the seawater segmentation results. This method utilizes an algorithm for automatically detecting a ship's waterline using image processing technology to automatically detect the ship's draft gauge scale value. It also overcomes a series of issues caused by manual visual inspection, completely records the waterline position throughout the entire observation period, and enables subsequent data processing. This method reduces system complexity while improving efficiency and accuracy.

[0095] The above-mentioned device also includes: a non-target object elimination module, which is used to eliminate non-target objects from multiple target detection results based on digital fusion non-maximum suppression.

[0096] The above-mentioned device also includes: a digital classification model training module, which is used to train the digital classification model based on the data augmentation balancing operation.

[0097] The above-mentioned seawater area segmentation module is used to input the water gauge image into the seawater area segmentation model; the seawater area segmentation model performs grayscale processing and binarization processing on the water gauge image in sequence; the seawater area segmentation model detects the seawater segmentation result of the water gauge image after binarization processing.

[0098] The above-mentioned water gauge card reading determination module is used to determine the classification results outside the multiple seawater areas based on multiple classification results and seawater segmentation results; divide the classification results outside the multiple seawater areas into at least one letter classification result in which the target object is the letter m or the letter M and multiple digital classification results in which the target objects are the numbers 0-9; based on the multiple digital classification results, determine the meter reading at a preset threshold to the left of the letter m or the letter M in the letter classification result; wherein the meter reading includes: numbers 0-9; take the meter reading with the highest confidence as the root node of the tree, take the numbers in the multiple digital classification results that are greater than the meter reading with the highest confidence as the left subtree, and take the numbers in the multiple digital classification results that are less than the meter reading with the highest confidence as the right subtree; based on the root node, the left subtree and the right subtree, determine the reading of the water gauge card of the ship.

[0099] The above-mentioned water gauge card reading determination module is also used to form a ray emitted from the larger value to the smaller value based on the numbers in the smallest two-digit classification results, and determine the equation of the ray; when the ray touches the sea water area, the meter reading of the sea water area is determined based on the equation of the ray.

[0100] The above-mentioned device includes: an image correction module, which is used to perform image correction on the water gauge image of the ship if the water gauge card of the ship is tilted.

[0101] The image correction module is used to obtain the standard PTZ value of the shore-based camera when it is facing the sea surface, as well as the real-time PTZ value of the shore-based camera after rotation; perform homography matrix conversion based on the standard PTZ value and the real-time PTZ value, and perform image correction on the water level image of the ship based on the result of the homography matrix conversion.

[0102] The device comprises: a data cleaning module for cleaning the reading data of the water gauge card using a normal distribution method.

[0103] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the ship draft card identification system described above can refer to the corresponding process in the aforementioned embodiment of the ship draft card identification method, and will not be repeated here.

[0104] Example 4:

[0105] The embodiment of the present invention further provides an electronic device for executing the above-mentioned ship water gauge card identification method; Figure 10 The structure diagram of an electronic device shown in the figure includes a memory 100 and a processor 101, wherein the memory 100 is used to store one or more computer instructions, and the one or more computer instructions are executed by the processor 101 to implement the above-mentioned ship water gauge card identification method.

[0106] Further, Figure 10 The electronic device shown further includes a bus 102 and a communication interface 103 , and the processor 101 , the communication interface 103 and the memory 100 are connected via the bus 102 .

[0107] The memory 100 may include high-speed random access memory (RAM) and may also include non-volatile memory, such as at least one disk storage. The communication connection between the system network element and at least one other network element is achieved through at least one communication interface 103 (which may be wired or wireless), and the Internet, wide area network, local area network, metropolitan area network, etc. may be used. The bus 102 may be an ISA bus, a PCI bus, or an EISA bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 10 Only one bidirectional arrow is used in the diagram, but this does not mean that there is only one bus or one type of bus.

[0108] The processor 101 may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed by hardware integrated logic circuits in the processor 101 or by software instructions. The above processor 101 may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the various methods, steps, and logic block diagrams disclosed in the embodiments of the present invention. The general-purpose processor may be a microprocessor or any conventional processor. The steps of the method disclosed in conjunction with the embodiments of the present invention can be directly implemented as being executed by a hardware decoding processor, or can be executed by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium well-known in the art, such as a random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, or register. The storage medium is located in the memory 100, and the processor 101 reads the information in the memory 100 and, in conjunction with its hardware, completes the steps of the method of the aforementioned embodiment.

[0109] An embodiment of the present invention also provides a computer-readable storage medium, which stores computer-executable instructions. When the computer-executable instructions are called and executed by a processor, the computer-executable instructions prompt the processor to implement the above-mentioned ship water gauge card identification method. The specific implementation can be found in the method embodiment and will not be repeated here.

[0110] The computer program product of the ship draft card identification method and device provided in the embodiments of the present invention includes a computer-readable storage medium storing program code. The instructions included in the program code can be used to execute the method in the previous method embodiment. The specific implementation can be found in the method embodiment and will not be repeated here.

[0111] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working process of the system and / or device described above can refer to the corresponding process in the aforementioned method embodiment and will not be repeated here.

[0112] In addition, in the description of the embodiments of the present invention, unless otherwise expressly specified or limited, the terms "mounted," "connected," and "connected" should be understood in a broad sense. For example, they may refer to fixed connections, detachable connections, or integral connections; they may refer to mechanical connections or electrical connections; they may refer to direct connections or indirect connections through an intermediate medium; and they may refer to internal communication between two components. Those skilled in the art will understand the specific meanings of the above terms in the present invention based on the specific circumstances.

[0113] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the portion that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the various embodiments of the method of the present invention. The aforementioned storage medium includes various media that can store program code, such as USB flash drives, mobile hard drives, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical disks.

[0114] In the description of the present invention, it should be noted that the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," and "outer," etc., indicating orientations or positional relationships, are based on the orientations or positional relationships shown in the accompanying drawings and are intended solely to facilitate and simplify the description of the present invention. They are not intended to indicate or imply that the devices or components referred to must have, be constructed, or operate in a specific orientation, and therefore should not be construed as limitations on the present invention. Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0115] Finally, it should be noted that the above-described embodiments are only specific implementation methods of the present invention, which are used to illustrate the technical solutions of the present invention, rather than to limit them. The scope of protection of the present invention is not limited thereto. Although the present invention has been described in detail with reference to the above-described embodiments, those skilled in the art should understand that any person skilled in the art can modify or easily conceive of changes to the technical solutions described in the above-described embodiments within the technical scope disclosed by the present invention, or replace some of the technical features therein with equivalents. Such modifications, changes, or replacements do not deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be based on the scope of protection of the claims.

Claims

1. A method for identifying a ship water gauge card, characterized in that: The method comprises: Obtain the water level image of the ship taken by the shore-based camera; Performing grid processing on the water gauge image to obtain a plurality of grid-processed water gauge images; inputting the plurality of grid-processed water gauge images into a target detection model in parallel to output a plurality of target detection results; wherein the target object detected in each target detection result is a number 0-9, a letter m, or a letter M; Inputting the plurality of target detection results into a digital classification model and outputting a plurality of classification results; Inputting the water gauge image into a seawater region segmentation model and outputting a seawater segmentation result of the water gauge image; wherein the pixel value of the seawater region position in the seawater segmentation result is 1; Determining a reading of the water gauge of the ship based on the plurality of classification results and the seawater segmentation results; The step of determining the reading of the draft gauge of the ship based on the multiple classification results and the seawater segmentation results includes: determining the classification results outside the multiple seawater areas based on the multiple classification results and the seawater segmentation results; dividing the multiple classification results outside the seawater areas into at least one letter classification result in which the target object is the letter m or the letter M and multiple digital classification results in which the target objects are numbers 0-9; based on the multiple digital classification results, determining the meter reading at a preset threshold to the left of the letter m or the letter M in the letter classification result; wherein the meter reading includes: numbers 0-9; taking the meter reading with the highest confidence as the root node, taking the numbers in the multiple digital classification results that are greater than the meter reading with the highest confidence as the left subtree, and taking the numbers in the multiple digital classification results that are less than the meter reading with the highest confidence as the right subtree; based on the root node, the left subtree and the right subtree determine the reading of the draft gauge of the ship; The method further includes: forming a ray emitted from a larger value to a smaller value based on the numbers in the smallest 2-digit classification results, and determining the equation of the ray; when the ray touches the seawater area, determining the meter reading of the seawater area based on the equation of the ray.

2. The method according to claim 1, characterized in that After the step of outputting multiple target detection results, the method further includes: Non-target objects are eliminated from the plurality of target detection results based on a digital fusion non-maximum suppression method.

3. The method according to claim 1, characterized in that The method further comprises: The digit classification model is trained based on a data augmentation balancing operation.

4. The method according to claim 1, wherein The step of inputting the water gauge image into a seawater region segmentation model and outputting a seawater segmentation result of the water gauge image comprises: Inputting the water gauge image into a seawater region segmentation model; The seawater area segmentation model sequentially performs grayscale processing and binarization processing on the water gauge image; The seawater region segmentation model detects the seawater segmentation result of the water gauge image after binarization processing.

5. The method according to any one of claims 1 to 4, characterized in that After the step of acquiring the water gauge image of the ship taken by the shore-based camera, the method further includes: If the draft gauge of the ship is tilted, image correction is performed on the draft gauge image of the ship.

6. The method according to claim 5, characterized in that If the draft gauge of the ship is tilted, the step of performing image correction on the draft gauge image of the ship comprises: Obtain the standard PTZ value of the shore-based camera facing the sea surface, as well as the real-time PTZ value of the shore-based camera after rotation; A homography matrix conversion is performed based on the standard PTZ value and the real-time PTZ value, and image correction is performed on the water level image of the ship based on the result of the homography matrix conversion.

7. The method according to any one of claims 1 to 4, characterized in that The method further comprises: The reading data of the water gauge card is cleaned by adopting the normal distribution method of the data.

8. A ship water gauge card identification device, characterized in that: The device comprises: A water gauge image acquisition module is used to acquire the water gauge image of the ship taken by a shore-based camera; a target detection module configured to perform grid processing on the water gauge image to obtain a plurality of grid-processed water gauge images; input the plurality of grid-processed water gauge images into a target detection model in parallel, and output a plurality of target detection results; wherein the target object detected in each target detection result is a number 0-9, the letter m, or the letter M; A digital classification module, configured to input the plurality of target detection results into a digital classification model and output a plurality of classification results; a seawater region segmentation module, configured to input the water gauge image into a seawater region segmentation model and output a seawater segmentation result of the water gauge image; wherein the pixel value of the seawater region position in the seawater segmentation result is 1; a draft gauge reading determination module, configured to determine a draft gauge reading of the ship based on the plurality of classification results and the seawater segmentation results; The draft gauge reading determination module is used to determine the classification results outside the multiple seawater areas based on the multiple classification results and the seawater segmentation results; divide the multiple classification results outside the multiple seawater areas into at least one letter classification result in which the target object is the letter m or the letter M and multiple digital classification results in which the target objects are numbers 0-9; based on the multiple digital classification results, determine the meter reading at a preset threshold to the left of the letter m or the letter M in the letter classification result; wherein the meter reading includes: numbers 0-9; use the meter reading with the highest confidence as a tree root node, use the numbers in the multiple digital classification results that are greater than the meter reading with the highest confidence as a left subtree, and use the numbers in the multiple digital classification results that are less than the meter reading with the highest confidence as a right subtree; based on the tree root node, the left subtree and the right subtree determine the reading of the draft gauge of the ship; The water gauge card reading determination module is also used to form a ray emitted from a larger value to a smaller value based on the numbers in the smallest two-digit classification results, and determine the equation of the ray; when the ray touches the sea water area, the meter reading of the sea water area is determined based on the equation of the ray.

Images