Method and device for identifying water gauge card of steamship

Abstract

The invention provides a steamship water gauge card identification method and device. The method comprises the following steps: acquiring a water gauge image of a steamship shot by a shore-based camera; performing grid processing on the water gauge image to obtain a plurality of grid-processed water gauge images; inputting the plurality of grid-processed water gauge images into a target detection model in parallel, and outputting a plurality of target detection results; inputting the plurality of target detection results into a digital classification model, and outputting a plurality of classification results; inputting the water gauge image into a seawater region segmentation model, and outputting a seawater segmentation result of the water gauge image; and determining the reading of the water gauge card of the steamship based on the plurality of classification results and the seawater segmentation result. According to the method, the ship waterline scale value can be automatically detected by utilizing an algorithm for automatically detecting the ship waterline through an image processing technology, a series of problems caused by manual visual inspection can be solved, the waterline position of the whole observation stage is completely recorded, and subsequent data processing becomes possible. The system complexity is reduced, and the efficiency and the accuracy are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a method and a device for identifying a ship water gauge card. Background Art

[0002] With the development of shipping technology, ship transportation has attracted more and more attention. As a scientific weighing method, the draft gauge weighing of ships has a certain degree of accuracy and has been widely used internationally. Its calculation results can be used as the basis for commodity delivery settlement, claims, freight calculation and customs clearance tax calculation. It is suitable for bulk solid commodities with low value and difficult to weigh, such as coal, iron ore, cement, grain and other commodities. The advantages of draft gauge weighing are saving time, labor and cost, and it can quickly calculate the weight of the entire ship's cargo, but the calculation process is relatively complicated, and there are many objective factors that affect the weighing results, especially the ship's scale observation level is the most important factor affecting the accuracy of the draft gauge.

[0003] The detection of the ship's waterline currently relies mainly on long-term trained observers to visually observe the ship's water gauge marks to obtain the actual draft of the ship. However, in order to obtain a more accurate value, the observer needs to check it multiple times to obtain the average value. This method of visually observing the value of the ship's waterline is often subjective and has great limitations, such as: inconvenient observation, greatly affected by the environment, and certain dangers in observation. Summary of the invention

[0004] In view of this, the purpose of the present invention is to provide a ship water gauge card recognition method and device, which can automatically detect the ship water gauge scale value by using the algorithm of automatically detecting the ship waterline by image processing technology, and can also overcome a series of problems caused by manual visual inspection, completely record the waterline position of the entire observation stage, and make subsequent data processing possible, which reduces the system complexity and improves efficiency and accuracy.

[0005] In a first aspect, an embodiment of the present invention provides a method for identifying a ship's draft gauge card, the method comprising: obtaining a draft gauge image of a ship taken by a shore-based camera; grid processing the draft gauge image to obtain a plurality of grid-processed draft gauge images; inputting the plurality of grid-processed draft gauge images into a target detection model in parallel, and outputting a plurality of target detection results; wherein the target object detected by each target detection result is a number 0-9, a letter m, or a letter M; inputting the plurality of target detection results into a digital classification model, and outputting a plurality of classification results; inputting the draft gauge image into a seawater area segmentation model, and outputting a seawater segmentation result of the draft gauge image; wherein the pixel value of the seawater area position in the seawater segmentation result is 1; and determining the reading of the ship's draft gauge card based on the plurality of classification results and the seawater segmentation results.

[0006] In an optional embodiment of the present application, after the above step of outputting multiple target detection results, the method further includes: eliminating non-target objects from the multiple target detection results based on digital fusion non-maximum suppression.

[0007] In an optional embodiment of the present application, the above method also includes: training a digital classification model based on a data augmentation balancing operation.

[0008] In an optional embodiment of the present application, the above-mentioned step of inputting the water gauge image into the seawater region segmentation model and outputting the seawater segmentation result of the water gauge image includes: inputting the water gauge image into the seawater region segmentation model; the seawater region segmentation model sequentially performs grayscale processing and binarization processing on the water gauge image; the seawater region segmentation model detects the seawater segmentation result of the water gauge image after the binarization processing.

[0009] In an optional embodiment of the present application, the above-mentioned step of determining the reading of the ship's draft gauge card based on multiple classification results and seawater segmentation results includes: determining multiple classification results outside the seawater area based on multiple classification results and seawater segmentation results; dividing the classification results outside the multiple seawater areas into at least one letter classification result in which the target object is the letter m or the letter M and multiple digital classification results in which the target objects are numbers 0-9; based on the multiple digital classification results, determining the meter reading at a preset threshold to the left of the letter m or the letter M in the letter classification result; wherein the meter reading includes: numbers 0-9; taking the meter reading with the highest confidence as the root node, taking the numbers in the multiple digital classification results that are greater than the meter reading with the highest confidence as the left subtree, and taking the numbers in the multiple digital classification results that are less than the meter reading with the highest confidence as the right subtree; determining the reading of the ship's draft gauge card based on the root node, the left subtree and the right subtree.

[0010] In an optional embodiment of the present application, the above method also includes: forming a ray emitted from a larger value to a smaller value based on the numbers in the smallest 2-digit classification results, and determining the equation of the ray; when the ray touches the seawater area, determining the rice reading of the seawater area based on the equation of the ray.

[0011] In an optional embodiment of the present application, after the above step of acquiring the water gauge image of the ship taken by the shore-based camera, the method further includes: if the water gauge card of the ship is tilted, performing image correction on the water gauge image of the ship.

[0012] In an optional embodiment of the present application, if the ship's water gauge card is tilted, the step of performing image correction on the ship's water gauge image includes: obtaining a standard PTZ value when the shore-based camera is facing the sea surface, and a real-time PTZ value after the shore-based camera is rotated; performing homography matrix conversion based on the standard PTZ value and the real-time PTZ value, and performing image correction on the ship's water gauge image based on the result of the homography matrix conversion.

[0013] In an optional embodiment of the present application, the above method also includes: using the normal distribution of data to clean the reading data of the water gauge card.

[0014] In a second aspect, an embodiment of the present invention further provides a ship draft gauge card identification device, the device comprising: a draft gauge image acquisition module, used to acquire a ship draft gauge image taken by a shore-based camera; a target detection module, used to perform grid processing on the draft gauge image to obtain a plurality of grid-processed draft gauge images; the plurality of grid-processed draft gauge images are input in parallel into a target detection model, and a plurality of target detection results are output; wherein the target object detected by each target detection result is a number 0-9, a letter m or a letter M; a digital classification module, used to input a plurality of target detection results into a digital classification model, and output a plurality of classification results; a seawater area segmentation module, used to input a draft gauge image into a seawater area segmentation model, and output a seawater segmentation result of the draft gauge image; wherein the pixel value of the seawater area position in the seawater segmentation result is 1; a draft gauge card reading determination module, used to determine the reading of the ship's draft gauge card based on a plurality of classification results and the seawater segmentation results.

[0015] The embodiments of the present invention bring the following beneficial effects: The embodiment of the present invention provides a method and device for identifying a ship's draft gauge card, which obtains a ship's draft gauge image taken by a shore-based camera; performs grid processing on the draft gauge image to obtain multiple grid-processed draft gauge images; inputs multiple grid-processed draft gauge images in parallel into a target detection model, and outputs multiple target detection results; wherein the target object detected by each target detection result is a number 0-9, a letter m, or a letter M; inputs multiple target detection results into a digital classification model, and outputs multiple classification results; inputs the draft gauge image into a seawater region segmentation model, and outputs a seawater segmentation result of the draft gauge image; wherein the pixel value of the seawater region position in the seawater segmentation result is 1; and determines the reading of the ship's draft gauge card based on multiple classification results and seawater segmentation results. In this method, an algorithm for automatically detecting the ship's waterline using image processing technology can be used to automatically detect the ship's draft gauge scale value, and a series of problems caused by manual visual inspection can be overcome, and the draft line position of the entire observation phase can be fully recorded, making subsequent data processing possible. This reduces system complexity and improves efficiency and accuracy.

[0016] Other features and advantages of the present disclosure will be set forth in the following description, or some features and advantages may be inferred or unambiguously determined from the description, or may be learned by implementing the above-mentioned technology of the present disclosure.

[0017] In order to make the above-mentioned objectives, features and advantages of the present disclosure more obvious and easy to understand, preferred embodiments are specifically cited below and described in detail with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0019] Figure 1 A flow chart of a method for identifying a ship water gauge card provided by an embodiment of the present invention; Figure 2 A schematic diagram of a method for identifying a ship water gauge card provided by an embodiment of the present invention; Figure 3 A schematic diagram of a water gauge image provided by an embodiment of the present invention; Figure 4 A schematic diagram of another water gauge image provided by an embodiment of the present invention; Figure 5 A binary sea surface recognition processing diagram provided by an embodiment of the present invention; Figure 6 A schematic diagram of image correction of a water gauge image provided by an embodiment of the present invention; Figure 7 A schematic diagram of unprocessed data provided by an embodiment of the present invention; Figure 8 A schematic diagram of processed data provided by an embodiment of the present invention; Fig. 9 A schematic diagram of the structure of a ship water gauge card identification device provided by an embodiment of the present invention; Fig.10 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0020] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0021] At present, the detection of the ship's waterline mainly relies on long-term trained observers to visually observe the ship's water gauge marks to obtain the actual draft of the ship. However, in order to obtain a more accurate value, the observer needs to check it multiple times to obtain the average value. This method of visually observing the value of the ship's waterline is often subjective and has great limitations, such as: inconvenient observation, greatly affected by the environment, and certain dangers in observation.

[0022] Based on this, a method and device for identifying a ship water gauge card provided by an embodiment of the present invention specifically provides a method for identifying the reading of a ship water gauge card based on artificial intelligence. In recent years, the performance of shooting equipment has been continuously improved, the resolution of the equipment can reach a very precise level, and the picture quality and clarity are very high. At the same time, the rapid development of computer architecture and algorithms has greatly improved the performance and operation speed of image processing, which has made it possible to obtain the waterline. An image can be defined as a two-dimensional function f(x, y) in a computer, where x and y are spatial (plane) coordinates, and the amplitude f at any pair of spatial coordinates (x, y) is called the grayscale at that point. When x, y and the grayscale value f are finite discrete values, the image can be called a numerical image. Digital image processing is to process the above digital images with the help of a computer. A digital image is composed of a finite number of elements, each of which has a special position and value. To convert the secondary image into a digital form, it is required to digitize x, y and the grayscale value f. On this basis, using the above values, designing a corresponding algorithm can obtain the precise scale value of the ship water gauge.

[0023] To facilitate understanding of this embodiment, a ship water gauge card identification method disclosed in an embodiment of the present invention is first introduced in detail.

[0024] Embodiment 1: The embodiment of the present invention provides a method for identifying a ship water gauge card. Figure 1 The flowchart of a method for identifying a ship water gauge card is shown, and the method for identifying a ship water gauge card comprises the following steps: Step S102, obtaining a water level image of the ship taken by a shore-based camera.

[0025] See also Figure 2 A schematic diagram of a ship water gauge card recognition method is shown. The main processes in this embodiment can be divided into seven, namely, actual usage scenario survey, water gauge picture material annotation based on deep learning, water gauge card digital (small target) detection module based on deep learning, digital classification module based on deep learning, seawater area segmentation module based on deep learning, and water gauge card reading calculation and inference module.

[0026] like Figure 2As shown, the input image in this embodiment is a water level image of a ship taken by a shore-based camera.

[0027] Shore-based cameras are devices installed at fixed locations near the coastline to monitor the safety of the coastline and offshore waters, the marine environment, and the entry and exit of ships. Shore-based cameras are mainly used to monitor the coastline and offshore waters to ensure the safety of the area. They can also monitor the marine environment, such as water quality and waves, to provide data support for environmental protection. In addition, shore-based cameras can also monitor the entry and exit of ships to ensure the normal operation and safety of the port.

[0028] In this embodiment, scene survey can be actually used. The scene that requires the use of intelligent water gauge recognition is the recognition of the waterline of the ship docked at the dock. The maximum load of the ship at the bulk cargo terminal is about 400,000 tons, and the ship's draft scale is usually in the range of about 25M. Considering the empty draft of the ship itself, the algorithm recognition range is limited to the range of 5-30 to improve the efficiency and accuracy of recognition.

[0029] In the recognition scenario, there are situations where the ship scales are rusted, the angles are tilted, the scales are bent, there are interferences in the picture, the hull is reflected under strong light and the sea surface is reflected, and there is light transmittance on the water surface. It is necessary to collect corresponding materials for these situations, and perform annotation and training.

[0030] This embodiment can perform deep learning to annotate water gauge image materials. The original materials in the field are collected and used. In order to ensure the recognition accuracy due to rust and tilt interference, it is necessary to ensure that two large scale marks are left in the picture. Figure 3 A schematic diagram of a water gauge image is shown in FIG. Figure 3 The size of the water ruler scale in the water ruler image is in an appropriate state.

[0031] See also Figure 4 Another schematic diagram of a water gauge image is shown in FIG. Figure 4 In the case of obstruction caused by a single rust or water surface fluctuation in the picture, the water gauge can be judged associatively based on the previous scale value to solve the recognition anomaly caused by scale obstruction in the case of a single rust or water surface fluctuation.

[0032] Step S104, grid-processing the water-rule image to obtain a plurality of grid-processed water-rule images; inputting the plurality of grid-processed water-rule images into the target detection model in parallel, and outputting a plurality of target detection results; wherein the target object detected by each target detection result is a number 0-9, a letter m or a letter M.

[0033] like Figure 2 As shown, this embodiment can perform water gauge card digital (small target) detection based on deep learning.

[0034] The input in this embodiment is a water level image of a ship taken by a shore-based camera. First, enter the digital detection image pre-processing module. Since the actual digital size of the business accounts for a small proportion of the entire image, the commonly used template detection based on deep learning is not effective. Small target detection is also a challenging link in target detection based on deep learning. To address this problem, this embodiment can perform an image cutting method based on 3×3 grid processing, and send the grid-processed image to the graphics card for parallel processing by setting the batch_size of the digital target detection model to ensure processing efficiency.

[0035] Batch_size refers to the number of samples used in each iteration during deep learning model training. It determines the amount of data used by the model when updating weights.

[0036] In some embodiments, non-target objects may be eliminated from multiple target detection results based on digital fusion non-maximum suppression.

[0037] In this embodiment, the results of parallel processing can be used to remove redundant target detections through digital fusion non-maximum suppression (NMS) method, and finally the digital detection of the ship water gauge card of the whole image is obtained.

[0038] Digital fusion non-maximum suppression is a post-processing technique commonly used in object detection tasks. It is mainly used to select the most appropriate bounding box from multiple candidate boxes and suppress those boxes that have high overlap with the optimal candidate box and low classification confidence.

[0039] In this embodiment, the target objects detected by each target detection result are all numbers 0-9, letters m or letters M, that is, there is only one type of target objects currently detected, including all occurring numbers and letters m / M: 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, m, M.

[0040] Step S106, inputting multiple target detection results into a digital classification model, and outputting multiple classification results.

[0041] like Figure 2 As shown, this embodiment can perform digital classification based on deep learning. The digital image is extracted by using the bounding_box area of ​​the digital target detection of the ship water gauge card and sent to the digital classification model based on deep learning for classification.

[0042] bounding_box is a rectangular box used to describe the position and range of the target in the image. A bounding box usually consists of a set of coordinates that indicate the position and size of the rectangle. Specifically, the bounding box can be determined by the x and y coordinates of the upper left corner of the rectangle and the x and y coordinates of the lower right corner, or by the coordinates of the center position of the bounding box and the width and height. In the object detection task, the model achieves object detection and positioning by predicting the bounding box of the target object.

[0043] In some embodiments, a digital classification model may also be trained based on a data augmentation balancing operation.

[0044] In the training of ship water gauge digital classification data, the numbers 2, 4, 6, 8, m, and M are particularly likely to appear. For odd numbers 1, 3, 5, 7, and 9, they only appear in meter readings. To address this data imbalance problem, data augmentation and balancing operations were performed during the training model phase, so that the classification effect of each category is optimized. Therefore, there is data association, so the deep learning-based digital classification module is serially run behind the deep learning-based water gauge digital (small target) detection module.

[0045] Step S108, inputting the water gauge image into the seawater region segmentation model, and outputting the seawater segmentation result of the water gauge image; wherein the pixel value of the seawater region position in the seawater segmentation result is 1.

[0046] like Figure 2 As shown, this embodiment can also perform seawater area segmentation based on deep learning. The seawater area segmentation module based on deep learning and the water gauge card digital (small target) detection module based on deep learning are parallel operation modules. For the same input image, seawater area segmentation is performed while digital detection and classification are performed, and the pixel value belonging to the seawater area position after segmentation is set to 1.

[0047] In some examples, the water gauge image can be input into the seawater region segmentation model; the seawater region segmentation model sequentially performs grayscale processing and binarization processing on the water gauge image; and the seawater region segmentation model detects the seawater segmentation result of the water gauge image after the binarization processing.

[0048] The specific processing method of this embodiment is to grayscale and binarize the image. Compared with color images, edge detection of grayscale images is more convenient and faster, and the amount of calculation is small, so the acquired color image is first converted into a grayscale image. The binarization of the image is to set the grayscale value of the pixel on the image to 0 to 255. The effect can be seen in Figure 5 The binary sea surface recognition processing diagram shown in FIG. 1 represents the grayscale value of each pixel obtained after sampling by a matrix, that is, the quantization of the grayscale image.

[0049] like Figure 5As shown in the figure, the foreground is in the middle of the image, the noise is greater than the middle, and the background is less than the middle. In a complex situation, the foreground is less than the threshold, and the background is greater than the threshold. For example, in the morning when the sunlight is not strong, the water gauge area is dark and the surrounding area is bright. By projecting the gray value onto the curve, the distribution characteristics of the gray value can be obtained.

[0050] Step S110, determining the reading of the water gauge card of the ship based on the multiple classification results and the seawater segmentation results.

[0051] After determining the classification results and the seawater segmentation results, this embodiment can determine the reading of the ship's water gauge card based on the multiple classification results and the seawater segmentation results, thereby calculating the reading of the ship's water gauge card.

[0052] In this embodiment, for a new water gauge image, appropriate image segmentation and coordinate encoding technology are used to enlarge the water gauge numbers. The target detection model based on deep learning extracts the numbers of the above segmented images respectively, and the extracted numbers are reverse-encoded to the original image through coordinates. For the overlapping numbers extracted from the segmented images, a weight-based non-maximum suppression method is used to improve the accuracy of digital positioning. Whenever a number is located, the image data in the extracted positioning frame is transferred to the deep learning classification network for reasoning, and the number is appropriately classified according to the reading recognition business.

[0053] In actual ship draft gauges, the number of odd numbers is significantly less than the number of even numbers. To address this, this embodiment introduces a small sample training technique to maintain a certain accuracy rate for the final classification results of odd and even numbers.

[0054] Ships often sail in seawater, and there are various interferences on the water gauge card, such as different degrees of rust and different degrees of occlusion. In this regard, this embodiment has developed a powerful post-processing reading inference algorithm, which can perform appropriate reading inference based on some of the recognized numbers when the numbers on individual water gauge cards are unclear or missing. The final reading is the intersection of the water gauge and the sea surface. For the sea surface contour, a deep learning segmentation algorithm is used. At the same time, when the seawater is relatively transparent or there is a water gauge reading reflected in the seawater, the contour definition method is used to exclude the reading in the seawater contour, so as to obtain a more accurate reading.

[0055] The embodiment of the present invention provides a method for identifying a ship's draft gauge card, which includes obtaining a draft gauge image of a ship taken by a shore-based camera; performing grid processing on the draft gauge image to obtain multiple grid-processed draft gauge images; inputting multiple grid-processed draft gauge images in parallel into a target detection model to output multiple target detection results; wherein the target object detected by each target detection result is a number 0-9, a letter m or a letter M; inputting multiple target detection results into a digital classification model to output multiple classification results; inputting the draft gauge image into a seawater region segmentation model to output a seawater segmentation result of the draft gauge image; wherein the pixel value of the seawater region position in the seawater segmentation result is 1; and determining the reading of the ship's draft gauge card based on multiple classification results and seawater segmentation results. In this method, an algorithm for automatically detecting the ship's waterline using image processing technology can be used to automatically detect the ship's draft gauge scale value, and a series of problems caused by manual visual inspection can be overcome, and the draft line position of the entire observation phase can be fully recorded, making subsequent data processing possible. This reduces system complexity and improves efficiency and accuracy.

[0056] Embodiment 2: This embodiment provides another method for identifying a ship's draft gauge, which is implemented on the basis of the above embodiment. This embodiment focuses on describing the specific steps of determining the reading of a ship's draft gauge based on multiple classification results and seawater segmentation results.

[0057] In some embodiments, classification results outside multiple seawater areas can be determined based on multiple classification results and seawater segmentation results; the classification results outside multiple seawater areas are divided into at least one letter classification result in which the target object is the letter m or the letter M and multiple digital classification results in which the target objects are numbers 0-9; based on the multiple digital classification results, the meter reading is determined at a preset threshold to the left of the letter m or the letter M in the letter classification result; wherein the meter reading includes: numbers 0-9; the meter reading with the highest confidence is used as the root node of the tree, the numbers in the multiple digital classification results that are greater than the meter reading with the highest confidence are used as the left subtree, and the numbers in the multiple digital classification results that are less than the meter reading with the highest confidence are used as the right subtree; based on the root node, the left subtree and the right subtree, the reading of the ship's water gauge card is determined.

[0058] like Figure 2 As shown, this embodiment can also calculate and infer the water gauge reading. After the inference is successfully completed, the reading calculation of the ship water gauge can be performed. First, the algorithm will find the letter classification results with the classification result of m / M among all the detected numbers, and find the meter reading in a certain threshold space to the left of m / M. Since the water gauge is arranged in a vertical direction, after successfully finding the meter reading, the remaining digital classification results can be divided into different intervals according to the image ordinate value of m / M.

[0059] By designing a tree structure to store the readings of the water gauge card, the number with the highest confidence (i.e. the meter reading with the highest confidence) is selected from all m / M as the root node of the tree, and the numbers greater than the root node (i.e. the numbers in the multiple digital classification results greater than the meter reading with the highest confidence) are stored as the left subtree, and the numbers less than the root node (i.e. the numbers in the multiple digital classification results less than the meter reading with the highest confidence) are stored as the right subtree. After looping through all the numbers, the reading of the entire water gauge card is generated.

[0060] In some embodiments, a ray emitted from a larger value to a smaller value can be formed based on the numbers in the smallest 2-digit classification results to determine the equation of the ray; when the ray touches the seawater area, the rice reading of the seawater area is determined based on the equation of the ray.

[0061] In actual business use, some numbers on the ship water gauge card are often blurred or rusted. Therefore, this embodiment also designs a reading inference algorithm. When some numbers are not fully recognized, the reading jump inference can be performed based on the Euclidean distance relationship between the numbers. The introduction of the reading inference algorithm can improve the final reading accuracy.

[0062] When the ship's water gauge readings fill the tree structure, the algorithm will take the two smallest readings to form a ray from the larger reading to the smaller reading. When the ray hits the seawater area, the reading algorithm module believes that the reading can be output according to the straight line equation.

[0063] In some embodiments, if the water gauge card of the ship is tilted, image correction may be performed on the water gauge image of the ship.

[0064] Due to the limited field of view of the shore-based camera, when the ship is docked forward or backward, the ship's water gauge in the captured image is often severely tilted, affecting the water gauge digital detection based on deep learning. For camera positions that will produce severely tilted images, this embodiment can adopt the method of first correcting and then processing the image.

[0065] In some embodiments, the standard PTZ value of the shore-based camera facing the sea surface and the real-time PTZ value of the shore-based camera after rotation can be obtained; homography matrix conversion is performed based on the standard PTZ value and the real-time PTZ value, and image correction is performed on the water level image of the ship based on the result of the homography matrix conversion.

[0066] PTZ value refers to the parameter value of the camera's pan / tilt control function, including Pan (horizontal rotation), Tilt (vertical pitch) and Zoom (zoom). Image correction first determines the PTZ value of the camera facing the sea surface. After the camera rotates, the homography matrix is ​​converted based on the newly acquired real-time PTZ and the PTZ facing the sea surface. After correction, the position of the ship's water gauge stuck in the image that was originally severely tilted becomes vertical. You can refer to Figure 6 A schematic diagram of image correction of a water gauge image is shown.

[0067] For shore camera shooting correction, you need to know whether the camera supports the acquisition of camera PTZ data. When the camera PTZ can be acquired, the tilted or bent scale is corrected to a nearly vertical angle by adjusting the horizontal and vertical matrix conversion. If the camera does not support PTZ acquisition, you can set the initial data to 0,0 and then perform matrix conversion.

[0068] In addition, during the identification process, it is necessary to ensure that the corrected water level gauge scale retains two large mark values ​​on the water surface to meet the identification requirements under various abnormal situations.

[0069] Since image correction will result in more dark areas, the digital detection model and seawater area segmentation model originally used under normal conditions will not be effective. Therefore, for severely tilted images, it is necessary to additionally collect ship water gauge card data from different camera positions and correct them before training.

[0070] In addition, this embodiment can also perform data cleaning of outliers during the recognition process. During the recognition process, environmental factors such as wind and waves, light, obstructions, rainy days, etc., which dynamically change during the recognition process, may cause abnormal image data to be recognized. Among hundreds of data, a certain scale of abnormal data will affect the correctness of the final data, so the data needs to be cleaned. Usually, the identification of outliers can be assisted by graphical methods (such as box plots, normal distribution diagrams) and modeling methods (such as linear regression, clustering algorithms, and K nearest neighbor algorithms).

[0071] In some embodiments, the reading data of the water gauge card may be cleaned by using a normal distribution method of the data.

[0072] In the process of water gauge identification, the reading value is consistent with the frequency of wind and wave surges on the sea surface. The correct identification data should be a wave-like curve graph. The highest and lowest values ​​of the data should be at the peak and trough of a certain period of data, so the data is regular. The normal distribution of data can be used to clean the data, which can not only remove more obvious identification errors, but also clean up invalid data when the wind and waves are too strong, ensuring that the data curve is close to the static horizontal line.

[0073] When using normal data distribution to clean outliers, the effect can be seen in Figure 7 A schematic diagram of unprocessed data and Figure 8 A schematic diagram of processed data is shown. It is obvious that the abnormal identification data has been cleaned, and the retained data presents a wave-like shape with ups and downs, indicating that the cleaned data is already real data, and in the case of long-term continuity, a single point error cannot have a significant impact on a large number of data sets.

[0074] Embodiment three: Corresponding to the above method embodiment, the present invention provides a ship water gauge card identification device. Fig. 9 A structural diagram of a ship water gauge card identification device, the ship water gauge card identification device comprises: A water gauge image acquisition module 91 is used to acquire a water gauge image of a ship taken by a shore-based camera; The target detection module 92 is used to perform grid processing on the water gauge image to obtain a plurality of grid-processed water gauge images; the plurality of grid-processed water gauge images are input into the target detection model in parallel, and a plurality of target detection results are output; wherein the target object detected by each target detection result is a number 0-9, a letter m or a letter M; A digital classification module 93, used to input multiple target detection results into a digital classification model and output multiple classification results; The seawater region segmentation module 94 is used to input the water gauge image into the seawater region segmentation model and output the seawater segmentation result of the water gauge image; wherein the pixel value of the seawater region position in the seawater segmentation result is 1; The water gauge reading determination module 95 is used to determine the water gauge reading of the ship based on the multiple classification results and the seawater segmentation results.

[0075] The embodiment of the present invention provides a ship water gauge card recognition device, which obtains a ship water gauge image taken by a shore-based camera; performs grid processing on the water gauge image to obtain multiple grid-processed water gauge images; inputs the multiple grid-processed water gauge images in parallel into a target detection model, and outputs multiple target detection results; wherein the target object detected by each target detection result is a number 0-9, a letter m or a letter M; inputs the multiple target detection results into a digital classification model, and outputs multiple classification results; inputs the water gauge image into a seawater region segmentation model, and outputs the seawater segmentation result of the water gauge image; wherein the pixel value of the seawater region position in the seawater segmentation result is 1; and determines the reading of the ship water gauge card based on the multiple classification results and the seawater segmentation results. In this method, the algorithm for automatically detecting the ship's waterline by using image processing technology can be used to automatically detect the ship's water gauge scale value, and a series of problems caused by manual visual inspection can be overcome, and the waterline position of the entire observation stage can be completely recorded, and subsequent data processing can be possible. It not only reduces the complexity of the system, but also improves efficiency and accuracy.

[0076] The above-mentioned device also includes: a non-target object elimination module, which is used to eliminate non-target objects from multiple target detection results based on a digital fusion non-maximum suppression method.

[0077] The above-mentioned device also includes: a digital classification model training module, which is used to train the digital classification model based on the data augmentation balancing operation.

[0078] The above-mentioned seawater area segmentation module is used to input the water gauge image into the seawater area segmentation model; the seawater area segmentation model sequentially performs grayscale processing and binarization processing on the water gauge image; the seawater area segmentation model detects the seawater segmentation result of the water gauge image after the binarization processing.

[0079] The above-mentioned water gauge card reading determination module is used to determine the classification results outside the multiple seawater areas based on multiple classification results and seawater segmentation results; divide the classification results outside the multiple seawater areas into at least one letter classification result in which the target object is the letter m or the letter M and multiple digital classification results in which the target objects are numbers 0-9; based on the multiple digital classification results, determine the meter reading at a preset threshold to the left of the letter m or the letter M in the letter classification result; wherein the meter reading includes: numbers 0-9; take the meter reading with the highest confidence as the root node, take the numbers in the multiple digital classification results that are greater than the meter reading with the highest confidence as the left subtree, and take the numbers in the multiple digital classification results that are less than the meter reading with the highest confidence as the right subtree; based on the root node, the left subtree and the right subtree, determine the reading of the water gauge card of the ship.

[0080] The water gauge card reading determination module is also used to form a ray emitted from a larger value to a smaller value based on the numbers in the smallest 2 digital classification results, and determine the equation of the ray; when the ray touches the sea water area, the meter reading of the sea water area is determined based on the equation of the ray.

[0081] The above-mentioned device comprises: an image correction module, which is used to perform image correction on the water gauge image of the ship if the water gauge card of the ship is tilted.

[0082] The above-mentioned image correction module is used to obtain the standard PTZ value of the shore-based camera facing the sea surface, and the real-time PTZ value of the shore-based camera after rotation; perform homography matrix conversion based on the standard PTZ value and the real-time PTZ value, and perform image correction on the water level image of the ship based on the result of the homography matrix conversion.

[0083] The above device comprises: a data cleaning module, which is used to clean the reading data of the water gauge card by adopting the normal distribution method of the data.

[0084] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the ship draft card identification system described above can refer to the corresponding process in the aforementioned embodiment of the ship draft card identification method, and will not be repeated here.

[0085] Embodiment 4: The embodiment of the present invention also provides an electronic device for executing the above-mentioned ship water gauge card identification method; see Fig.10 A structural schematic diagram of an electronic device is shown, the electronic device includes a memory 100 and a processor 101, wherein the memory 100 is used to store one or more computer instructions, and the one or more computer instructions are executed by the processor 101 to implement the above-mentioned ship water gauge card identification method.

[0086] Further, Fig.10 The electronic device shown further includes a bus 102 and a communication interface 103 , and the processor 101 , the communication interface 103 and the memory 100 are connected via the bus 102 .

[0087] The memory 100 may include a high-speed random access memory (RAM), and may also include a non-volatile memory, such as at least one disk storage. The communication connection between the system network element and at least one other network element is realized through at least one communication interface 103 (which may be wired or wireless), and the Internet, wide area network, local area network, metropolitan area network, etc. may be used. The bus 102 may be an ISA bus, a PCI bus, or an EISA bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Fig.10 Only one bidirectional arrow is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0088] The processor 101 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit or software instructions in the processor 101. The above processor 101 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present invention can be implemented or executed. The general processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in conjunction with the embodiments of the present invention can be directly embodied as a hardware decoding processor for execution, or a combination of hardware and software modules in the decoding processor for execution. The software module may be located in a storage medium mature in the art, such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory 100, and the processor 101 reads the information in the memory 100 and completes the steps of the method of the above embodiment in combination with its hardware.

[0089] An embodiment of the present invention also provides a computer-readable storage medium, which stores computer-executable instructions. When the computer-executable instructions are called and executed by a processor, the computer-executable instructions prompt the processor to implement the above-mentioned ship water gauge card identification method. The specific implementation can be found in the method embodiment, which will not be repeated here.

[0090] The computer program product of the ship draft card identification method and device provided in the embodiments of the present invention includes a computer-readable storage medium storing program code. The instructions included in the program code can be used to execute the method in the previous method embodiment. The specific implementation can be found in the method embodiment, which will not be repeated here.

[0091] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the system and / or device described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0092] In addition, in the description of the embodiments of the present invention, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0093] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods of each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0094] In the description of the present invention, it should be noted that the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", etc., indicating the orientation or positional relationship, are based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the terms "first", "second", and "third" are used for descriptive purposes only, and cannot be understood as indicating or implying relative importance.

[0095] Finally, it should be noted that the above-described embodiments are only specific implementations of the present invention, which are used to illustrate the technical solutions of the present invention, rather than to limit them. The protection scope of the present invention is not limited thereto. Although the present invention is described in detail with reference to the above-mentioned embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-mentioned embodiments within the technical scope disclosed by the present invention, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A method for identifying a ship water gauge card, characterized in that: The method comprises: Obtain the water gauge image of the ship taken by the shore-based camera; Performing grid processing on the water gauge image to obtain a plurality of grid-processed water gauge images; inputting the plurality of grid-processed water gauge images into a target detection model in parallel to output a plurality of target detection results; wherein the target object detected by each target detection result is a number 0-9, a letter m or a letter M; Inputting a plurality of the target detection results into a digital classification model and outputting a plurality of classification results; Input the water gauge image into the seawater region segmentation model, and output the seawater segmentation result of the water gauge image; wherein the pixel value of the seawater region position in the seawater segmentation result is 1; The reading of the water gauge card of the ship is determined based on the plurality of classification results and the seawater segmentation results.

2. The method according to claim 1, characterized in that: After the step of outputting a plurality of target detection results, the method further comprises: Non-target objects are eliminated from the plurality of target detection results based on a digital fusion non-maximum suppression method.

3. The method according to claim 1, characterized in that: The method further comprises: The digit classification model is trained based on a data augmentation balancing operation.

4. The method according to claim 1, characterized in that: The step of inputting the water gauge image into a seawater region segmentation model and outputting a seawater segmentation result of the water gauge image comprises: Inputting the water gauge image into a seawater region segmentation model; The seawater area segmentation model sequentially performs grayscale processing and binarization processing on the water gauge image; The seawater region segmentation model detects the seawater segmentation result of the water gauge image after binarization processing.

5. The method according to claim 1, characterized in that The step of determining the reading of the water gauge card of the ship based on the plurality of classification results and the seawater segmentation results comprises: Determining the classification results outside a plurality of seawater areas based on the plurality of classification results and the seawater segmentation results; Dividing the classification results outside the plurality of seawater areas into at least one letter classification result in which the target object is the letter m or the letter M and a plurality of number classification results in which the target objects are numbers 0-9; Based on the plurality of the digital classification results, determining a meter reading at a preset threshold to the left of the letter m or the letter M of the letter classification result; wherein the meter reading includes: numbers 0-9; The meter reading with the highest confidence is used as a tree root node, the numbers in the multiple digital classification results that are greater than the meter reading with the highest confidence are used as a left subtree, and the numbers in the multiple digital classification results that are less than the meter reading with the highest confidence are used as a right subtree; Based on the root node, the left subtree and the right subtree determine the reading of the water gauge card of the ship.

6. The method according to claim 5, characterized in that The method further comprises: Forming a ray emitted from a larger value to a smaller value based on the number in the smallest 2-digit classification result, and determining an equation of the ray; When the ray hits a seawater area, a meter reading of the seawater area is determined based on an equation of the ray.

7. The method according to any one of claims 1 to 6, characterized in that: After the step of acquiring the water gauge image of the ship taken by the shore-based camera, the method further comprises: If the water gauge card of the ship is tilted, image correction is performed on the water gauge image of the ship.

8. The method according to claim 7, characterized in that If the water gauge card of the ship is tilted, the step of performing image correction on the water gauge image of the ship comprises: Obtain the standard PTZ value of the shore-based camera facing the sea surface, as well as the real-time PTZ value of the shore-based camera after rotation; A homography matrix conversion is performed based on the standard PTZ value and the real-time PTZ value, and image correction is performed on the water level image of the ship based on the result of the homography matrix conversion.

9. The method according to any one of claims 1 to 6, characterized in that: The method further comprises: The reading data of the water gauge card is cleaned by using a normal distribution method of the data.

10. A ship water gauge card identification device, characterized in that: The device comprises: A water gauge image acquisition module is used to acquire the water gauge image of the ship taken by a shore-based camera; The target detection module is used to perform grid processing on the water gauge image to obtain a plurality of grid-processed water gauge images; the plurality of grid-processed water gauge images are input into the target detection model in parallel, and a plurality of target detection results are output; wherein the target object detected by each target detection result is a number 0-9, a letter m or a letter M; A digital classification module, used to input the plurality of target detection results into a digital classification model and output a plurality of classification results; A seawater region segmentation module, used for inputting the water gauge image into a seawater region segmentation model, and outputting a seawater segmentation result of the water gauge image; wherein the pixel value of the seawater region position in the seawater segmentation result is 1; A water gauge card reading determination module is used to determine the water gauge card reading of the ship based on the multiple classification results and the seawater segmentation results.

Images