Unmanned aerial vehicle-based ship water gauge weighing method and system

Abstract

The invention relates to the technical field of water gauge metering, in particular to a ship water gauge weighing method and system based on an unmanned aerial vehicle, and the method comprises the steps: obtaining continuous multi-frame ship water gauge images through a camera device carried by the unmanned aerial vehicle, and carrying out the preprocessing of the ship water gauge images; searching the preprocessed ship water gauge image to obtain a region of interest; and inputting the region of interest into the image processing model, performing character recognition on each frame of water gauge image to obtain a water level recognition result, and obtaining a water gauge weighing result based on the water level recognition result. According to the method, the weighing efficiency and accuracy can be improved, and the labor cost and the error rate are reduced.

Description

Technical Field

[0001] The invention relates to the technical field of water gauge measurement, and in particular to a ship water gauge weight measurement method and system based on an unmanned aerial vehicle. Background Art

[0002] As a tool for measuring water surface height, the water gauge plays a key role in the shipping field. It is specifically used to indicate the depth of the ship immersed in water, that is, the draft depth. This kind of ruler or scale device has evolved into various forms according to different application scenarios and needs. Specifically, the water gauge can be classified into several types, such as the traditional and stable fixed water gauge, which is firmly fixed on the hull, moves with the ship, and directly reflects the draft changes; there are floating water gauges designed to adapt to water level fluctuations, and its scale part can rise and fall with the water surface to improve the accuracy of readings; there are also telemetry water gauges and electronic water gauges realized by modern technology, which provide more convenient and high-precision water level data through remote monitoring or electronic display technology.

[0003] However, in the daily operation of ships, the most common and basic one is the water gauge that is directly fixed on the hull. This design is simple and effective. It is not only convenient for the crew to observe and record the draft of the ship at any time, but also ensures the directness and accuracy of the data. It is an indispensable auxiliary tool for safe navigation and cargo loading management. In short, the water gauge commonly used on ships is a special ruler fixed on the hull to reflect the draft depth of the ship in real time.

[0004] The water level gauge cannot be measured under complex conditions such as local dirt and damage, low image resolution, insufficient natural light, and excessive artificial light. However, in actual field applications, the distance between the water level gauge and the shooting equipment is far, resulting in low image resolution and more interference noise. In addition, changes in the intensity of light in the field have a greater impact on the imaging effect. On sunny days, the water surface is glaring, backlit, the water body is transparent, and the reflection of the water level gauge is strong; on rainy days, the grayscale values ​​of the water level gauge and the water surface are close, and the image signal-to-noise ratio is reduced; at night, artificial light is overexposed, making it difficult for traditional image processing technology based on grayscale and edge information to accurately and effectively detect the position of the water level line. At the same time, the measurement results of occlusion conditions such as floating objects wrapped around the water level gauge sheet at the water level line cannot be effectively identified. This leads to relatively large errors in the subsequent water level gauge weight measurement. Summary of the invention

[0005] The technical problem to be solved by the present invention is to provide a method and system for measuring the weight of a ship draft gauge based on an unmanned aerial vehicle in view of the deficiencies in the above-mentioned prior art, thereby solving the problem of low efficiency and low accuracy of draft gauge weighing under complex conditions.

[0006] The objective of the present invention is achieved by the following technical solutions: In a first aspect, the present invention provides a method for measuring the weight of a ship's draft gauge based on an unmanned aerial vehicle, comprising: Acquire multiple frames of continuous ship water gauge images through a camera device carried by the drone, and pre-process the ship water gauge images; Search the preprocessed ship water gauge image to obtain the area of ​​interest; The region of interest is input into the trained image processing model, and character recognition is performed on each frame of the water gauge image to obtain the water level recognition result, and the water gauge weight result is obtained based on the water level recognition result.

[0007] As a further improvement of the present invention, the preprocessing of the ship water gauge image specifically includes: performing Gaussian filtering or bilateral filtering on the multiple frames of ship water gauge images to smooth the images.

[0008] As a further improvement of the present invention, after the ship water gauge image is processed by bilateral filtering, the pixel value of a certain point in the image is:

[0009] In the formula, (x, y) is the pixel coordinate point, Gx, y is the center point area of ​​the set size, (i, j) is the pixel point in the center point area, is the bilateral filter weight, is the gray value, It is the weighted average of the pixel brightness in the area.

[0010] As a further improvement of the present invention, the pre-processed ship water level image is searched to obtain the region of interest, specifically including: identifying the part with water body and water level mark in the ship water level image, and taking the fixed area of ​​the part with water body and water level mark as the region of interest.

[0011] As a further improvement of the present invention, the image processing model includes a data recognition module and a feature extraction module. The data recognition module is used to perform edge detection on the input water gauge image, obtain the water gauge connected domain based on the detection result, and identify the waterline position based on the water gauge connected domain; the feature extraction module is used to identify and extract the features of the water gauge characters to obtain the integer and decimal parts of the water gauge.

[0012] As a further improvement of the present invention, the data identification module specifically includes: The input color image is converted into grayscale, and the edge calculation of the grayscale converted area is performed through the edge detection algorithm to obtain a binary image; the image is fully scanned, and the connected domains are marked according to the scanning results; the centroid, second-order moment, and minimum enclosing rectangle of each connected domain are calculated; according to the centroid, second-order moment, and minimum enclosing rectangle of the connected domain, the rules are set to preliminarily eliminate the interfering connected domains and retain the candidate connected domains of the water ruler engraving.

[0013] As a further improvement of the present invention, after the edge calculation is performed on the area after the grayscale transformation by the edge detection algorithm to obtain the binary image, it also includes searching for the area where the decimal part of the water gauge is located according to the geometric shape constraints and spatial position distribution rules of the water gauge characters; the area where the decimal part of the water gauge is located is cropped out from the input image to obtain a mask of the water gauge area of ​​interest, filtering and smoothing the color image of interest to the water gauge, using color differences to distinguish between the hull and the water body, and determining the waterline position in the image; after determining the waterline position, locating the integer part of the water gauge engraving and identifying the value represented by the integer engraving, forming a connected domain through the decimal part of the water gauge and the integer part of the water gauge engraving, reading the value of the decimal part of the water gauge, and completing the water gauge reading.

[0014] As a further improvement of the present invention, the feature extraction module specifically includes: calculating the gradient of each pixel point of the image in the region of interest in the x and y axis directions; dividing the water level gauge image into a number of units, and evenly dividing them into 9 intervals in the gradient direction; performing histogram statistics on all gradient directions in each unit to obtain a 9-bit feature vector for each unit; and performing classification and recognition based on the detected water level gauge character features and corresponding digital labels to obtain the final water level gauge value.

[0015] As a further improvement of the present invention, the method of acquiring continuous multiple frames of ship water gauge images through the camera device carried by the drone also includes the step of filtering out redundant background of the input image, specifically including: taking the baseline as the screening threshold, processing the horizontal histogram and vertical histogram of the image pixel points, and filtering out redundant background of the water gauge image and the water gauge character "E" scale image.

[0016] In a second aspect, the present invention further provides a drone-based ship draft gauge weighing system, which is used to implement the drone-based ship draft gauge weighing method, comprising: The UAV data acquisition module is used to obtain multiple frames of continuous ship water gauge images through the camera device carried by the UAV, and pre-process the ship water gauge images; The data processing module is used to search the pre-processed ship water gauge image to obtain the area of ​​interest; The image recognition module is used to input the region of interest into the image processing model, perform character recognition on each frame of the water gauge image, obtain the water level recognition result, and obtain the water gauge weight result based on the water level recognition result.

[0017] The beneficial effect of the present invention is that the ship water gauge weighing method based on the drone of the present invention uses the high-definition camera device carried by the drone to continuously collect multiple frames of images of the ship water gauge. The drone hovers above the water gauge mark, and by adjusting the angle and height, the captured image is ensured to be clear and complete. Compared with traditional manual observation, the drone image collection is safer and more efficient, and can reduce the influence of human factors on the observation results. The collected ship water gauge image is preprocessed to eliminate noise in the image, enhance contrast, correct image distortion, etc. The image quality is improved, laying a good foundation for subsequent character recognition and data analysis. Using image search or image segmentation technology, the region of interest (ROI) containing the water gauge scale is searched from the preprocessed image. This step usually involves feature extraction and matching of the image to quickly locate the key area, reduce the calculation amount of the subsequent character recognition process, and improve the recognition efficiency and accuracy. The region of interest is input into the image processing model to recognize the characters in each frame of the water gauge image. According to the character recognition results, the water level value corresponding to each frame of the image is calculated. Then, the water gauge weighing result is calculated through mathematical model in combination with the ship's displacement, cargo capacity and other parameters. This realizes the automation and intelligence of water gauge weighing, improves the weighing efficiency and accuracy, and reduces labor costs and error rates. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0019] Figure 1 This is a flow chart of a method for measuring the weight of a ship's draft gauge based on an unmanned aerial vehicle in an embodiment of the present invention; Figure 2 It is a schematic diagram of the structure of a ship draft gauge weighing system based on an unmanned aerial vehicle in an embodiment of the present invention. DETAILED DESCRIPTION

[0020] In order to make the purpose and technical solution of the present invention clearer and easier to understand, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. The specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0021] The technical solution of the present invention will be clearly and completely described below in conjunction with the accompanying drawings and specific embodiments, wherein the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments.

[0022] Example 1 like Figure 1 As shown, this embodiment provides a method for measuring the weight of a ship's draft gauge based on a drone, and the specific implementation method is as follows.

[0023] Firstly, a plurality of continuous frames of ship water gauge images are acquired by a camera device carried by a UAV, and the ship water gauge images are preprocessed.

[0024] Choose an aerial photography drone with stable hovering performance and high-definition video recording capabilities to ensure that the images of the ship's water gauge area or water gauge can be accurately obtained under different environmental conditions. The drone is equipped with a high-definition gimbal camera with good optical zoom and image stabilization functions to clearly capture water gauge images. At the same time, the drone is equipped with an efficient wireless image module that can stably transmit the captured video stream data back to the gimbal in real time.

[0025] First, the read color image is converted to grayscale, and then the edge scanning and detection are performed on the area with the most obvious changes in the image. For this embodiment, the edge mainly exists between the water ruler lettering, and between the water ruler lettering and the steel plate background, which presents color anomalies and mutations, pixel breaks and discontinuities.

[0026] Gaussian filtering or bilateral filtering is performed on the acquired multi-frame ship water level images to smooth the images. The difference between bilateral filtering and Gaussian filtering is: first, Gaussian filtering has linear characteristics, while bilateral filtering is nonlinear; second, bilateral filtering can effectively remove noise and retain the details of the image edge, but although Gaussian filtering can make the image smooth, it is relatively weak in detail processing ability. Therefore, this embodiment selects bilateral filtering to better complete the noise processing of the image. Specifically: it optimizes the filter weight coefficient into the product of the image brightness information and the Gaussian function, and the product value is then convolved with the image brightness information.

[0027] After the ship water gauge image is processed by bilateral filtering, the pixel value of a point in the image is:

[0028] In the formula, (x, y) is the pixel coordinate point, Gx, y is the center point area of ​​the set size, (i, j) is the pixel point in the center point area, is the bilateral filter weight, is the gray value, It is the weighted average of the pixel brightness in the area.

[0029] The pre-processed ship water gauge image is searched to obtain the region of interest; the part with water body and water gauge mark in the ship water gauge image is identified, and the fixed area of ​​the part with water body and water gauge mark is used as the region of interest. Specifically, the ROI (region of interest) of this embodiment is usually composed of the water surface in the lower part of the image and the waterline in the middle part and the hull with water gauge engraving above, and it is easy to detect this area. The ROI detection adopts an image segmentation method. This method adopts a fixed ratio based on the observation and experience summary of the water body in the image and the water gauge mark area, specifically a fixed area within 1 / 3 above and below the image center line as the ROI area of ​​a certain frame image. The subsequent recognition processing is only performed in the ROI area, including the positioning detection and recognition of the engraving of the decimal part and integer part of the water gauge, and the recognition of the boundary line (waterline) between the hull and the water body.

[0030] In addition, obtaining continuous multiple frames of ship water gauge images through the camera device carried by the drone also includes the step of filtering out redundant background of the input image, specifically including: taking the baseline as the screening threshold, processing the horizontal histogram and vertical histogram of the image pixel points, and filtering out redundant background of the water gauge image and the water gauge character "E" scale image.

[0031] The region of interest is input into the image processing model, and character recognition is performed on each frame of the water gauge image to obtain the water level recognition result, and the water gauge weight result is obtained based on the water level recognition result.

[0032] The image processing model includes a data recognition module and a feature extraction module. The edge detection module is used to perform edge detection on the input water gauge image and obtain the water gauge connected domain based on the detection result; the feature extraction module is used to recognize and extract the features of the water gauge characters.

[0033] The data identification module specifically includes: The input color image is converted into grayscale, and the edge calculation of the grayscale converted area is performed through the edge detection algorithm to obtain a binary image; the image is fully scanned, and the connected domains are marked according to the scanning results; the centroid, second-order moment, and minimum enclosing rectangle of each connected domain are calculated; according to the centroid, second-order moment, and minimum enclosing rectangle of the connected domain, the rules are set to preliminarily eliminate the interfering connected domains and retain the candidate connected domains of the water ruler engraving.

[0034] The edge detection algorithm is obtained based on the Canny edge detection operator.

[0035] After edge calculation is performed on the grayscale transformed area by edge detection algorithm to obtain a binary image, the method also includes searching for the area where the decimal part of the water gauge is located according to the geometric shape constraints and spatial position distribution rules of the water gauge characters; the area where the decimal part of the water gauge is located is cropped out from the input image to obtain a mask of the water gauge area of ​​interest, filtering and smoothing the color image of interest to the water gauge, using color differences to distinguish between the hull and the water body, and determining the waterline position in the image; after determining the waterline position, locating the integer part of the water gauge engraving and identifying the value represented by the integer engraving, forming a connected domain through the decimal part of the water gauge and the integer part of the water gauge engraving, reading the value of the decimal part of the water gauge, and completing the water gauge reading.

[0036] Among them, the black value in the mask is 0, and the white value is 1. The mask is multiplied by the original water ruler ROI area, and the block with a value of 0 identifies the pixel of the corresponding color image and does not participate in subsequent processing.

[0037] In addition, this embodiment can filter and smooth the color image of the water gauge ROI area, such as using bilateral filtering or weighted median filtering, to smooth the noise while avoiding the smoothing of the waterline edge. Considering that the hull color of a large bulk carrier is generally one or a combination of white, black, blue, and red, and the dock water is generally gray, muddy yellow or green, the color difference can be used to distinguish the hull and the water, so as to determine the waterline position in the image.

[0038] The feature extraction module specifically includes: calculating the gradient of each pixel point in the image within the area of ​​interest in the x and y axis directions; dividing the water level image into several units and evenly dividing them into 9 intervals in the gradient direction; performing histogram statistics on all gradient directions in each unit to obtain a 9-bit feature vector for each unit; performing classification and recognition based on the detected water level character features and the corresponding digital labels to obtain the final water level value.

[0039] This embodiment uses a feature extractor to extract features from an image. For example, common feature extraction operators include LBP, Haar-like, and HoG. This embodiment uses HoG as a feature extraction operator for a ship water gauge image.

[0040] The implementation of the HoG feature extraction algorithm includes six steps: first, in order to reduce external influences, the size of the water gauge image is adjusted from any size to a specified uniform size, and the water gauge image is binarized to avoid interference and achieve normalization of the color space; second, the gradient value of each pixel in the x and y directions is calculated; then, the water gauge image is divided into several cells through intervention means; then, the gradient direction is evenly divided into 9 intervals (bins), and the gradient directions of all pixels in each cell are histogram-statisticed to obtain the 9-dimensional feature vector of the cell; then, blocks are constructed, each block consists of several cells; finally, the required feature vector is calculated.

[0041] Read the water ruler engraving images in the training set one by one, extract the HoG feature vector according to the above HoG feature extraction steps, pair the HoG feature vector of each image with its digital label, and then construct a feature space with labels. Next, a suitable classifier is needed to divide the feature space to ensure that each feature vector can be classified into the correct category. This can be verified by comparing the true label with the maximum probability label output by the classifier, and iteratively training the classifier according to the calculated classification accuracy or classification loss function.

[0042] At this point, the integer part of the water gauge, the decimal part of the engraving and the waterline position have been detected, and the value of the integer part of the water gauge has been identified. The last step is to read the value of the decimal part of the water gauge to complete the water gauge reading.

[0043] In summary, the drone image collection method reduces the risk of manual observation and improves operational safety. Through image preprocessing and character recognition technology, the influence of human factors on the weighing results is reduced, and the accuracy of weighing is improved. According to the character recognition results, the water level value corresponding to each frame of the image is calculated. Then, combined with the ship's displacement, cargo capacity and other parameters, the water gauge weighing results are calculated through a mathematical model. The automation and intelligence of water gauge weighing are realized, the weighing efficiency and accuracy are improved, and the labor cost and error rate are reduced.

[0044] Example 2 In another embodiment of the present invention, a ship draft gauge weighing system based on a drone is provided, which can be used to implement the above-mentioned ship draft gauge weighing method based on a drone, specifically as follows: Figure 2 As shown, the UAV-based ship draft gauge weighing system includes a UAV data acquisition module, a data processing module and an image recognition module.

[0045] The UAV data acquisition module is used to obtain multiple frames of continuous ship water gauge images through the camera device carried by the UAV, and pre-process the ship water gauge images; The data processing module is used to search the pre-processed ship water gauge image to obtain the area of ​​interest; The image recognition module is used to input the region of interest into the image processing model, perform character recognition on each frame of the water gauge image, obtain the water level recognition result, and obtain the water gauge weight result based on the water level recognition result.

[0046] The drone data acquisition module is communicatively connected with the data processing module, and the data processing module can be arranged in the drone or on the terminal device.

[0047] The terminal device includes a processor and a memory, the memory is used to store a computer program, the computer program includes program instructions, and the processor is used to execute the program instructions stored in the computer storage medium. The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. It is the computing core and control core of the terminal, which is suitable for implementing one or more instructions, and is specifically suitable for loading and executing one or more instructions to implement the corresponding method flow or corresponding function; the processor described in the embodiment of the present invention can be used for the operation of the ship water level image recognition method of the unmanned aerial vehicle.

Claims

1. A method for measuring the weight of a ship's draft gauge based on an unmanned aerial vehicle, characterized in that: include: Acquire multiple frames of continuous ship water gauge images through a camera device carried by the drone, and pre-process the ship water gauge images; Search the preprocessed ship water gauge image to obtain the area of ​​interest; The region of interest is input into the trained image processing model, and character recognition is performed on each frame of the water gauge image to obtain the water level recognition result, and the water gauge weight result is obtained based on the water level recognition result.

2. The method for measuring ship draft gauge weight based on drone according to claim 1 is characterized in that: The preprocessing of the ship water gauge image specifically includes: performing Gaussian filtering or bilateral filtering on the multiple frames of ship water gauge images to smooth the images.

3. The method for measuring ship draft based on drone according to claim 2 is characterized in that: After the ship water gauge image is processed by bilateral filtering, the pixel value of a point in the image is: In the formula, (x, y) is the pixel coordinate point, Gx, y is the center point area of ​​the set size, (i, j) is the pixel point in the center point area, is the bilateral filter weight, is the gray value, It is the weighted average of the pixel brightness in the area.

4. The method for measuring ship draft gauge weight based on unmanned aerial vehicle according to claim 2 is characterized in that: The method of searching the preprocessed ship draft gauge image to obtain the region of interest specifically includes: identifying the portion of the ship draft gauge image having water bodies and draft gauge marks, and taking a fixed region of the portion having water bodies and draft gauge marks as the region of interest.

5. The method for measuring ship draft gauge weight based on unmanned aerial vehicle according to claim 1, characterized in that: The image processing model includes a data recognition module and a feature extraction module. The data recognition module is used to perform edge detection on the input water gauge image, obtain the water gauge connected domain based on the detection result, and identify the waterline position based on the water gauge connected domain; The feature extraction module is used to identify and extract the features of the water ruler characters to obtain the integer and decimal parts of the water ruler.

6. The method for measuring ship draft gauge weight based on unmanned aerial vehicle according to claim 5 is characterized in that: The data identification module specifically includes: The input color image is converted into grayscale, and the edge calculation of the grayscale converted area is performed through the edge detection algorithm to obtain a binary image; the image is fully scanned, and the connected domains are marked according to the scanning results; the centroid, second-order moment, and minimum enclosing rectangle of each connected domain are calculated; according to the centroid, second-order moment, and minimum enclosing rectangle of the connected domain, the rules are set to preliminarily eliminate the interfering connected domains and retain the candidate connected domains of the water ruler engraving.

7. The method for measuring ship draft gauge weight based on unmanned aerial vehicle according to claim 6, characterized in that: After edge calculation is performed on the grayscale transformed area by the edge detection algorithm to obtain a binary image, the method further includes searching for the area where the decimal part of the water gauge is located according to the geometric shape constraints and spatial position distribution rules of the water gauge characters; cropping the area where the decimal part of the water gauge is located from the input image to obtain a mask of the water gauge area of ​​interest, filtering and smoothing the color image of interest to the water gauge, using color differences to distinguish the hull and the water body, and determining the waterline position in the image; after determining the waterline position, locating the integer part of the water gauge engraving and identifying the value represented by the integer engraving, forming a connected domain through the decimal part of the water gauge and the integer part of the water gauge engraving, reading the value of the decimal part of the water gauge, and completing the water gauge reading.

8. The method for measuring ship draft gauge weight based on unmanned aerial vehicle according to claim 7 is characterized in that: The feature extraction module specifically includes: calculating the gradient of each pixel point of the image in the region of interest in the x and y axis directions; dividing the water gauge image into a number of units, and evenly dividing them into 9 intervals in the gradient direction; performing histogram statistics on all gradient directions in each unit to obtain a 9-bit feature vector for each unit; and performing classification and recognition based on the detected water gauge character features and corresponding digital labels to obtain the final water gauge value.

9. The method for measuring ship draft gauge weight based on unmanned aerial vehicle according to claim 1, characterized in that: The method of obtaining multiple frames of continuous ship water gauge images by using a camera device carried by an unmanned aerial vehicle also includes the step of filtering out redundant background of the input image, specifically including: taking the baseline as the screening threshold, processing the horizontal histogram and the vertical histogram of the image pixel points, and filtering out redundant background of the water gauge image and the water gauge character "E" scale image.

10. A ship draft gauge weighing system based on an unmanned aerial vehicle, used to implement the ship draft gauge weighing method based on an unmanned aerial vehicle according to any one of claims 1 to 9, characterized in that: include: The UAV data acquisition module is used to obtain multiple frames of continuous ship water gauge images through the camera device carried by the UAV, and pre-process the ship water gauge images; The data processing module is used to search the pre-processed ship water gauge image to obtain the area of ​​interest; The image recognition module is used to input the region of interest into the image processing model, perform character recognition on each frame of the water gauge image, obtain the water level recognition result, and obtain the water gauge weight result based on the water level recognition result.

Images