Unmanned aerial vehicle ship water gauge intelligent identification system based on image identification

Abstract

The invention discloses an unmanned aerial vehicle ship water gauge intelligent recognition system based on image recognition, which is based on a miniature unmanned aerial vehicle platform, integrates monocular and binocular camera combination, measures the distance between an unmanned aerial vehicle and a ship body in real time through a stereoscopic vision technology, and provides key data support for path planning and obstacle avoidance. A double-Bluetooth beacon base station is deployed on a shore base, a real-time navigation network is constructed, and a take-off point and state switching node positioning service is provided for the unmanned aerial vehicle; the system has a path memory function, can completely record full-voyage trajectory information, and automatically executes original path return voyage after a task is finished; through cooperative application of multi-sensor fusion perception, intelligent path planning, a dynamic obstacle avoidance technology, deep learning and a computer vision technology, a complete ship water gauge intelligent identification system is constructed, the port ship draught detection efficiency is remarkably improved, the labor cost is reduced, and key technical support is provided for intelligent operation and maintenance of ships.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of ship water gauge detection, in particular to an unmanned aerial vehicle (UAV) ship water gauge intelligent identification system based on image recognition, which is suitable for unmanned intelligent detection scenarios of large water transport tools such as port cargo ships. BACKGROUND

[0002] The UAV can quickly and accurately calculate the specific draft value of the ship water gauge by shooting the ship water gauge image and with the help of advanced image processing and analysis technology. This achievement has become an important breakthrough and application highlight of computer vision technology in the field of ship water gauge detection and identification. To ensure that the actual draft data of the ship water gauge can be obtained intelligently and accurately, the current primary task is to build a scientific, reasonable and complete full-automatic UAV ship water gauge intelligent identification system. The system needs to achieve the autonomous planning and execution of the scanning flight task of the ship water gauge by the UAV, and to overcome the core technical challenge of using computer vision algorithms to accurately calculate the water gauge image shot by the UAV to obtain the actual draft value.

[0003] Currently, there are mainly two operation modes of image processing technology applied to water gauge identification: one is to fix the camera at a specific position to shoot the ship water gauge image; the other is to manually control the UAV to complete the collection of the ship water gauge image, and then the image is returned to the shore host for subsequent processing and calculation. However, both methods have the problems of low efficiency and high labor cost, and it is difficult to shoot multiple water gauges of a ship in a week, which requires human resources to shoot the water gauge on the shore or at sea. More importantly, once the key elements in the scene (such as the replacement of the ship) change, manual image collection work needs to be carried out again, which undoubtedly further increases the complexity and workload of the operation.

[0004] Path planning is one of the core links of the UAV autonomous shooting of the water gauge image, and its purpose is to enable the UAV to autonomously plan the optimal path from the starting point to the ending point through the on-site environment and the equipment carried, to ensure that the UAV can efficiently and comprehensively shoot the ship water gauge, and also to save manpower, so that the UAV can autonomously realize path planning without human intervention through the built-in algorithm.

[0005] In the existing ship scanning and photographing technology, the SLAM algorithm helps the unmanned aerial vehicle to realize autonomous path planning. Through the laser radar and the camera, the unmanned aerial vehicle constructs a three-dimensional map of the ship in real time and accurately locates its own position, dynamically plans an optimal scanning path to cover the entire ship surface. However, the process of constructing the map is time-consuming and requires a large amount of computing resources; and the three-dimensional map constructed is only for a specific ship docking scene. When the scene elements (such as ship replacement) change, the complete modeling process needs to be re-executed, resulting in low scene migration efficiency and difficulty in adapting to the actual operation requirements of the dynamic port. SUMMARY

[0006] In order to solve the above problems existing in the prior art, the present application proposes an unmanned aerial vehicle ship water gauge intelligent identification system based on image recognition, which can improve the port ship water gauge identification calculation efficiency and reduce the labor cost.

[0007] The basic idea of the present application is as follows: an unmanned aerial vehicle ship water gauge intelligent identification system based on image recognition, based on a micro unmanned aerial vehicle platform, integrating a combination of monocular and binocular cameras: a monocular camera is installed forward for real-time environment perception, and a binocular camera is arranged laterally to measure the distance between the unmanned aerial vehicle and the ship body in real time through stereo vision technology, providing key data support for path planning and obstacle avoidance. Two Bluetooth beacon base stations are deployed on the shore base to construct a real-time navigation network to provide take-off point and state switching node positioning services for the unmanned aerial vehicle.

[0008] For the differentiated detection needs of the "sea surface" and "shore surface" of the cargo ship, a dual-mode flight path is designed: a straight flight mode is adopted in the shore base stage, and the distance measurement and image recognition technology of the binocular camera is used to automatically trigger the sea surrounding mode switching when flying over the bow / stern; a dynamic spiral surrounding algorithm is adopted in the sea stage to dynamically adjust the surrounding radius according to the real-time distance measurement data to ensure a safe distance from the ship body. In the whole process, the monocular camera continuously detects obstacles in front, and when an obstacle is found on the planned path, an alternative path is generated based on the obstacle avoidance algorithm to realize real-time obstacle avoidance.

[0009] The system has a path memory function and can record the whole voyage trajectory information. After the task is completed, it automatically executes the original path return. During the return process, the binocular camera continues to collect ship water gauge image data to provide supplementary information for subsequent analysis.

[0010] During the task of taking water gauge image around the ship by the unmanned aerial vehicle, the real-time collected water gauge image is transmitted to the host on the shore in real time by means of wireless network communication technology. The host constructs a model specially used for identifying water gauge characters by means of deep learning algorithm, so as to carry out character recognition operation on the image transmitted by the unmanned aerial vehicle. Once the water gauge characters are successfully recognized, the host will store the image in the hard disk thereof. After the unmanned aerial vehicle completes the flight task, the host will carry out image processing and calculation work on all the images saved during the flight process, and then the actual draft value of the ship water gauge is obtained. The present application constructs a complete ship water gauge intelligent identification system through the collaborative application of multi-sensor fusion perception, intelligent path planning, dynamic obstacle avoidance technology, deep learning and computer vision technology, significantly improves the port ship draft detection efficiency, reduces the labor cost, and provides key technical support for ship intelligent operation and maintenance.

[0011] In order to achieve the above purpose, the technical scheme of the present application is as follows: an unmanned aerial vehicle ship water gauge intelligent identification system based on image recognition, comprising an unmanned aerial vehicle, a camera module, a shore-based navigation module and a shore-based host, the camera module is installed on the unmanned aerial vehicle, the shore-based navigation module is installed on the wharf, and the shore-based host is arranged on the shore. The controller of the unmanned aerial vehicle is installed with an autonomous path planning system, the autonomous path planning system comprises a binocular camera distance measuring module, a ship head and tail identification module, a shore-based path flight planning module, an obstacle avoidance module, a state conversion module, a sea path flight planning module, a memory path and return module and an image transmission module; the shore-based host is installed with a calculation module, and the calculation module comprises an image transmission module, a water gauge character recognition module and a water gauge draft calculation module.

[0012] The controller is connected with the camera module through a data line, and the controller is connected with the shore-based navigation module through Bluetooth.

[0013] Further, the camera module comprises a binocular camera and a monocular camera; the monocular camera is installed at the front of the unmanned aerial vehicle and is used for real-time perception of the front environment; the binocular camera is installed on one side of the unmanned aerial vehicle and is used for real-time measurement of the distance between the unmanned aerial vehicle and the ship body by means of stereo vision technology. The shore-based navigation module is a double-Bluetooth beacon base station, which is used for accurate positioning and flight mode switching of the unmanned aerial vehicle in the port shore area.

[0014] Further, the double-Bluetooth beacon base stations are arranged along the shoreline of the port shore area, and the distance S between the double-Bluetooth beacon base stations is 120-150 m.

[0015] Further, the working method of the unmanned aerial vehicle ship water gauge intelligent identification system based on image recognition comprises the following steps: A, the internal parameter calibration of binocular camera in camera module is carried out, and the external parameter matrix and the internal parameter of camera are calculated respectively.

[0016] B, the unmanned aerial vehicle takes off to the predetermined height, and flies along the shore base flight path planned by the shore base path flight planning module from the starting Bluetooth beacon base station in the shore base navigation module to the second Bluetooth beacon base station.

[0017] C, the target detection of the forward direction of the unmanned aerial vehicle is carried out by the monocular camera in the camera module, if no obstacle is found in the forward direction, step E is turned; otherwise, step D is turned. D, the target detection of the forward direction of the unmanned aerial vehicle is carried out by the monocular camera in the camera module based on the rapid expansion random tree obstacle avoidance algorithm, and the path is re-planned in real time to ensure the safety of flight.

[0018] E, the ship body is photographed by the binocular camera in the camera module, if the ship head or tail feature is recognized by the ship head and tail recognition module or the unmanned aerial vehicle reaches the second Bluetooth beacon base station, step F is turned; otherwise, step C is turned. F, the flight mode is switched by the state conversion module, and the sea path flight planning module starts to work. G, the distance between the unmanned aerial vehicle and the ship body is detected in real time by the binocular camera in the camera module according to the parameters obtained by calibration, which is used to adjust the helical radius of the unmanned aerial vehicle to maintain a safe distance.

[0019] H, if the ship head or tail feature is recognized by the ship head and tail recognition module, step I is turned; otherwise, step G is turned.

[0020] I, in the flight process of the unmanned aerial vehicle, the images photographed are transmitted to the shore base host in real time through the image transmission module, and the actual draft value of the ship is calculated by the character recognition module and the water level calculation module after the images are received by the shore base host.

[0021] J, the flight path is memorized by the unmanned aerial vehicle through the memory path and return module, the full voyage trajectory information is recorded completely, and the unmanned aerial vehicle returns automatically along the original path after the task is completed.

[0022] Further, the binocular camera distance measuring module photographs images by the binocular camera, identifies corresponding feature points in the two images, and calculates the horizontal displacement of the feature points in the two images to determine the parallax value. Once the accurate parallax value is obtained, the parallax is converted into actual distance information by using the geometric relationship of the binocular camera. According to the pinhole imaging model and the triangulation principle, the following formula is used to calculate the distance from the object to the camera: (1) wherein, represents the distance from the object to the camera, is the focal length of the camera, is the baseline length between two cameras, is the parallax value. Through this process, the two-dimensional image information captured by the binocular camera is converted into depth information in three-dimensional space.

[0023] Further, the bow and stern recognition module uses the Canny algorithm to extract the ship contour in the image, accurately identifying the edges of the ship from the complex background. The specific steps are as follows: First, the original image is smoothed by Gaussian filtering, and the following two-dimensional Gaussian function based on standard deviation is used to convolve the image: (2) Second, calculate the amplitude and direction of the image gradient. By applying the Sobel operator, the gradients of the image in the horizontal direction and the vertical direction are calculated respectively. Then, according to the gradient values in these two directions, the gradient amplitude and the direction angle of each pixel point are calculated.

[0024] Finally, non-maximum suppression is performed, which ensures that only the local maximum along the gradient direction is retained as an edge point, and the rest is suppressed. Two thresholds are set: a higher threshold and a lower threshold . Pixels higher than are considered strong edges, while pixels between and are considered weak edges. Pixels below are considered non-edge points. By connecting weak edge points connected to strong edges, complete edge paths are formed, resulting in clear and continuous ship contours.

[0025] Further, in the shore-based flight path planning module, both Bluetooth beacons are equipped with high-precision GPS receiving modules to obtain accurate GPS position information in real time. The two Bluetooth beacons are stably connected through a wireless network, thereby realizing real-time sharing of GPS position information. Through the Bluetooth wireless communication link, the UAV starts from the starting point and flies along the straight-line trajectory pointing to the GPS position of the second beacon.

[0026] Further, the obstacle avoidance module uses a rapid expansion of the random tree algorithm to re-plan the obstacle avoidance path. The obstacle avoidance module is started from the moment when the monocular camera identifies the obstacle. The position of the obstacle identified by the UAV is set as the starting point, and the target point is set as the state transition point, i.e. the second Bluetooth beacon point coordinate. Next, a sample point is randomly generated in the state space. The node closest to the newly selected sample point is found in the existing tree. The following Euclidean distance formula is used to calculate the distance between two points: (3) wherein, and represent the horizontal and vertical coordinates of the first point, and represent the horizontal and vertical coordinates of the second point. This formula determines which node in the current tree is closest to the randomly sampled point. Based on the fixed step strategy, the step size is expanded in the direction of the random point from the nearest node. The expansion rule is as follows: (4) (5) wherein, and are the horizontal and vertical coordinates of the random sample point, and are the horizontal and vertical coordinates of the nearest node in the current tree, is the distance from the nearest node to the random sample point, denotes the fixed step size, which is set to 0.5m-1m, and the step size is fixed each time, and are the horizontal and vertical coordinates of the new node generated by the nearest node according to the fixed step size in the direction of the nearest node to the random sample point. For polygonal obstacles, the ray method is used to verify whether the path generated by the new node and its connecting line intersects with the obstacle boundary. If the new path collides with any obstacle, this path is discarded, and the next iteration is continued. If no collision occurs, the random point is added to the tree. When a path is found from the starting point to the target point, it is considered a feasible path, and the new path replaces the original path.

[0027] Further, the state conversion module continuously monitors the target ship body by the binocular camera during the flight task of the UAV. When the bow or stern features are detected in the images captured by the binocular camera or the UAV reaches the second Bluetooth beacon, it is determined that the UAV has completed the scanning task on the side of the ship body close to the shore, at which time the flight mode conversion is triggered, and the UAV switches from the straight flight mode on the shore to the spiral flight mode on the sea. Then when the bow or stern features are detected again in the images captured by the binocular camera, it is determined that the UAV has completed the entire shooting task, and the UAV switches from the spiral flight mode on the sea to the homing mode.

[0028] Further, the sea flight path planning module automatically triggers the flight mode switching instruction when the UAV flies over the bow / tail identification point, and smoothly converts the UAV from the current flight mode to the spiral flight mode on the sea. In the spiral flight mode, the UAV performs the surrounding flight with the leftmost center point of the ship body in the bow / tail image captured by the binocular camera as the center, which is accurately determined by image processing algorithm and geometric calculation. At the same time, the binocular camera detects the distance between the UAV and the ship body in real time, and dynamically plans the spiral flight radius according to the detected distance, wherein the spiral flight radius calculation formula is as follows: (6) wherein is the basic radius of the spiral flight radius, which is set to 50m-60m, which is half of the length of the ship body, to ensure that the flight radius is sufficient to cover the entire ship body, is the safety distance preset by the algorithm, which is set to 3m-4m, to ensure that the UAV and the ship body always maintain a certain safety interval and effectively avoid collision, is an adjustment variable, which is set to 0.5-0.8 by default, is the distance between the UAV and the ship body measured by the binocular camera, when , the value increases, making the UAV away from the ship body, and keeping the UAV and the ship body at a safe distance, when , the value decreases, making the UAV close to the ship body, to avoid affecting the shooting effect of the binocular camera due to too far shooting distance.

[0029] Further, the memory path and return module, the unmanned aerial vehicle is equipped with a storage device complete record of the whole journey trajectory information, including position, attitude, speed of key parameters, and adopts efficient data storage format to ensure the integrity and readability of data. After the task is completed, the unmanned aerial vehicle starts the autonomous return program, and accurately reproduces the original flight path according to the stored trajectory information. During the return process, the binocular camera continuously collects the ship surface data, supplements the detection of the possible missed area, and integrates and stores the collected data with the data in the flight process, realizes the automatic planning of the complete scanning flight around the ship surface.

[0030] Further, the image transmission module, after completing the network interface IP address configuration between the host on the shore and the unmanned aerial vehicle, sets the host on the shore as the server of TCP communication, and the unmanned aerial vehicle as the client to actively connect to the host, and both sides establish stable TCP connection through wireless network. On this basis, the unmanned aerial vehicle transmits the image data shot by it to the host on the shore in real time and reliably, realizes remote receiving and processing of image information.

[0031] Further, the water gauge character recognition module, in the model training stage, adopts the ship water gauge image shot by artificial to construct the training data set, labels the water gauge scale area and digital characters in the image, and trains the character detection model with high precision recognition ability based on YOLOv10 target detection algorithm. After training, the character detection model is deployed on the shore-based host, and is used for automatic recognition of the water gauge image collected from the unmanned aerial vehicle.

[0032] Further, the water gauge draft calculation module, in the initialization stage, carries out space scale calibration on the image through the reference calibration object with known size, so as to establish the mapping relationship between pixels and actual distance. In the subsequent processing, according to the pixel interval between the recognized character area and the edge of the draft line, and combining the actual length corresponding to the unit pixel, the pixel difference value is converted into the actual draft depth.

[0033] Compared with the prior art, the beneficial effects of the present application are as follows: 1、The present application realizes full-automatic shooting of the water gauge of the large cargo ship through intelligent path planning technology, significantly improves the detection efficiency, and ensures that the ship surface can be effectively detected.

[0034] 2、The autonomous path planning method of the present application reduces the dependence on manual operation and reduces the labor cost; at the same time, the use of unmanned aerial vehicle platform also reduces the large equipment and complex operation required in the traditional detection method, further reduces the material cost.

[0035] 3、The application adopts a multi-sensor fusion scheme, integrates binocular cameras and monocular cameras on a micro unmanned aerial vehicle platform, realizes synchronous acquisition of forward environmental information and lateral depth data of the ship body, and improves the accuracy and comprehensiveness of environmental perception.

[0036] 4、In the whole flight process, the monocular camera continuously performs forward environmental perception, when it is detected that there is an obstacle in the planned path, the system immediately starts the obstacle avoidance algorithm based on deep learning, generates an alternative flight path in milliseconds through multi-target tracking and path re-planning technology, and ensures flight safety.

[0037] 5、The application deploys double Bluetooth beacon base stations in the port shore-based area, constructs a high-precision navigation network, ensures the safety and accuracy of the unmanned aerial vehicle flight, and improves the autonomous navigation ability of the unmanned aerial vehicle in complex environment.

[0038] 6、The application designs a dual-mode flight path algorithm, which realizes flexible switching between shore-based straight flight and sea spiral flight according to the differentiated operation requirements of the cargo ship "reaching the sea surface" and "reaching the shore surface", and improves the adaptability and efficiency of the unmanned aerial vehicle in different operation environments.

[0039] 7、The unmanned aerial vehicle has a flight path memory function, can record complete trajectory information, and can return to the original path autonomously after the task is completed, realizing the whole-process unmanned operation of take-off-operation-return, and improving the continuity and efficiency of the operation.

[0040] 8、The application trains and deploys the character recognition model based on YOLOv10 on the water gauge image shot by the artificial, can effectively deal with the light change, shooting angle difference and scale transformation in complex environment, realize high-precision recognition of water gauge scale number, and further improve the measurement accuracy of ship draft depth in single image, reduce the artificial observation error, and enhance the intelligence and reliability of the system.

[0041] 9、The application realizes deep fusion of multi-sensor collaborative perception, intelligent path planning and real-time obstacle avoidance technology, constructs a TCP wireless communication architecture with the shore-based host as the server and the unmanned aerial vehicle as the client, guarantees stable transmission of image data from the flight platform to the ground system, supports remote real-time image receiving and processing, improves the efficiency and flexibility of the whole water gauge detection process, is suitable for remote ship monitoring scenes such as ports and wharfs, and has good engineering application prospect. BRIEF DESCRIPTION OF DRAWINGS

[0042] Figure 1 The flow chart of the unmanned aerial vehicle of the application automatically planning and shooting water gauge.

[0043] Figure 2This is a schematic diagram showing the installation position of the camera of the present invention on a drone.

[0044] Figure 3 This is a schematic diagram of the installation of the Bluetooth beacon base station according to the present invention.

[0045] Figure 4 This is a flowchart of the obstacle avoidance algorithm of the present invention.

[0046] Figure 5 This is a flowchart illustrating the process of calculating draft based on an image by the shore-based main unit of the present invention.

[0047] Figure 6 This is a diagram illustrating distance conversion. Detailed Implementation

[0048] The invention will now be further described with reference to the accompanying drawings.

[0049] like Figure 2 As shown, a stereo camera and a monocular camera are vertically mounted on the drone. The monocular camera's shooting direction is the drone's forward direction, while the stereo camera's shooting direction is to the left of the drone's forward direction. The stereo camera is then calibrated, including calculating its extrinsic parameter matrix and intrinsic parameters. Zhang Zhengyou's calibration method is used, which involves photographing a checkerboard pattern at different locations and employing a calibration algorithm to solve for the camera parameters. This yields the stereo camera's intrinsic parameter matrix and distortion coefficients.

[0050] In the aforementioned shore-based navigation module, two Bluetooth beacon base stations are first strategically positioned within a designated area of ​​the port, with a distance of S meters between them (120≤S≤150). Their GPS positioning function is then used to achieve real-time data synchronization, thereby constructing a high-precision navigation and positioning system. The binocular camera is mounted on one side of the drone (e.g., Figure 2 As shown), point the binocular camera at the ship and select the Bluetooth beacon base station behind the drone as the drone's takeoff starting point (e.g., Figure 3 As shown), the Bluetooth beacon base station ahead serves as the state transition control node during flight. Figure 1 As shown, when a cargo ship docks within the effective coverage area of ​​the dual base station signal, the UAV can automatically identify and execute the corresponding flight mode switching operation according to the pre-set flight logic rules. In the binocular camera ranging module, images are captured by the binocular camera, corresponding feature points in the two images are identified, and the horizontal displacement of these points in the two images is calculated to determine the disparity value. Once accurate disparity information is obtained, the disparity can be converted into actual distance information using the geometric relationship of the binocular camera. Based on the pinhole imaging model and the principle of triangulation, the distance from the object to the camera is calculated using the following formula: (1) in, Indicates the distance from the object to the camera. It's the camera's focal length. It is the baseline length between the two cameras. Instead, it refers to the parallax value mentioned earlier. This process transforms the two-dimensional image information captured by the binocular camera into depth information in three-dimensional space.

[0051] The aforementioned bow and stern recognition module employs the Canny algorithm to extract the ship's outline from the image, aiming to accurately identify the ship's edges from complex backgrounds. First, to reduce noise interference in the image, a Gaussian filter is used to smooth the original image. This step uses a standard deviation-based algorithm... Two-dimensional Gaussian function: (2) The image is convolved to effectively remove high-frequency noise while preserving important edge information as much as possible. Next, the magnitude and direction of the image gradient are calculated, which is a crucial step in edge detection. This is achieved by applying Sobel... Operators calculate the image in the horizontal direction respectively. and vertical direction The gradient on. Then... Calculate the gradient magnitude of each pixel based on the gradient values ​​in these two directions. and direction angle Then, nonmaximum suppression is performed, a process that ensures suppression occurs only along the gradient direction. Local maxima are preserved as edge points, while the rest are suppressed. Two thresholds are set: a higher threshold and a lower threshold. and a lower threshold Higher than Pixels that are considered strong edges, while those that are between and Pixels between these points are considered weak edges. Below Pixels that are weak are considered non-edge points. By connecting weak edge points that are connected to strong edges, a complete edge path is formed, ultimately resulting in a clear and continuous ship outline.

[0052] In the aforementioned shore-based flight path planning module, both Bluetooth beacons are equipped with high-precision GPS receiver modules, enabling them to acquire accurate GPS location information in real time. The beacons quickly establish a stable connection via wireless communication technology, thereby achieving real-time sharing of GPS information. Within this network, the UAV departs from its starting point and flies along a straight trajectory pointing to the GPS location of the second beacon, such as... Figure 3 As shown, the precise guidance provided by the high-resolution Bluetooth wireless communication link ensures the accuracy of the flight path, enabling the drone's binocular camera to capture images of the ship's surface at close range from the shore.

[0053] As Figure 4 shown in the UAV obstacle avoidance module, the rapid expansion of random tree algorithm is used to re-plan the obstacle avoidance path, and the tree structure is constructed from the UAV position coordinate point when the monocular camera recognizes the obstacle. The tree initially contains only this starting point. The target point is set as the state transition point, i.e. the second Bluetooth beacon point coordinate. Next, a sample point is randomly selected in the state space as a new target direction. In order to improve efficiency, sometimes the target point is directly selected as the sample point with a certain probability, which can speed up the exploration process to the target direction. Find the nearest node to the newly selected sample point in the existing tree. Here, the Euclidean distance formula is used to calculate the distance between two points: (3) wherein, and represent the position coordinates of the two nodes. This formula helps to determine which node in the current tree is closest to the randomly sampled point. Based on the fixed step strategy, a new node is generated by expanding from the nearest node to the sample point in the direction. The expansion rule is as follows: (4) (5) wherein, is the distance from the nearest node to the random sample point, and represents the fixed step size, which is set to 0.5m, ensuring that the new node is always located on the line between the nearest node and the sample point, and the step size of each expansion is fixed. In the algorithm re-planning, it is crucial to determine whether the new node collides with the obstacle. For polygonal obstacles, the ray method can be used to verify whether the new node and its connecting line intersect with the obstacle boundary. If the new node collides with any obstacle, this expansion is abandoned, and the algorithm continues the next iteration. When the new node is close enough to the target point, it is considered that a feasible path has been found.

[0054] In the state transition module, the dual-camera carried by the UAV continuously monitors the target ship body during the flight task. When the ship head and tail features are detected in the images captured by the dual-camera or the UAV reaches the second Bluetooth beacon, the system determines that the UAV has completed the scanning task on the shore side of the ship body, and triggers the flight mode transition, i.e. the UAV switches from the straight flight mode on the shore to the spiral flight mode on the sea. Then when the ship head or tail features are detected again in the images captured by the dual-camera, the system determines that the UAV has completed the entire shooting task, and the UAV switches from the spiral flight mode on the sea to the return mode.

[0055] In the aforementioned maritime flight path planning module, after the UAV flies past the bow / stern identification point, the system automatically triggers a flight mode switching command, and the UAV smoothly transitions from the current flight mode to a maritime spiral orbiting flight mode. In the spiral orbiting flight mode, the UAV orbits around the center point of the leftmost part of the hull in the bow / stern image captured by the binocular camera, as the center of the circle. Figure 3 As shown, the center point is precisely determined through advanced image processing algorithms and geometric calculations, ensuring the accuracy and stability of the orbital trajectory. Simultaneously, the binocular camera detects the distance between the UAV and the ship in real time and dynamically plans the spiral flight radius based on the detected distance. The formula for calculating the spiral flight radius is as follows: (6) in The range is set to 50m, roughly half the length of the ship, to ensure the flight radius is sufficient to cover the entire ship. The preset safety distance for the algorithm is set to 3 meters to ensure that a certain safe distance is always maintained between the drone and the ship, effectively avoiding collisions. To adjust the variable, the default value is set to 0.5. hour, The value increases, causing the drone to move away from the ship's hull and maintain a safe distance between the drone and the ship. hour, The value is reduced to bring the drone closer to the ship's hull, preventing the camera from being affected by the shooting distance being too far.

[0056] In the aforementioned memory path and return-to-home module, the UAV's storage device can completely record the trajectory information of the entire flight, including key parameters such as position, attitude, and speed, and uses an efficient data storage format to ensure data integrity and readability. After the mission is completed, the UAV initiates an autonomous return-to-home procedure, accurately reproducing the original flight path based on the stored trajectory information. During the return process, simultaneously, the binocular camera continuously collects data on the ship's surface, supplementing any potentially missed areas, and integrates and stores the collected data with the data from the flight process, achieving automatically planned and complete scanning flight around the ship's surface.

[0057] In the image transmission module, first, static IP addresses are configured for the onshore host and the unmanned aerial vehicle to ensure that they are in the same local area network; then a TCP server program is deployed on the onshore host end to listen to a specified port for receiving connection requests; a TCP client program is run on the unmanned aerial vehicle end to actively connect to the host IP and the corresponding port, and a stable TCP communication link is established between the two through Wi-Fi or 4G wireless network. On this basis, the unmanned aerial vehicle end collects water gauge images in real time, compresses and encodes the image data, and sends the image data to the onshore host in frames, and the host receives and decodes the image data and performs subsequent identification and processing, thereby completing the remote transmission and analysis of the image.

[0058] In the water gauge character recognition module, a large number of ship water gauge images are obtained by manual shooting, covering typical scenes under different lighting conditions, shooting angles, weather environments and scale changes to improve the generalization ability of the model. Then, the key areas in the images are accurately labeled using a labeling tool, and the labeled objects include water gauge scale lines, digital characters and their bounding box information, forming a structured image dataset. On this basis, a model training process is built based on the YOLOv10 target detection framework, data augmentation strategies such as random cropping, brightness transformation, rotation and flipping are introduced to further improve the robustness of the model, and model hyperparameters such as learning rate, batch size and training rounds are reasonably configured, combined with GPU acceleration to efficiently complete model training. During the training process, the model performance is continuously evaluated by the validation set to ensure that it has high recognition accuracy and recall rate. After training, the optimized model is deployed on the onshore host end to receive and process real-time water gauge images transmitted from the unmanned aerial vehicle. When the system is running, the model can automatically detect and recognize the key characters in the input image, output structured recognition results with position information, and save the recognized character images to the host hard disk, providing accurate data support for the subsequent draft calculation module, realizing the full-process automation from image acquisition, recognition to data analysis.

[0059] In the water gauge draft calculation module, according to the relative position between the recognized water gauge characters and the draft line and the pixel proportion they occupy in the image, the pixel distance and the actual size can be converted to obtain the actual length corresponding to a unit pixel, and the ship draft value can be calculated accordingly. Since a pixel is the smallest division unit in an image, it can be used as the basic unit for distance measurement. According to the standard of the metric water gauge, ordinary water gauge characters such as Figure 6As shown, the height is 10 cm, and the distance between adjacent characters is also 10 cm. The number of pixels occupied by the recognized character height is counted, and the actual distance represented by a single pixel in the image is calculated. Taking the smallest character closest to the waterline and accurately recognized as the reference, the unit pixel distance is converted by the height pixel number, and the pixel difference between the waterline and the outer frame of the character is calculated by combining the coordinate information of the waterline and the character, and then multiplied by the actual distance corresponding to the unit pixel, to obtain the draft value. As shown, Figure 6 Taking the first complete small character above the waterline as the standard, the smallest character closest to the waterline is the number 2, and the actual height is 10 cm. The number of pixels contained in the height of the number 2 is counted, and the actual distance represented by each pixel point is calculated and recorded as The number of pixels contained in the number 2 is and the distance represented by each pixel point in the image is accurately calculated as (7) After analysis, the relative position of the waterline and the character mainly includes two types: one is that the waterline is located between the lower edges of two characters; the other is that the waterline coincides with the lower edge of a character. In the first case, the waterline is located between the lower edges of two characters, and the first accurately recognized small character appearing on the upper end of the waterline is taken as the standard to calculate the actual distance represented by the unit pixel From the waterline, search upwards, and the first accurately recognized large character is recorded as with a vertical coordinate in the image of The first accurately recognized small character is recorded as with a vertical coordinate in the image of The actual distance between the first accurately recognized smallest character and the waterline in the image is calculated by multiplying the difference between the vertical coordinates of the first accurately recognized smallest character and the waterline and the actual distance of the unit pixel , and then the final draft value is calculated as : (8) The second case is that the waterline coincides with the lower edge of a character. In this case, the waterline calculation method is simplified in the first case. We only need to know the first complete large character found from the waterline upwards, and do not need to know the accurately recognized complete large character and the vertical coordinate of the waterline in the image, nor do we need to calculate the actual distance represented by the unit pixel. The specific calculation formula is as follows: (9) According to the above method, the actual draft values of multiple water gauges of a ship can be obtained by automatically and intelligently measuring the water gauges.

[0060] The present application is not limited to the embodiment, any equivalent concept or change within the technical scope disclosed in the present application is included in the protection scope of the present application.

Claims

1. A drone-based intelligent draft gauge identification system for ships, characterized in that: It includes a drone, a camera module, a shore-based navigation module, and a shore-based main unit. The camera module is installed on the drone, the shore-based navigation module is installed on the dock, and the shore-based main unit is configured on the shore. The controller of the UAV is equipped with an autonomous path planning system, which includes a binocular camera ranging module, a bow and stern recognition module, a shore-based path flight planning module, an obstacle avoidance module, a state transition module, a maritime path flight planning module, a path memory and return module, and an image transmission module; the shore-based host is equipped with a computing module, which includes an image transmission module, a draft gauge character recognition module, and a draft gauge calculation module. The controller is connected to the camera module via a data cable and to the shore-based navigation module via Bluetooth.

2. The image recognition-based intelligent water level identification system for unmanned aerial vehicles (UAVs) as described in claim 1, characterized in that: The camera module includes a binocular camera and a monocular camera; the monocular camera is installed at the front of the drone to perceive the forward environment in real time; the binocular camera is installed on one side of the drone to measure the distance between the drone and the ship in real time using stereo vision technology. The shore-based navigation module is a dual Bluetooth beacon base station, used for the UAV to perform precise positioning and flight mode switching in the port shore-based area.

3. The image recognition-based intelligent water level identification system for unmanned aerial vehicles (UAVs) and ships according to claim 1, characterized in that: The dual Bluetooth beacon base stations are arranged along the shoreline of the port's shore base area, and the distance S between the dual Bluetooth beacon base stations is 120-150m.

4. The image recognition-based intelligent water level identification system for unmanned aerial vehicles (UAVs) and ships according to claim 1, characterized in that: The working method of the image recognition-based unmanned aerial vehicle (UAV) ship draft gauge intelligent identification system includes the following steps: A. Perform internal parameter calibration on the stereo camera in the camera module, and calculate the extrinsic parameter matrix and the camera's intrinsic parameters respectively; B. The UAV takes off and reaches the predetermined altitude. The UAV flies along the shore-based flight path planned by the shore-based path planning module from the starting Bluetooth beacon base station in the shore-based navigation module to the second Bluetooth beacon base station. C. Use the monocular camera in the camera module to detect targets in the direction the drone is moving. If no obstacles are found in the direction of movement, proceed to step E. Otherwise, proceed to step D; D. The obstacle avoidance module is based on the fast expanding random tree obstacle avoidance algorithm. It uses a monocular camera in the camera module to perform target detection and path replanning in the direction of the UAV's movement, and generates alternative flight paths in real time to ensure flight safety. E. Take pictures of the ship's hull using the binocular camera in the camera module. If the bow or stern identification module identifies the bow or stern features or the drone arrives at the second Bluetooth beacon base station, proceed to step F; otherwise, proceed to step C. F. The state transition module switches the flight mode, and the sea path flight planning module starts working. G. The binocular camera in the camera module detects the distance between the UAV and the ship in real time based on the parameters obtained from the calibration, and is used to adjust the UAV's spiral orbit radius to maintain a safe distance. H. If the bow or stern recognition module identifies the bow or stern features, proceed to step I; otherwise, proceed to step G. I. During flight, the UAV transmits the captured images to the shore-based host in real time through the image transmission module. After receiving the images, the shore-based host calculates the actual draft of the ship through the character recognition module and the draft calculation module. J. The UAV remembers its flight path through the path memory and return-to-home module, fully recording the trajectory information of the entire flight. After the mission is completed, it returns autonomously along the original path.

5. The image recognition-based intelligent water level identification system for unmanned aerial vehicles (UAVs) as described in claim 1, characterized in that: The binocular camera ranging module captures images using a binocular camera, identifies corresponding feature points in two images, and calculates the horizontal displacement of these feature points in the two images to determine the disparity value. Once an accurate disparity value is obtained, the disparity is converted into actual distance information using the geometric relationship of the binocular camera. Based on the pinhole imaging model and the principle of triangulation, the distance from the object to the camera is calculated using the following formula: (1) in, Indicates the distance from the object to the camera. It's the camera's focal length. It is the baseline length between the two cameras. It is the disparity value; through this process, the two-dimensional image information captured by the binocular camera is converted into depth information in three-dimensional space; The aforementioned bow and stern recognition module uses the Canny algorithm to extract the ship's outline from the image, accurately identifying the ship's edges from a complex background; the specific steps are as follows: First, Gaussian filtering is used to smooth the original image, using the following standard deviation-based method. Use a two-dimensional Gaussian function to convolve the image: (2) Secondly, the magnitude and direction of the image gradient are calculated. By applying the Sobel operator, the gradient in the horizontal direction is calculated separately. and vertical direction The gradient in both directions is calculated; subsequently, the gradient magnitude of each pixel is calculated based on the gradient values ​​in these two directions. and direction angle ; Finally, non-maximum suppression is performed. This process ensures that only local maxima along the gradient direction are preserved as edge points, while the rest are suppressed. Two thresholds are set: a higher threshold and a lower threshold. and a lower threshold Higher than Pixels that are considered strong edges, while those that are between and Pixels between are considered weak edges; below The pixels are considered non-edge points; by connecting the weak edge points that are connected to the strong edge, a complete edge path is formed, and finally a clear and continuous ship outline is obtained. In the aforementioned shore-based flight path planning module, both Bluetooth beacons are equipped with high-precision GPS receiver modules to obtain accurate GPS location information in real time; the two Bluetooth beacons are stably connected through a wireless network, thereby realizing real-time sharing of GPS location information; through the Bluetooth wireless communication link, the UAV starts from the starting point and flies along a straight trajectory pointing to the GPS location of the second beacon; The obstacle avoidance module uses a fast expanding random tree algorithm to replan the obstacle avoidance path. The module starts running when the monocular camera detects an obstacle, setting the location of the obstacle detected by the drone as the starting point and the target point as the state transition point, i.e., the coordinates of the second Bluetooth beacon point. Next, sample points are randomly generated in the state space. The node closest to the newly selected sample point is found in the existing tree. The distance between the two points is then calculated using the following Euclidean distance formula: (3) in, and These represent the x and y coordinates of the first point, respectively. and These represent the x and y coordinates of the second point, respectively. This formula determines which node in the current tree is closest to the random sampling point. Based on a fixed step-size strategy, the step size is expanded from the nearest node towards the random point. The expansion rules are as follows: (4) (5) in, and These are the x and y coordinates of the random sample points, respectively. and These are the x and y coordinates of the nearest node in the current tree, respectively. It is the distance from the nearest node to the random sample point. This indicates a fixed step size, set to 0.5m-1m, where the step size for each expansion is fixed. and These are the nearest nodes based on a fixed step size. Generate the x and y coordinates of a new node by extending the step length direction from the nearest node towards the random sample point; for polygonal obstacles, use the ray method to verify whether the path generated by the newly added node and its connection intersects the obstacle boundary; if the new path collides with any obstacle, the path is discarded and the next iteration continues; if no collision occurs, the random point is added to the tree; when a path to avoid obstacles from the starting point to the target point is found, it is considered a feasible path, and the new path replaces the original path. During the UAV's flight mission, the state transition module continuously monitors the target ship's hull using its binocular camera. When the binocular camera detects bow or stern features in the image or the UAV reaches the second Bluetooth beacon, it determines that the UAV has completed the scanning task on the shore side of the ship. At this point, a flight mode transition is triggered, and the UAV switches from shore-based straight-line flight mode to sea-based spiral circling flight mode. Then, when the binocular camera detects bow or stern features again in the image, it determines that the UAV has completed the entire shooting task, and the UAV switches from sea-based spiral circling flight mode to return-to-home mode. The aforementioned maritime flight path planning module automatically triggers a flight mode switching command after the UAV flies past the bow / stern identification point, smoothly transitioning the UAV from the current flight mode to a maritime spiral orbiting flight mode. In the spiral orbiting flight mode, the UAV orbits around the center point of the leftmost part of the ship's hull in the bow / stern image captured by the binocular camera. This center point is precisely determined through image processing algorithms and geometric calculations. Simultaneously, the binocular camera detects the distance between the UAV and the ship's hull in real time and dynamically plans the spiral flight radius based on the detected distance. The formula for calculating the spiral flight radius is as follows: (6) in The base radius for the spiral flight radius is set at 50m-60m, which is half the length of the hull, to ensure that the flight radius is sufficient to cover the entire hull. The preset safety distance for the algorithm is set to 3m-4m to ensure that a certain safe distance is always maintained between the drone and the ship, effectively avoiding collisions. To adjust the variable, the default setting is 0.5-0.

8. It is the distance between the drone and the ship's hull measured by a binocular camera. hour, The value increases, causing the drone to move away from the ship's hull and maintain a safe distance between the drone and the ship. hour, The value is reduced to bring the drone closer to the ship's hull, thus preventing the binocular camera from being affected by the shooting distance being too far. In the aforementioned memory path and return-to-home module, the UAV's storage device fully records the trajectory information of the entire flight, including key parameters such as position, attitude, and speed, and uses an efficient data storage format to ensure data integrity and readability. After the mission is completed, the UAV initiates an autonomous return-to-home procedure, accurately reproducing the original flight path based on the stored trajectory information. During the return-to-home process, the binocular camera continuously collects data on the ship's surface, supplements the detection of any potentially missed areas, and integrates and stores the collected data with the data from the flight process, achieving an automatically planned and complete scanning flight around the ship's surface.

6. The image recognition-based intelligent water level identification system for unmanned aerial vehicles (UAVs) as described in claim 1, characterized in that: After configuring the network interface IP address between the onshore host and the UAV, the image transmission module sets the onshore host as a TCP communication server and the UAV as a client to actively connect to the host. The two parties establish a stable TCP connection through the wireless network. On this basis, the UAV transmits the image data it captures to the onshore host in real time and reliably, realizing the remote reception and processing of image information. The aforementioned draft gauge character recognition module uses manually captured ship draft gauge images to construct a training dataset during the model training phase. By annotating the draft gauge scale areas and numerical characters in the images, a character detection model with high-precision recognition capabilities is trained based on the YOLOv10 object detection algorithm. After training, the character detection model is deployed on a shore-based host for automatic recognition of draft gauge images received from drones. The water gauge draft calculation module performs spatial scale calibration on the image using a reference calibration object of known size during the initialization phase, thereby establishing a mapping relationship between pixels and actual distances. In subsequent processing, based on the pixel spacing between the identified character area and the edge of the waterline, combined with the actual length corresponding to each unit pixel, the pixel difference is converted into the actual draft depth.

Images