Ship water height measurement method, device, electronic equipment and system

Abstract

The present application relates to a kind of ship water height measurement method, device, electronic equipment and system, belong to computer vision technical field, wherein the ship water height measurement method includes obtaining the ship image of target ship after pretreatment;Through ship detection model, the ship image after pretreatment is detected, and ship detection frame is obtained;Through the reconstruction image corresponding to ship detection frame is handled to obtain ship edge prior information by nesting edge detection algorithm, and the motion information of target ship is calculated by optical flow, and accurate waterline position and ship accurate boundary are determined in combination with ship edge prior information and motion information;The camera parameter and camera shooting distance of the target camera are obtained;Based on the camera parameter, the camera shooting distance, the accurate waterline position and the ship accurate boundary, the water height of ship is calculated.The full-process automation processing is realized, and manual intervention is not needed.In the guarantee of high accuracy, it has strong real-time processing capacity, can satisfy the real-time requirement of channel monitoring and bridge anti-collision system.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, and in particular to a method, apparatus, electronic device and system for measuring the water height of a ship. Background Technology

[0002] With the rapid development of the water transportation industry, the continuous improvement of waterway standards, and the increasingly obvious trend towards larger ships, ship tonnage is constantly increasing, and traffic density is constantly rising. Ship-bridge collisions are frequent, seriously threatening the safety of people and property in bridge-adjacent waters. The causes of ship-bridge collisions are numerous, mainly including human error, ship mechanical failure, driver fatigue, navigation equipment malfunction, and ship yaw. However, collisions caused by ships exceeding their height limits are particularly serious, easily leading to bridge collapse and resulting in enormous casualties and economic losses.

[0003] Traditional ship height measurement technology often uses lasers to limit the height of passing vessels. This involves emitting a laser beam, receiving the reflected beam, and measuring the time difference between the round trip to calculate the ship's height and distance. However, high-quality laser equipment is expensive, the system is complex, and it requires extremely high data processing and storage capabilities, necessitating powerful computing resources for real-time processing and analysis. Therefore, its practical application is very limited. Meanwhile, complex aquatic environments such as heavy fog, strong light reflection, wave fluctuations, and low visibility pose significant challenges to the practical application of traditional ship detection technologies. In these harsh environments, the visibility of ship targets is significantly reduced, the ship's outline is blurred, and the accuracy of traditional detection technologies drops sharply. In particular, ship height measurement is affected by errors caused by changes in ship attitude, measurement errors due to perspective projection, and uncertainties at the lower edge where the ship's upper edge meets the water surface. Traditional methods are insufficient to meet the practical requirements of high-precision detection. More critically, existing bridge collision warning systems generally suffer from poor adaptability to complex environments, delayed response, and high false alarm rates, failing to issue effective warnings in advance when ships approach bridges, resulting in delayed preventative measures.

[0004] In summary, current ship height measurement technology still has significant shortcomings in terms of accuracy, adaptability, and real-time performance. Summary of the Invention

[0005] In view of this, it is necessary to provide a method, device, electronic equipment and system for measuring the water height of ships, in order to solve the problem that the current ship water height measurement technology still has obvious deficiencies in accuracy, adaptability and real-time performance.

[0006] To address the aforementioned problems, in a first aspect, the present invention provides a method for measuring the water height of a ship, comprising: Acquire a preprocessed image of the target vessel, the image being captured by the target camera; The preprocessed ship image is detected using a ship detection model to obtain a ship detection bounding box; The reconstructed image corresponding to the ship detection box is processed by a nested edge detection algorithm to obtain the ship edge prior information, and the motion information of the target ship is calculated by optical flow. The precise waterline position and the precise boundary of the ship are determined by combining the ship edge prior information and the motion information. Obtain the camera parameters and camera shooting distance of the target camera, wherein the camera shooting distance is the measured distance between the target ship and the target camera; The ship's height above water is calculated based on the camera parameters, the camera shooting distance, the precise waterline position, and the ship's precise boundary.

[0007] In one possible implementation, before acquiring the preprocessed image of the target vessel, the process further includes: The real-time image of the target ship is dehazed using a dark channel prior dehazing algorithm to obtain a dehazed image. The dehazed image is then subjected to contrast enhancement processing to obtain an enhanced image; The enhanced image is then subjected to noise reduction processing to obtain a denoised image; The denoised image is sharpened using an anti-sharpening mask algorithm to obtain a preprocessed ship image.

[0008] In one possible implementation, before detecting the preprocessed ship image using a ship detection model to obtain the ship detection box, the method further includes: Acquire training ship images, and perform label annotation and dataset partitioning on the training ship images to obtain training data; The training data is scaled and normalized to obtain the training dataset; The ship detection model is obtained by training the preset lightweight model using the training dataset through forward propagation calculation.

[0009] In one possible implementation, the step of processing the reconstructed image corresponding to the ship detection box using a nested edge detection algorithm to obtain ship edge prior information, calculating the motion information of the target ship using optical flow, and combining the ship edge prior information and the motion information to determine the precise waterline position and the precise ship boundary includes: The region of interest is extracted from the ship detection frame to obtain the region of interest; Super-resolution reconstruction is performed on the region of interest to obtain a reconstructed image; The reconstructed image is processed by a nested edge detection algorithm to obtain prior information about the ship's edges; The motion information of the target ship is calculated using the dense optical flow method and coefficient optical flow method of the pyramid. Based on the prior information about the ship's edge and the motion information, the precise waterline positioning and precise boundary of the target ship are calculated.

[0010] In one possible implementation, calculating the ship's above-water height based on the camera parameters, the camera shooting distance, the precise waterline position, and the ship's precise boundary includes: Based on the precise waterline position and the precise boundary of the ship, a coordinate system transformation is performed to obtain the boundary information of the pixel coordinate system. The coordinate system transformation process is to map the precise boundary information of the ship from the world coordinate system to the camera coordinate system, then to the image coordinate system through perspective projection, and finally to the pixel coordinate system through affine transformation. The ship's water height is calculated based on the boundary information, the camera parameters, and the camera shooting distance.

[0011] In one possible implementation, after calculating the ship's water height based on the camera parameters, the camera shooting distance, the precise waterline position, and the ship's precise boundary, the method further includes: The ship edge detection error is calculated based on the precise ship boundary information. The ship's attitude error and perspective error are calculated based on the current measurement parameters. The ship's water height is corrected based on the ship's edge detection error, the ship's attitude error, and the ship's perspective error.

[0012] In one possible implementation, the calculation of the ship attitude error and ship perspective error based on the current measurement parameters includes: Ship feature points are extracted based on the ship detection frame; The ship feature points are then matched with a pre-acquired reference image using a brute-force method to obtain the feature point matching results. Calculate the rotation matrix and translation vector based on the feature point matching results; The ship's attitude error and perspective error are calculated based on the rotation matrix and the translation vector.

[0013] Secondly, embodiments of this application also provide a vessel water height measurement device, comprising: a preprocessing module, a detection module, an optimization module, an acquisition module, and a calculation module; the preprocessing module is used to acquire a preprocessed image of the target vessel, the image being captured by a target camera; the detection module is used to detect the preprocessed image of the vessel using a vessel detection model to obtain a vessel detection box; the optimization module is used to process the reconstructed image corresponding to the vessel detection box using a nested edge detection algorithm to obtain prior information about the vessel's edges, and to calculate the motion information of the target vessel using optical flow, combining the prior information about the vessel's edges and the motion information to determine the precise waterline position and the precise boundary of the vessel; the acquisition module is used to acquire the camera parameters and camera shooting distance of the target camera, the camera shooting distance being the measured distance between the target vessel and the target camera; the calculation module is used to calculate the vessel water height based on the camera parameters, the camera shooting distance, the precise waterline position, and the precise boundary of the vessel.

[0014] Thirdly, the present invention also provides an electronic device, including a memory and a processor, wherein, The memory is used to store programs; The processor, coupled to the memory, is used to execute the program stored in the memory to implement the steps in the ship water height measurement method described in any of the above implementations.

[0015] Fourthly, the present invention also provides a ship water height measurement system, which includes the electronic equipment, data acquisition equipment and result output equipment as described above. The data acquisition equipment and the result output equipment are respectively connected to the electronic equipment. The data acquisition equipment is used to acquire real-time images of the target ship, and the result output equipment is used to output and display the water height of the target ship.

[0016] Fifthly, the present invention also provides a computer-readable storage medium for storing a computer-readable program or instructions, which, when executed by a processor, can implement the steps in the ship water height measurement method described in any of the above implementations.

[0017] The beneficial effects of this invention are as follows: The ship water height measurement method provided by this invention first acquires real-time images of the ship, then preprocesses the real-time images, and then detects the preprocessed images to obtain ship detection boxes. Next, edge optimization is performed using nested edge detection algorithms and optical flow calculation to obtain accurate boundary information. Finally, accurate ship water height is calculated based on the accurate boundary information. This achieves fully automated processing from image acquisition, target detection, waterline recognition to height calculation, requiring no manual intervention. It is suitable for scenarios such as waterway monitoring, port management, and bridge early warning, significantly reducing labor costs and operational complexity. Based on deep learning target detection and efficient optical flow calculation and feature matching algorithms, it possesses strong real-time processing capabilities while ensuring high accuracy, meeting the real-time requirements of waterway monitoring and bridge collision avoidance systems. Integrating image enhancement technology, deep learning target detection, and super-resolution reconstruction, it can adapt to complex water environments such as heavy fog, strong light, low illumination, and wave interference, and is suitable for height measurement of different weather conditions, distances, and ship types. Attached Figure Description

[0018] Figure 1 A flowchart illustrating the steps of a method for measuring the water height of a ship according to an embodiment of this application; Figure 2 A schematic diagram of the system architecture of a ship water height measurement method provided in an embodiment of this application; Figure 3 A schematic diagram of the detection frame for measuring the water height of a ship, provided in an embodiment of this application; Figure 4 A schematic diagram of ship height profile detection using a comparative Mask RCNN method for ship height measurement provided in an embodiment of this application; Figure 5 Another flowchart of the method for measuring the water height of a ship provided in an embodiment of this application; Figure 6 A test diagram showing the error correction of the ship's detected height error in a ship's water height measurement method provided in an embodiment of this application; Figure 7 A functional block diagram of a ship water height measuring device provided in an embodiment of this application; Figure 8 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0019] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0020] In the description of the embodiments of the present invention, unless otherwise stated, "multiple" means two or more. "And / or" describes the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can represent three situations: A exists alone, A and B exist simultaneously, and B exists alone.

[0021] The terms "first," "second," etc., used in the embodiments of this invention are for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Therefore, a technical feature defined with "first" or "second" may explicitly or implicitly include at least one of that feature.

[0022] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of the invention. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0023] This invention provides a method, apparatus, electronic device, and system for measuring the water height of a ship, which will be described below.

[0024] Figure 1 This is a schematic flowchart of an embodiment of the ship's water height measurement method provided by the present invention. The execution entity of the ship's water height measurement method can be a computer, a portable smart device, or a cloud server, etc., and this embodiment does not limit it in this way. Figure 1 As shown, methods for measuring the water height of ships include: S101. Obtain a preprocessed image of the target vessel, wherein the image is captured by the target camera.

[0025] It should be noted that this invention aims to overcome the many shortcomings of existing ship surface height measurement technologies in complex aquatic environments, and to solve key technical problems in terms of accuracy, robustness, adaptability, and real-time performance. Specifically, it includes the following aspects: First, under adverse weather conditions (such as heavy fog, strong light reflection, low light, rain, and snow) and complex aquatic environments (such as wave interference, water surface reflection, and cluttered backgrounds), traditional visual detection methods struggle to accurately extract ship targets. In particular, ship outlines are prone to blurring, breakage, or false detections, severely impacting the accuracy of subsequent height measurements. This invention significantly improves the recognizability and outline extraction accuracy of ship targets in complex environments by introducing image enhancement technology, edge prior modeling, and a target detection model trained on a large number of samples.

[0026] Second, existing methods for measuring ship height generally neglect the combined effects of multiple error factors, including measurement deviations caused by changes in ship motion and attitude, geometric distortions due to camera perspective projection, and inaccurate positioning of the lower edge at the junction of the ship's upper edge and the water surface. If these errors are not effectively corrected, they will directly lead to distorted height measurement results, making it difficult to meet the requirements of actual navigation safety early warning. This invention, by constructing a multi-error correction mechanism and combining techniques such as optical flow, perspective transformation correction, and super-resolution reconstruction, can effectively eliminate the above-mentioned errors, ensuring the accuracy and stability of height measurement.

[0027] Third, existing bridge collision warning systems mostly rely on data from a single sensor, resulting in slow response to environmental changes, frequent false alarms and missed alarms, and difficulty in providing timely and reliable warning information when ships approach bridges. This invention, by integrating multi-source information such as visual detection and laser ranging, combined with high-precision height measurement and error correction, constructs a highly reliable and real-time ship height measurement and warning technology solution, providing strong technical support for bridge navigation safety.

[0028] In summary, the core technical problem to be solved by this invention is: how to achieve accurate, stable, and real-time measurement of the water level of ships in complex aquatic environments, overcoming the shortcomings of traditional methods in terms of equipment cost, environmental adaptability, and error control, thereby providing reliable technical support for ship navigation safety and bridge collision warning.

[0029] It should be understood that, compared with the prior art, the present invention has the following beneficial effects: 1. High Measurement Accuracy: This invention significantly improves the measurement accuracy of ship altitude by introducing a multi-source error comprehensive correction mechanism, including item-by-item correction of ship edge detection error, attitude change error, and perspective projection error. Daytime field tests showed that, within a distance range of 0.5-6.5 km, the corrected ship detection error was reduced by approximately 40.175%; nighttime field tests showed that, within a distance range of 0.5-4.5 km, the corrected ship detection error was reduced by approximately 20.15%, meeting the high-precision measurement requirements for bridge navigation safety early warning.

[0030] 2. High degree of automation: This invention realizes fully automated processing from image acquisition, target detection, waterline recognition, height calculation to error correction, without the need for manual intervention. It is applicable to scenarios such as waterway monitoring, port management, and bridge early warning, and significantly reduces labor costs and operational complexity.

[0031] 3. Wide range of applications: This invention integrates image enhancement technology, deep learning target detection, super-resolution reconstruction and multimodal error correction methods, and can adapt to complex water environments such as heavy fog, strong light, low illumination and wave interference. It is suitable for altitude measurement of different weather conditions, different distances and different types of ships.

[0032] 4. Good real-time performance: Based on deep learning target detection and efficient optical flow calculation and feature matching algorithms, this invention has strong real-time processing capabilities while ensuring high accuracy, which can meet the real-time requirements of waterway monitoring and bridge collision avoidance systems.

[0033] 5. High scalability: The present invention adopts a modular design, and each step can be optimized or replaced independently, which facilitates the future integration of more sensor data such as radar, infrared imaging, etc., or the introduction of more advanced deep learning models, and has good room for functional expansion and performance optimization.

[0034] In specific implementations, such as Figure 2 The following is a complete implementation flow of this embodiment. First, image acquisition and preprocessing are performed: images or video streams containing ships are acquired through cameras or monitoring equipment, and preprocessing operations such as dehazing, noise reduction, and sharpening are performed on the acquired images. Second, ship target detection is performed: using a detection model trained on a large number of ship samples, ship target detection is performed on the preprocessed images to obtain the ship's detection bounding box. Then, ship waterline detection and edge optimization are performed: optical flow is used to refine the ship's edges, extracting the top edge of the ship and the waterline edge to determine the ship's precise boundary. Next, parameter acquisition is performed: camera parameters (focal length, field of view, gimbal angle) and laser ranging distance are acquired. Finally, height calculation is performed: based on the principle of monocular vision ranging, combined with the acquired camera parameters and laser ranging distance, the ship's height above water is calculated. After height calculation, error correction is added: the calculated ship height is corrected for ship edge detection errors, attitude errors, perspective errors, etc., to obtain a corrected, high-precision height value.

[0035] It should be noted that, firstly, real-time images of the target vessel are acquired, and then a series of preprocessing steps are performed to obtain a preprocessed image.

[0036] In some embodiments, before step S101, the method further includes: performing dehazing processing on the real-time image of the target ship using a dark channel prior dehazing algorithm to obtain a dehazed image; performing contrast enhancement processing on the dehazed image to obtain an enhanced image; performing noise reduction processing on the enhanced image to obtain a denoised image; and performing sharpening processing on the denoised image using an anti-sharpening mask algorithm to obtain a preprocessed ship image.

[0037] It should be understood that image acquisition should begin first: cameras or surveillance cameras should be deployed on bridges, docks, or surveillance vessels to ensure that the camera's field of view covers the area of ​​water to be surveyed. Raw images or continuous video frames containing the vessel should be acquired. In the specific implementation, dehazing is then performed: when the weather conditions are fog or low visibility, a dehazing algorithm based on dark channel prior is used to process the image.

[0038] First, the input haze image Extract the gray values ​​of the R, G, and B channels, calculate the minimum value of each channel within a local window, and generate the dark channel image. The formula is:

[0039] In the formula: A 15x15 local window; Let y be the grayscale value of pixel y in channel c; It is the minimum value operator.

[0040] Selecting the top 0.1% of the brightest pixels in the dark channel image, the corresponding average value of the three channels of the original image is the global atmospheric light value. A This avoids interference from sky intensity and improves estimation accuracy.

[0041] Deriving transmittance from atmospheric light values Introducing weighting coefficients w To preserve visual depth, the formula is:

[0042] Where: weighting coefficient , This represents the c-channel component of atmospheric light.

[0043] Reconstructing a fog-free image based on an atmospheric scattering model and setting a minimum transmittance threshold. To prevent pixel overflow, the formula is:

[0044] In the formula: To restore the grayscale value of the c channel of the image; This is the maximum value operator.

[0045] It should be noted that a contrast enhancement process is then performed: to address the issue of insufficient contrast in the dehazed image, the Limiting Contrast Adaptive Histogram Equalization (CLAHE) algorithm is used to enhance the grayscale difference between the ship and the water surface, avoid excessive noise amplification, and highlight the target outline.

[0046] First, the dehazed image is divided into 8×8 / 16×16 non-overlapping sub-blocks, and histogram equalization is performed independently on each sub-region to achieve adaptive enhancement of local contrast.

[0047] Next, a contrast limit threshold of 2.0~3.0 was set, and histogram peaks were cropped to suppress noise. Bilinear interpolation was used to eliminate sub-block boundary artifacts, ensuring smooth image transitions and improving the distinction between the target and the background.

[0048] Specifically, noise reduction processing is also required: For noise introduced during image acquisition and transmission, filtering algorithms are adapted according to the noise type, balancing noise reduction effectiveness with detail preservation. The specific solution is as follows: For Gaussian noise, Gaussian filtering is used to suppress Gaussian white noise. The kernel function follows a two-dimensional Gaussian distribution, and the formula is:

[0049] Core size selected: 3×3 / 5×5, standard deviation It adapts to the noise level.

[0050] For salt-and-pepper noise, median filtering is used to remove impulse noise and preserve edge details. The formula is:

[0051] In the formula: W For a 3×3 / 5×5 filter window, It is a median operator.

[0052] For mixed noise, the Non-Local Means (NLM) algorithm is used, which utilizes image self-similarity weighted averaging for noise reduction. The core formula is:

[0053] In the formula: is the output grayscale value of pixel i after denoising using the NLM algorithm; These are the global normalization coefficients; i,j These represent the target pixels to be denoised and the global contrast pixels in the image that participate in the weighting, respectively. I It is the set of all pixels in the entire image; For pixels i and j The similarity weight between the two pixels is determined by the gray-level similarity of the corresponding neighborhood blocks of the two pixels. The higher the similarity, the larger the weight value. is the initial gray value of pixel j in the original noisy image.

[0054] Specifically, the core parameters of the algorithm are precisely set to adapt to the ship image scene: a fixed search window size of 21×21 is used to define the search range of globally similar pixels, balancing denoising accuracy and computational efficiency; the similar block size is 7×7, used to measure the local structural similarity between pixels; and the filter intensity parameter... The weight attenuation is controlled; smaller values ​​preserve more detail, while larger values ​​result in stronger noise reduction. For water surface monitoring scenarios with high real-time requirements, a fast bilateral filter can be used, significantly improving algorithm speed while maintaining basic noise reduction performance, thus adapting to real-time image processing needs.

[0055] Finally, sharpening is performed: the Unsharp Mask algorithm is used to enhance high-frequency components, highlight the edges of the ship, and avoid the ringing effect.

[0056] First, a blurred image is generated using Gaussian filtering. Then, a sharpening mask is constructed and superimposed onto the original image. The formula is as follows:

[0057] Where: sharpening coefficient It balances sharpening intensity with image naturalness. It uses the Laplacian or Sobel operator to extract image gradients, helping to enhance ship edge contours and controlling the sharpening amplitude to avoid artifact distortion.

[0058] S102. The preprocessed ship image is detected using a ship detection model to obtain a ship detection frame.

[0059] It should be noted that, as Figure 3 The diagram shows the detection box. After obtaining the preprocessed image, the target is detected by a pre-trained ship detection model to obtain the ship detection box.

[0060] In some embodiments, before step S102, the method further includes: acquiring training ship images, labeling and dividing the training ship images to obtain training data; scaling and normalizing the training data to obtain a training dataset; and training a preset lightweight model by forward propagation calculation based on the training dataset to obtain a ship detection model.

[0061] It should be understood that the dataset preparation involves using images collected in the previous A1 step for model training, collecting no fewer than 7,000 images (including visible light and infrared), and manually labeling them using labelimg. The dataset is then divided as follows: the training set contains no fewer than 5,600 images, the validation set contains no fewer than 700 images, and the test set contains no fewer than 700 images.

[0062] In the actual implementation, preprocessing is performed: the preprocessed image is scaled to the input size required by the model (usually 640×640 pixels), and the pixel values ​​are normalized.

[0063]

[0064] in: These are the normalized pixel values ​​of the image; 255 represents the pixel value of the scaled image; 255 represents the maximum grayscale value of the 8-bit image.

[0065] It should be understood that training the ship detection model involves using preprocessed images to train the lightweight YOLO11n model, with forward propagation computation performed during training. The model output consists of three parts: bounding box coordinates (x, y, w, h), confidence score, and class probability. .

[0066] YOLO11 uses an anchor box mechanism, where each anchor box predicts the position offset and size scaling of the bounding box:

[0067] in: The coordinates, width, and height of the prediction box center; Output the offset for the model; The coordinates of the top left corner of the grid; Preset the width and height of the anchor frame; This is the Sigmoid activation function.

[0068] By employing the CIoU loss function and comprehensively considering factors such as overlap, center point distance, and aspect ratio consistency, the boundary regression becomes more accurate.

[0069] in: This represents the border loss value; IoU The intersection-union (IoU) ratio of the predicted bounding box to the ground truth bounding box; The square of the Euclidean distance from the center point; c The length of the diagonal of the minimum bounding rectangle; This is the balance coefficient; v The aspect ratio consistency factor; These are the predicted bounding boxes and the ground truth bounding boxes.

[0070] The degree of overlap between the predicted bounding box and the ground truth bounding box is calculated to evaluate localization accuracy and for non-maximum suppression filtering.

[0071] Where: Area represents the region's area; These are the intersection and union operations, respectively.

[0072] Cross-entropy loss is used to measure the prediction error of target confidence, constraining the model to accurately determine whether the target exists.

[0073] In the formula: For confidence loss; True confidence level label; Calculate the confidence level for model predictions.

[0074] During training, the non-maximum suppression (NMS) algorithm is used to sort the boxes by confidence level, remove redundant boxes with IoU > 0.45, and retain the best detection boxes.

[0075] Specifically, the results are then filtered: Detection results are filtered based on a preset confidence threshold (usually set to 0.25-0.35), removing detection boxes with confidence levels below the threshold. Then, false positives with excessively small areas are filtered out based on a minimum area threshold (usually set to 100-500 pixels). For each ship, the detection box with the highest confidence level is retained as the final detection result.

[0076] Finally, detection box extraction is performed: the detection box information for each ship is extracted from the model output, including the coordinates of the top-left corner of the bounding box. lower right corner coordinates The system includes a detection category ID and a confidence score. The ship category ID is determined based on the annotations used during training; for example, "0" can represent a ship. The pixel height of the ship in the image is calculated based on the detection bounding box.

[0077] in: This represents the pixel height of the ship.

[0078] S103. The ship detection box is edge-optimized by a nested edge detection algorithm combined with optical flow calculation to obtain precise boundary information, which includes the precise boundary of the ship and the precise positioning of the waterline; wherein, the process of edge optimization to obtain the precise boundary information includes: processing the reconstructed image corresponding to the ship detection box by a nested edge detection algorithm to obtain the ship edge prior information, and calculating the motion information of the target ship by optical flow calculation, and combining the ship edge prior information and the motion information to determine the precise boundary information.

[0079] It should be understood that by using nested edge detection algorithms and optical flow calculations to process the ship detection box, mainly for edge optimization, the precise boundary information of the target ship can be extracted.

[0080] In some embodiments, step S103 includes: extracting the region of interest from the ship detection box to obtain the region of interest; performing super-resolution reconstruction on the region of interest to obtain a reconstructed image; processing the reconstructed image using a nested edge detection algorithm to obtain ship edge prior information; calculating the motion information of the target ship using the dense optical flow method of the pyramid and the coefficient optical flow method; and calculating the precise waterline positioning and precise ship boundary of the target ship based on the ship edge prior information and the motion information.

[0081] In the specific implementation, the region of interest (ROI) is extracted by performing subsequent processing within the area defined by the detection bounding box. The ROI is determined by extending outwards by a certain number of pixels (typically 10-20 pixels) from the boundary of the ship's detection bounding box. This ROI includes the ship's hull and part of the water surface, providing sufficient contextual information for subsequent edge detection and waterline extraction.

[0082]

[0083] in: These are the coordinates of the top left and bottom right corners of the detection box; To expand the number of pixels.

[0084] It should be noted that ROI super-resolution reconstruction involves super-resolution reconstruction of the extracted region of interest (ROI) to further improve the quality of image details and lay the foundation for high-precision edge detection.

[0085] Specifically, the attention-based residual network super-resolution method includes the following steps: 1. Feature Extraction: Shallow features of the ROI image are extracted using a 3×3 convolution kernel to generate an initial feature map.

[0086] in: This is the initial shallow feature map; This is a convolution operation; Image of the region of interest; The weights and biases of the convolution kernel; It is a linear rectification activation function.

[0087] 2. Residual Attention Block Processing: The feature map is input into the residual attention block for deep feature extraction. The residual attention block consists of multiple residual units and attention modules, where the residual units employ a skip connection structure to accelerate network convergence.

[0088] in: The output characteristics of the residual unit; Input features for the residual blocks; These represent the kernel weights and biases.

[0089] Specifically, the attention module includes channel attention and spatial attention mechanisms. The channel attention mechanism adaptively adjusts the weights of different feature channels through global pooling and fully connected layers, enabling the network to focus on important feature regions such as ship edges and textures.

[0090] in: Channel attention weight map; is the Sigmoid activation function; MLP is a multi-layer fully connected layer; AvgPool is the average pooling operation; MaxPool is the max pooling operation.

[0091] The spatial attention mechanism generates a spatial weight map through convolutional layers and a sigmoid function, focusing on edges and detail regions in the image.

[0092] in: Spatial attention weight map; This is a 7×7 convolution operation; This involves stitching the feature maps together along the channel dimension.

[0093] Finally, by multiplying the input features, channel attention, and spatial attention element by element, we can obtain the enhanced feature map:

[0094] in: For attention-enhanced feature maps; This is an element-wise multiplication operation. 3. Upsampling layer: Upsampling convolution is performed on the attention enhancement features to improve the feature map resolution.

[0095] in: The output features are from upsampled convolutions; The weights and biases of the upsampled convolution kernel. Low-resolution feature maps are transformed into high-resolution images through subpixel convolution (ESPCN) reconstruction:

[0096] in: To reconstruct images for super-resolution; For the restructuring of ESPCN.

[0097] In the specific implementation, the prior information of the ship outline is obtained by using a nested edge detection algorithm (HED) on the ROI after super-resolution reconstruction to obtain the prior information of the ship's edge.

[0098] The steps for optical flow calculation (video stream) are as follows: To improve the positioning accuracy of ship edges and waterlines in dynamic scenes, for consecutive video frames, the Farneback dense optical flow method based on pyramids or the Lucas-Kanade sparse optical flow method are used to calculate the ship's motion information. The optical flow field reflects the motion vectors of pixels between adjacent frames and can be effectively used for temporal constraints on waterline positions.

[0099] For pixels in an image Its optical flow vector Assuming constant grayscale, its basic constraint equations are:

[0100] in: These are the spatial gradients of the image grayscale in the x and y directions, respectively; This represents the change in grayscale over time.

[0101] Farneback optical flow is based on polynomial expansion, assuming that the image signal can be approximated as a quadratic polynomial in the local neighborhood:

[0102] The dense optical flow field is obtained by solving for the optimal matching of polynomial coefficients between two frames of images. This method sets the pyramid layer number to 3, the pyramid scaling factor to 0.5, the number of iterations to 3, and the window size to 15×15 pixels.

[0103] The Lucas-Kanade optical flow method uses weighted least squares to solve for the optical flow vector within a local window.

[0104] Where w represents the Gaussian window weight. This method employs a pyramid structure, setting the window size to 21×21 pixels and the pyramid to 3 layers to enhance robustness to large displacement motions.

[0105] The optical flow vector at the edge of the ship is extracted from the optical flow field. The continuity of the optical flow amplitude and direction is analyzed for subsequent precise waterline positioning.

[0106] Finally, waterline detection is performed: the waterline is a key reference for measuring the ship's waterline height. Based on edge detection and optical flow information, the precise positioning of the waterline is achieved through the following steps.

[0107] Based on the lower edge position of the ship detection frame, the region of interest along the waterline is extracted to avoid computational redundancy in a global search. Let the coordinates of the ship detection frame be... The waterline ROI region is then defined as:

[0108] in, This is the starting offset (usually 10 pixels). The height of the region (usually 20-30 pixels).

[0109] Next, a Hough Transform is performed on the edge image within this ROI region to detect any possible horizontal lines. Points in the edge image are then... Mapping to parameter space :

[0110] The parameter space is discretized and voted on, and the cumulative values ​​are statistically analyzed to determine candidate lines. Based on the characteristics of ship imaging, the waterline should be approximately horizontal; therefore, lines meeting these criteria are selected. (like (a straight line).

[0111] For the detected candidate line set A weighted fit is performed based on the line length, continuity, and relative position to the detection box. Let the waterline position be... The weighted median or least squares method is used for estimation:

[0112] in, Let be the ordinate of the candidate line. The weighting coefficients are determined by the line length and edge strength.

[0113] By incorporating optical flow information, temporal consistency constraints are applied to the waterline position. Let the waterline position in the current frame be... Based on the waterline position of the previous frame and the vertical motion component of optical flow field estimation Constructing time series prediction:

[0114] The final waterline position is smoothed using Kalman filtering or weighted averaging to eliminate noise and jitter from single-frame detection.

[0115] in, The confidence weight is typically set between 0.6 and 0.8.

[0116] S104. Obtain the camera parameters and camera shooting distance of the target camera, wherein the camera shooting distance is the measured distance between the target ship and the target camera.

[0117] It should be understood that camera intrinsic parameters are obtained by reading the camera calibration file or through Zhang Zhengyou's calibration method, resulting in the camera intrinsic parameter matrix K and distortion coefficients. The intrinsic parameter matrix K is expressed as:

[0118] in, and Let x be the pixel focal length in the x and y directions. and The coordinates of the optical center (principal point) in the image coordinate system.

[0119] Secondly, camera extrinsic parameters need to be acquired: This involves obtaining the external parameters of the gimbal or camera, including the gimbal's horizontal and vertical angles. These parameters can be acquired in real-time through the gimbal control system, and the extrinsic parameter matrix can be represented as:

[0120] in: It is a rotation matrix; It is a translation vector.

[0121] Finally, the current measurement parameters also include laser ranging distance acquisition: obtaining the distance D from the target vessel to the camera using a laser rangefinder.

[0122] S105. Calculate the ship's water height based on the camera parameters, the camera shooting distance, the precise waterline position, and the ship's precise boundary.

[0123] It should be noted that after obtaining the precise boundary information of the vessel, the final water height is calculated by combining it with the current measurement parameters.

[0124] In some embodiments, step S104 includes: performing coordinate system transformation based on the precise waterline position and the precise boundary of the ship to obtain boundary information of the pixel coordinate system. The coordinate system transformation process is to map the precise boundary information of the ship from the world coordinate system to the camera coordinate system, then to the image coordinate system through perspective projection, and finally to the pixel coordinate system through affine transformation; and calculating the ship's water height based on the boundary information, the camera parameters, and the camera shooting distance.

[0125] It should be understood that the coordinate system definition and transformation must be performed first: height calculation involves four types of core coordinates, and their transformation relationships are as follows.

[0126] (1) The World Coordinate System (WCS) is an absolute reference system fixed in real space. Assume there is a point P in the WCS with coordinates... .

[0127] (2) The Camera Coordinate System (CCS) has the camera optical center as its origin and the optical axis as its Z-axis. Point P is mapped to the camera coordinate system through a rigid body transformation, and its coordinates are: Through the extrinsic matrix The formula is as follows:

[0128] Where R is the rotation matrix and t is the translation variable.

[0129] (3) The Image Coordinate System (ICS) is based on the principal point. The origin is a point in the camera coordinate system. Mapped to a two-dimensional image coordinate system via perspective projection The formula is as follows:

[0130] (4) The pixel coordinate system (PCS) has its origin at the top left corner of the image. The unit is changed to pixels. The point of pixel coordinates. Transformed into pixel coordinate system via affine transformation This is accomplished through the intrinsic parameter matrix K:

[0131] In the specific implementation, the height is calculated after coordinate system transformation: the ship's water height is measured based on a camera pinhole model. The ship's edge contour image is projected into a pixel coordinate system, and a mapping relationship between the 3D world coordinate system, the camera coordinate system, and the pixel coordinate system is established through the camera's intrinsic and extrinsic parameter matrices, thereby calculating the ship's actual height.

[0132] In the camera pinhole model A point in the camera coordinate system. Let P be the position of point P in the camera coordinate system after imaging. According to the triangular similarity relationship, we have:

[0133] Where: the negative sign indicates that the image is inverted. However, the image obtained by an actual camera is not inverted. Here, to make the camera pinhole model more realistic, the imaging plane can be equivalently placed symmetrically in front of the camera, on the same side of the camera coordinate system as the three-dimensional space point. The formula can be expressed as:

[0134] Will Placing it on the left side of the equation and rearranging, we get:

[0135] Based on this model, the relationship between points in three-dimensional space and the image can be obtained, and this relationship can be used to calculate the ship's waterline height. The calculation process can be simplified as follows:

[0136] Where: H is the ship's water measurement height; d is the laser ranging distance; h is the ship's pixel height in the image; and f is the camera's focal length.

[0137] The method described in this embodiment can improve the accuracy of height calculation, specifically as follows: Figure 4 As shown, the height calculation method of this approach can more accurately identify the highest point of the ship and its position on the water surface compared to Mask RCNN, thus resulting in more accurate calculations.

[0138] Compared with existing technologies, the ship water height measurement method provided in this embodiment first acquires real-time images of the ship, then preprocesses the real-time images, and then detects the preprocessed images to obtain ship detection boxes. Next, edge optimization is performed using nested edge detection algorithms and optical flow calculation to obtain accurate boundary information. Finally, accurate ship water height is calculated based on this precise boundary information. This achieves fully automated processing from image acquisition, target detection, waterline recognition to height calculation, requiring no manual intervention. It is suitable for scenarios such as waterway monitoring, port management, and bridge early warning, significantly reducing labor costs and operational complexity. Based on deep learning target detection and efficient optical flow calculation and feature matching algorithms, it possesses strong real-time processing capabilities while ensuring high accuracy, meeting the real-time requirements of waterway monitoring and bridge collision avoidance systems. Integrating image enhancement technology, deep learning target detection, and super-resolution reconstruction, it can adapt to complex water environments such as heavy fog, strong light, low illumination, and wave interference, and is suitable for height measurement of different weather conditions, distances, and ship types.

[0139] Based on the above embodiments, a specific scenario implementation method for measuring the ship's water height according to this application is proposed. First, a real-time image of the target ship is acquired using a camera. Then, the image undergoes preprocessing such as defogging and noise reduction. After this, a pre-trained ship detection model is used for target detection. Next, nested edge detection algorithms and optical flow calculations are combined to optimize the edges of the detection boxes, obtaining precise boundaries. Finally, the water height is accurately calculated based on the precise boundaries and camera parameters. This method integrates image enhancement technology, deep learning target detection, super-resolution reconstruction, and multimodal error correction methods, enabling it to adapt to complex aquatic environments such as heavy fog, strong light, low illumination, and wave interference. It is suitable for measuring the height of ships under different weather conditions, at different distances, and for different types of ships.

[0140] Please refer to Figure 5 , Figure 5 This is a flowchart illustrating one embodiment of the ship's water height measurement method of this application. Depending on different requirements, the order of the steps in this flowchart can be changed, and some steps can be omitted. This ship's water height measurement method can be applied to the aforementioned ship water height measurement device, but is not limited thereto, and the embodiments of this application do not limit it in this regard.

[0141] This embodiment is a further improvement on the aforementioned embodiment. The main improvement is that, in this embodiment, by introducing a multi-source error comprehensive correction mechanism, including the item-by-item correction of ship edge detection error, attitude change error and perspective projection error, the measurement accuracy of ship's water height is significantly improved.

[0142] The specific process of this embodiment is as follows: Figure 5 As shown, it includes the following steps: Step S501: Calculate the ship edge detection error based on the ship's precise boundary information.

[0143] It should be noted that the ship edge detection error is calculated first, specifically based on the prior information of the ship edge in the ship's precise boundary.

[0144] It should be understood that ship edge detection errors occur because factors such as water surface reflection, waves, and shadows make it difficult to determine the lower edge. In some cases, the water surface itself may also fluctuate, leading to instability in the edge position. Environmental conditions, such as weather conditions, lighting, and hull obstruction, can also cause blurring in the identification of the upper edge, such as the top of the ship's side or mast, resulting in contour detection errors.

[0145] This portion of the error can be extracted using prior information about the ship's edges. Let the ship contour detection error be... Positioning error at the lower edge Error in upper edge recognition composition:

[0146] in: The error is due to the lower edge offset caused by waterline fluctuations and reflection interference. The upper edge recognition error is caused by obstruction or blurring of the ship's top / mast.

[0147] First, a dataset of ship images collected under good visibility conditions is used as training samples for the network. The ship images under good visibility conditions are then trained using the HED (Hyper-Edge Edge Detection) algorithm to obtain a pre-trained model of the ship outline.

[0148]

[0149] In the formula: For clear images of ships under good visibility conditions; The output of the pre-trained model is the ship edge feature map.

[0150] Secondly, the preprocessed images are input in parallel into HED and Mask R-CNN. To improve the segmentation accuracy and output adaptability, some Spatial Feature Transform (SFT) layers are inserted into Mask R-CNN. The SFT layer can be represented as:

[0151] in: For feature maps; For scaling and displacement parameters; It is the product of elements.

[0152] Next, the ship edge feature map generated by HED is embedded into the fully convolutional network (FCN) branch of the Mask RCNN object detection network to guide the network to produce high-quality ship image segmentation:

[0153] in: The image to be tested after preprocessing; This is a fully convolutional operation; This is a feature splicing operation.

[0154] Next, the ship category is classified and bounding box regression is performed through a fully connected layer to output an accurate ship mask. With detection box :

[0155] In the formula: Confidence level for ship category; This is for fully connected layer operations.

[0156] Finally, based on the mask Extracting the closed profile of a ship Determine the precise coordinates of the upper and lower edges:

[0157] The extracted precise contours are substituted into the height calculation to eliminate edge detection errors and correct the edge pixel heights. :

[0158] In the formula: This is for uncorrected ship pixel height.

[0159] Step S502: Calculate the ship attitude error and ship perspective error based on the current measurement parameters.

[0160] It should be understood that it is necessary to calculate the ship's attitude error and perspective error to further correct the calculation and measurement results of the target ship's water height.

[0161] In some embodiments, step S502 includes: extracting ship feature points based on the ship detection box; performing brute-force matching between the ship feature points and a pre-acquired reference image to obtain feature point matching results; calculating a rotation matrix and a translation vector based on the feature point matching results; and calculating the ship attitude error and the ship perspective error based on the rotation matrix and the translation vector.

[0162] In practical implementation, ship attitude error and perspective error: Under close-range measurement conditions, due to the influence of camera tilt angle and ship attitude, the perspective error generated during the ship height measurement process will have a great impact on the accuracy of the ship's water height measurement results. The camera tilt angle can be expressed by the following formula:

[0163] In the formula, d is the laser ranging distance; h is the height of the ship's pixels in the image; The camera tilt angle.

[0164] When the distance between the ship and the bridge exceeds a certain range, the camera's optical axis is in the same horizontal plane as the ship, and the camera's tilt angle is approximately zero. At this point, the influence of the camera's tilt angle can be basically ignored. Therefore, the ship's height above water can be directly calculated from images taken at a distance without any deviation.

[0165] Based on this, this method considers using a ship image more than 3 kilometers away as a reference image and a close-range image of the ship under test as the image to be calibrated. ORB (Oriented Fast and Rotated BRIEF) feature points are extracted from both images. A brute-force matching method is used to match the feature points in the two images, and the KNN matching algorithm is used to filter out mismatched points. Based on the feature point matching results and epipolar geometric constraints, the three-dimensional rotation relationship between the ship image to be calibrated and the distant reference image is recovered. This three-dimensional rotation relationship is then projected onto the two-dimensional ship image, achieving correction of ship attitude and perspective errors at close range.

[0166] The specific steps are as follows: 1. ORB Feature-Based Ship Feature Point Acquisition: A ship image is a matrix composed of brightness and color, making it extremely difficult to estimate the rotation between ships at the matrix level. This method aims to select representative points from two ship images that remain unchanged even after the ship's attitude changes. Then, based on these points, the rotation relationship between the two images is calculated.

[0167] These representative points are ship feature points, which consist of keypoints and descriptor points. Keypoints refer to the location of the feature point in the image and also contain information such as orientation and size. Descriptors are typically vectors that describe the pixel information surrounding the keypoint in a deliberately designed manner. Utilizing ORB feature points, composed of FAST keypoints and BRIEF (Binary Robust Independent Elementary Feature) binary descriptors, significantly accelerates the entire ship image feature extraction process.

[0168] Specifically, FAST is a corner detection method that primarily detects areas with significant changes in local pixel grayscale. The detection process is as follows: 1. Select pixels in the image Assuming its brightness is ; 2. Set a threshold ; 3. In pixels Centered on a circle with a radius of 3, select 16 pixels. 4. If the selected circle has N consecutive points with a brightness greater than... or less So pixels These can be considered as feature points; 5. Repeat the above four steps, performing the same operation on each pixel.

[0169] Meanwhile, to avoid the "clustering" phenomenon of FAST corner points, after the first pass of detection, non-maximum suppression is required to retain only the corner points with the maximum response value in a certain area, thus avoiding the problem of corner point concentration.

[0170] Finally, after extracting the FAST keypoints, a descriptor is calculated for each point. The descriptor describes the image region surrounding the feature point extracted in the previous step. ORB uses BRIEF feature description, a binary descriptor whose description vector consists of many 0s and 1s. These 0s and 1s encode the size relationship between two random pixels (e.g., p and q) near the keypoint: if p is larger than q, a 1 is used; otherwise, a 0 is used. If 128 such p and q are selected, a 128-dimensional vector of 0s and 1s is obtained. BRIEF uses random point selection for comparison, which is very fast, and because it uses binary representation, it is also very convenient to store, making it suitable for real-time image matching.

[0171] In the FAST feature point extraction stage, ORB calculates the orientation of key points. It can use the orientation information to calculate the rotated Steer BRIEF features (BRIEF descriptors that have been rotated and aligned with the feature point orientations), which makes ORB descriptors have good rotation invariance.

[0172] 2. Ship Feature Point Matching Method Based on Brute-Force Matching: After extracting feature points from the ship reference image and the real-time ship image, this method uses brute-force matching to match feature points between the reference image and the real-time image, which can preserve the accuracy to the greatest extent.

[0173] First, select a feature point descriptor in the reference image and calculate the Hamming distance between this feature point descriptor and all feature point descriptors in the real-time image. Let the Hamming distance between the feature point descriptors be denoted as . :

[0174] in, For the i-th bit of the binary representation of the m-th feature point descriptor in the reference image, For the i-th bit of the binary representation of the n-th feature point descriptor in the real-time image, The XOR relationship is used. Then, the calculated Hamming distances are sorted, and the closest one is taken as the matching point for the feature point descriptor. This process is repeated until all feature points in the reference image have been iteratively calculated.

[0175] 3. KNN Matching Algorithm for False Matches: Due to the local nature of image features, false matches are widespread. This method uses a ratio test and the KNN matching algorithm to remove false matches. Initially, K is 2, meaning that for each match, two nearest neighbor descriptors are returned. A match is considered correct only if the distance ratio between the first and second matches is sufficiently large (the ratio threshold is usually around 2). Let the distance ratio be P, then the distance ratio of the k-th feature point descriptor is calculated as follows:

[0176] When the distance ratio of the feature point descriptor If the match is correct, the feature point descriptor pair is considered to be a correct match; otherwise, it is discarded.

[0177] 4. Solving for rotation matrix R and translation vector t under epipolar geometric constraints: Obtain two frames of images. Let the motion between the two images be R (rotation) and t (translation) of the ship in the first to the second image. The camera centers at the two time points are respectively... Now, let's consider... There is a feature point It is The middle corresponds to the feature points Both are obtained through feature matching. If the match is correct, it means that they are indeed projections of the same spatial point onto two imaging planes. First, connect the lines. and connection In three-dimensional space, they will intersect at point P. At this point... Three points can define an epipolar plane. Connection to the image plane The intersection points are the epipoles. and . As the baseline, the polar plane and the two image planes Intersection lines between It is an epipolar line.

[0178] From the perspective of the first frame, the ray It represents the possible spatial location of a pixel—because all points on this ray will project onto the same pixel. Meanwhile, if the location of P is unknown, then when the epipolar line is seen in the second image... That is, the possible projection location of P, which is the ray. The projection in the second camera, now determined by feature point matching, is... The pixel position is known, so the spatial position of P and the motion of the ship relative to the camera can be inferred.

[0179] In the coordinate system of the first frame, let the spatial position of P be...

[0180] Based on the pinhole camera model, we know that there are two pixels. The pixel position is:

[0181] Here, K is the camera intrinsic parameter matrix, and R and t represent the camera motion in the two coordinate systems. Specifically, what is calculated here is... and ,because and Transform the coordinates from the first coordinate system to the second coordinate system.

[0182] When using homogeneous coordinates, a vector is equal to itself multiplied by any non-zero constant. This is often used to express a projection relationship. For example, and If they are projected in a homogeneous coordinate system, they are equal. This equality is called equality up to a scale, denoted as:

[0183] The two projection relationships mentioned above can be written as:

[0184] Will Placing it into a normalized plane, we get:

[0185] in, This represents the outer product on the left side. It is a combination of t and Vectors that are perpendicular to each other. Again When performing the inner product, we get 0. Since the left side of the equation is strictly zero, multiplying it by any non-zero constant also results in zero. Therefore, we can obtain a concise expression:

[0186] Resubstitute ,have

[0187] Both of these formulas are called epipolar constraints. The geometric meaning of epipolar geometry is... All three are coplanar. The epipolar constraint includes both translation and rotation. We denote the middle part as the essential matrix E, which further simplifies the epipolar constraint:

[0188] The epipolar constraint concisely defines the spatial relationship between two matching points. The eight-point algorithm is used to estimate E, and then R and t are calculated.

[0189] Consider a pair of matching points whose normalized coordinates are: Based on the epipolar geometry constraints, the following relationship exists:

[0190] Expand matrix E into vector form, as shown below:

[0191] Therefore, the epipolar constraint can be written as... The relevant linear form:

[0192] Similarly, the same representation applies to other point pairs. Putting all the points into a single equation transforms it into a system of linear equations. (This represents the i-th feature point, and so on).

[0193] These eight equations form a system of linear equations. Its coefficient matrix, composed of the positions of the feature points, is 8×9 in size. E lies in the null space of this matrix. If the coefficient matrix is ​​full rank, then its null space dimension is 1, meaning e forms a line. This is consistent with the scale equivalence of e. If the matrix formed by the eight pairs of matching points satisfies the condition of having a rank of 8, then the elements of E can be obtained from the above equations.

[0194] From the essential matrix The motion R,t of the target ship relative to the camera is recovered using singular value decomposition (SVD). The SVD singular value matrix of E is:

[0195] Where U and V are orthogonal matrices. Let R be a singular value matrix. In the SVD decomposition, for any E, there exist two possible t, with R corresponding to it:

[0196] in, Indicates along Axis rotation We obtain the rotation matrix. Furthermore, since -E and E are equivalent, taking the negative sign for any t will yield the same result. Therefore, when decomposing E to t, R, there are a total of 4 possible solutions.

[0197] By substituting any point into the four possible solutions and checking the depth of that point under both cameras, we can determine which solution is correct.

[0198] Thus, we have obtained the rotation R and translation t of the target ship with perspective projection error at medium to close range relative to the ship at a long distance. This information can be used to correct the ship's errors.

[0199] 5. Correction of ship attitude error and perspective error: Error correction can be performed using the rotation matrix and translation vector obtained from F2.4.

[0200] Let the pixel position of a certain ship feature point in the image be... The true three-dimensional coordinates of this feature point in the world coordinate system are... The attitude error can be corrected using the following formula:

[0201] Where K is the camera intrinsic parameter; It is the calibrated camera extrinsic parameter matrix.

[0202] To address perspective errors, the image is reconstructed using the perspective transformation matrix H. For a given image point... Its corrected new position It can be calculated using the following relationship:

[0203] Through perspective error correction, actual measurement results show that within a distance range of 0.5-3 kilometers, the pixel error was reduced by 14% after correction. This result proves that perspective distortion correction can significantly improve the accuracy of ship height measurement and meet practical application requirements.

[0204] Step S503: Correct the ship's water height based on the ship edge detection error, the ship attitude error, and the ship perspective error.

[0205] It should be understood that the corrected height calculation is as follows: The accurate formula for calculating the ship's water height after error correction is given below.

[0206] in, H The desired height of the target vessel above water; h The vertical pixel height for ship imaging; Errors in ship profile extraction; Errors caused by the ship's navigation attitude; Error caused by camera tilt angle; d The distance from the ship to the camera can be obtained using a laser rangefinder; f This refers to the camera's focal length.

[0207] Specifically, daytime field tests showed that, within a distance range of 0.5-6.5 kilometers, the corrected ship detection error was reduced by approximately 40.175%; nighttime field tests showed that, within a distance range of 0.5-4.5 kilometers, the corrected ship detection error was reduced by approximately 20.15%. The results indicate that, after the aforementioned error correction, the ship detection accuracy is significantly improved, meeting the high-precision measurement requirements for bridge navigation safety early warning.

[0208] Compared with existing technologies, the ship water height measurement method provided in this embodiment significantly improves the measurement accuracy of ship water height by introducing a multi-source error comprehensive correction mechanism, including item-by-item correction of ship edge detection error, attitude change error and perspective projection error.

[0209] Based on the same concept as the ship water height measurement method in the above embodiments, this application also provides a ship water height measurement device, which can be used to perform the above-described ship water height measurement method. For ease of explanation, the structural schematic diagram of the ship water height measurement device embodiment only shows the parts related to the embodiments of this application. Those skilled in the art will understand that the illustrated structure does not constitute a limitation on the device, and it may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0210] like Figure 7 As shown, the ship's water height measuring device 700 includes a preprocessing module 701, a detection module 702, an optimization module 703, and a calculation module 704. In some embodiments, the above modules can be programmable software instructions stored in memory and executable by a processor. It is understood that in other embodiments, the above modules can also be program instructions or firmware embedded in the processor.

[0211] The preprocessing module 701 is used to acquire a preprocessed image of the target ship, the ship image being captured by the target camera; The detection module 702 is used to detect the preprocessed ship image using a ship detection model to obtain a ship detection frame; The optimization module 703 is used to process the reconstructed image corresponding to the ship detection box through a nested edge detection algorithm to obtain ship edge prior information, and to calculate the motion information of the target ship through optical flow. The precise waterline position and the precise ship boundary are determined by combining the ship edge prior information and the motion information. The acquisition module 704 is used to acquire the camera parameters and camera shooting distance of the target camera, wherein the camera shooting distance is the measured distance between the target ship and the target camera. The calculation module 705 is used to calculate the ship's water height based on the camera parameters, the camera shooting distance, the precise waterline position, and the ship's precise boundary.

[0212] The ship water height measuring device 700 provided in the above embodiments can realize the technical solutions described in the above embodiments of the ship water height measuring method. The specific implementation principles of each module or unit can be found in the corresponding content in the above embodiments of the ship water height measuring method, and will not be repeated here.

[0213] Please refer to Figure 8 , Figure 8 This is a schematic diagram of an embodiment of the electronic device of this application. In this embodiment of the invention, the electronic device 800 includes a processor 801, a memory 802, and a display 803. Figure 7 Only some components of the electronic device 800 are shown, but it should be understood that it is not required to implement all the components shown, and more or fewer components may be implemented instead.

[0214] In some embodiments, processor 801 may be a central processing unit (CPU), microprocessor, or other data processing chip, used to run program code stored in memory 802 or process data, such as the ship water height measurement method of the present invention.

[0215] In some embodiments, processor 801 may be a single server or a group of servers. The server group may be centralized or distributed. In some embodiments, processor 801 may be local or remote. In some embodiments, processor 801 may be implemented on a cloud platform. In one embodiment, the cloud platform may include a private cloud, public cloud, hybrid cloud, community cloud, distributed cloud, intranet, multi-cloud, etc., or any combination thereof.

[0216] In some embodiments, memory 802 may be an internal storage unit of electronic device 800, such as a hard disk or memory of electronic device 800. In other embodiments, memory 802 may also be an external storage device of electronic device 800, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc. equipped on electronic device 800.

[0217] Furthermore, the memory 802 may include both internal storage units of the electronic device 800 and external storage devices. The memory 802 is used to store application software and various types of data installed on the electronic device 800.

[0218] In some embodiments, display 803 may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, or an OLED (Organic Light-Emitting Diode) touchscreen. Display 803 is used to display information from electronic device 800 and to display visual user applications. Components 801-803 of electronic device 800 communicate with each other via a system bus.

[0219] In one embodiment, when processor 801 executes the ship's water height measurement program in memory 802, the following steps can be performed: Acquire a preprocessed image of the target vessel, the image being captured by the target camera; The preprocessed ship image is detected using a ship detection model to obtain a ship detection bounding box; The reconstructed image corresponding to the ship detection box is processed by a nested edge detection algorithm to obtain the ship edge prior information, and the motion information of the target ship is calculated by optical flow. The precise waterline position and the precise boundary of the ship are determined by combining the ship edge prior information and the motion information. Obtain the camera parameters and camera shooting distance of the target camera, wherein the camera shooting distance is the measured distance between the target ship and the target camera; The ship's height above water is calculated based on the camera parameters, the camera shooting distance, the precise waterline position, and the ship's precise boundary.

[0220] It should be understood that when the processor 801 executes the ship water height measurement program in the memory 802, in addition to the functions mentioned above, it can also perform other functions, as can be found in the description of the corresponding method embodiments above.

[0221] Furthermore, this embodiment of the invention does not specifically limit the type of electronic device 800 mentioned. Electronic device 800 can be a mobile phone, tablet computer, personal digital assistant (PDA), wearable device, laptop computer, or other portable electronic device. Exemplary embodiments of portable electronic devices include, but are not limited to, portable electronic devices running iOS, Android, Microsoft, or other operating systems. The aforementioned portable electronic device can also be other portable electronic devices, such as a laptop computer with a touch-sensitive surface (e.g., a touch panel). It should also be understood that in some other embodiments of the invention, electronic device 800 may not be a portable electronic device, but rather a desktop computer with a touch-sensitive surface (e.g., a touch panel).

[0222] Accordingly, this application also provides a computer-readable storage medium for storing computer-readable programs or instructions. When the programs or instructions are executed by a processor, they can implement the steps or functions of the ship water height measurement methods provided in the above-described method embodiments.

[0223] Those skilled in the art will understand that all or part of the processes of the methods described in the above embodiments can be implemented by a computer program instructing related hardware (such as a processor, controller, etc.), and the computer program can be stored in a computer-readable storage medium. The computer-readable storage medium may be a disk, optical disk, read-only memory, or random access memory, etc.

[0224] The above provides a detailed description of the ship water height measurement method, apparatus, electronic equipment, and system provided in this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A method for measuring the water height of a ship, characterized in that, include: Acquire a preprocessed image of the target vessel, the image being captured by the target camera; The preprocessed ship image is detected using a ship detection model to obtain a ship detection bounding box; The reconstructed image corresponding to the ship detection box is processed by a nested edge detection algorithm to obtain the ship edge prior information, and the motion information of the target ship is calculated by optical flow. The precise waterline position and the precise boundary of the ship are determined by combining the ship edge prior information and the motion information. Obtain the camera parameters and camera shooting distance of the target camera, wherein the camera shooting distance is the measured distance between the target ship and the target camera; The ship's height above water is calculated based on the camera parameters, the camera shooting distance, the precise waterline position, and the precise ship boundary. The process of processing the reconstructed image corresponding to the ship detection box using a nested edge detection algorithm to obtain ship edge prior information, calculating the motion information of the target ship using optical flow, and combining the ship edge prior information and the motion information to determine the precise waterline position and the precise ship boundary includes: The region of interest is extracted from the ship detection frame to obtain the region of interest; Super-resolution reconstruction is performed on the region of interest to obtain a reconstructed image; The reconstructed image is processed by a nested edge detection algorithm to obtain prior information about the ship's edges; The motion information of the target ship is calculated using the dense optical flow method and coefficient optical flow method of the pyramid. Based on the prior information about the ship's edge and the motion information, the precise waterline positioning and precise boundary of the target ship are calculated. The step of calculating the ship's above-water height based on the camera parameters, the camera shooting distance, the precise waterline position, and the ship's precise boundary further includes: The ship edge detection error is calculated based on the precise ship boundary information. The ship's attitude error and perspective error are calculated based on the current measurement parameters. The ship's water height is corrected based on the ship's edge detection error, the ship's attitude error, and the ship's perspective error; The calculation of ship attitude error and ship perspective error based on current measurement parameters includes: Ship feature points are extracted based on the ship detection frame; The ship feature points are then matched with a pre-acquired reference image using a brute-force method to obtain the feature point matching results. Calculate the rotation matrix and translation vector based on the feature point matching results; The ship's attitude error and perspective error are calculated based on the rotation matrix and the translation vector.

2. The method for measuring the water height of a ship according to claim 1, characterized in that, Before acquiring the preprocessed image of the target vessel, the process also includes: The real-time image of the target ship is dehazed using a dark channel prior dehazing algorithm to obtain a dehazed image. The dehazed image is then subjected to contrast enhancement processing to obtain an enhanced image; The enhanced image is then subjected to noise reduction processing to obtain a denoised image; The denoised image is sharpened using an anti-sharpening mask algorithm to obtain a preprocessed ship image.

3. The method for measuring the water height of a ship according to claim 1, characterized in that, Before detecting the preprocessed ship image using a ship detection model to obtain the ship detection bounding box, the method further includes: Acquire training ship images, and perform label annotation and dataset partitioning on the training ship images to obtain training data; The training data is scaled and normalized to obtain the training dataset; The ship detection model is obtained by training the preset lightweight model using the training dataset through forward propagation calculation.

4. The method for measuring the water height of a ship according to claim 1, characterized in that, The calculation of the ship's above-water height based on the camera parameters, the camera shooting distance, the precise waterline position, and the ship's precise boundary includes: Based on the precise waterline position and the precise boundary of the ship, a coordinate system transformation is performed to obtain the boundary information of the pixel coordinate system. The coordinate system transformation process is to map the precise boundary information of the ship from the world coordinate system to the camera coordinate system, then to the image coordinate system through perspective projection, and finally to the pixel coordinate system through affine transformation. The ship's water height is calculated based on the boundary information, the camera parameters, and the camera shooting distance.

5. A device for measuring the water height of a ship, characterized in that, The ship surface height measuring device is applied to the ship surface height measuring method as described in any one of claims 1 to 4, and includes: a preprocessing module, a detection module, an optimization module, an acquisition module, and a calculation module; The preprocessing module is used to acquire preprocessed images of the target vessel, which are captured by the target camera. The detection module is used to detect the preprocessed ship image using a ship detection model to obtain a ship detection frame; The optimization module is used to process the reconstructed image corresponding to the ship detection box through a nested edge detection algorithm to obtain the ship edge prior information, and to calculate the motion information of the target ship through optical flow. The precise waterline position and the precise ship boundary are determined by combining the ship edge prior information and the motion information. The acquisition module is used to acquire the camera parameters and camera shooting distance of the target camera, wherein the camera shooting distance is the measured distance between the target ship and the target camera; The calculation module is used to calculate the ship's height above water based on the camera parameters, the camera shooting distance, the precise waterline position, and the ship's precise boundary.

6. An electronic device, the electronic device comprising a processor and a memory, characterized in that, The memory is used to store instructions, and the processor is used to call the instructions in the memory to cause the electronic device to execute the ship's water height measurement method as described in any one of claims 1 to 4.

7. A ship's water height measurement system, characterized in that, The ship's water height measurement system includes the electronic device, data acquisition device, and result output device as described in claim 6. The data acquisition device and the result output device are respectively connected to the electronic device. The data acquisition device is used to acquire real-time images of the target ship, and the result output device is used to output and display the water height of the target ship.

Images