Underwater target identification and positioning method based on physical model and deep learning fusion

Abstract

The invention discloses an underwater target identification and positioning method based on fusion of a physical model and deep learning, and belongs to the technical field of computer vision, and the method comprises the steps: carrying out the transmissivity estimation and physical restoration of a collected underwater monocular image according to an underwater light propagation model, and carrying out the image correction; key feature points are extracted, and high-quality matching point pairs are screened in combination with the joint similarity; further deriving a basic matrix through the high-quality matching point pairs meeting the epipolar geometric constraint relation, solving an essential matrix, and obtaining relative attitude parameters between the cameras in combination with weighted re-projection-LM optimization; and finally, refraction correction triangulation is carried out according to the Snell's law, pixel-level fusion is carried out after scale normalization and space alignment are carried out on the refraction correction triangulation and dense depth output by the MiDaS, three-dimensional space coordinates of the target are inverted, and high-precision recognition and positioning of the underwater target are achieved. The system is light in structure, efficient in calculation, suitable for being integrated on various underwater autonomous or remote control robot platforms and used for tasks such as target recognition, tracking and positioning.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer vision, in particular to an adaptive monocular vision underwater target recognition and positioning method based on fusion of physical model and deep learning. BACKGROUND

[0002] Underwater medium absorption and scattering are strong, illumination is uneven, color shift is high, and suspended particle concentration is high, resulting in low imaging contrast, missing details, rising noise, and aggravated geometric distortion, which significantly restricts target recognition and spatial positioning based on vision. Monocular cameras are widely used by AUVs and ROVs due to their small size, low power consumption, and convenient deployment, but in complex water areas, the recognition stability and positioning accuracy are difficult to meet the high-reliability operation requirements. Existing solutions mainly include three categories: binocular vision, structured light and laser ranging, and monocular geometric constraint-based, which have high synchronization and assembly requirements, are sensitive to power consumption and water quality, and rely heavily on texture and illumination, etc., and have insufficient robustness. Underwater operations also face constraints such as rapid changes in attitude, severe fluctuations in illumination, time-varying environmental parameters, and limited computing power, which pose rigid requirements on the lightweight and real-time performance of algorithms. Based on the above status, there is an urgent need for monocular vision positioning methods that synergistically optimize physical restoration, robust matching, reliable attitude solving, refractive error compensation, and depth information fusion, to achieve stable recognition and high-precision positioning under different turbidity, different illumination, and different water depth conditions. SUMMARY

[0003] In view of the problems of poor target recognition robustness, insufficient positioning accuracy, and difficulty in suppressing refractive error in the background art, the present application provides an adaptive monocular vision underwater target recognition and positioning method based on fusion of physical model and deep learning. The method takes the underwater imaging physical degradation model as a priori, uses dark channel prior and convolutional neural network image enhancement for image restoration and detail compensation; uses SuperPoint for key point detection and description, combines R2D2 to construct local features, and selects high-quality matching points according to confidence; estimates the fundamental matrix through the eight-point method based on weighted RANSAC, combines the intrinsic matrix to calculate the essential matrix and performs singular value decomposition to obtain the relative attitude parameters; under the refractive correction triangulation model, the output of monocular depth estimation MiDaS is fused at the pixel level to complete scale restoration and three-dimensional coordinate inversion, thereby realizing stable recognition and high-precision positioning under complex water conditions.

[0004] In a first aspect, embodiments of this application provide an adaptive monocular vision underwater target recognition and localization method based on the fusion of physical models and deep learning, comprising the following steps: preprocessing the acquired image: acquiring underwater environmental parameters, and performing detail compensation, color correction, and contrast enhancement on the acquired underwater image to obtain a high-fidelity underwater image; feature extraction and matching: extracting multi-scale key points of the high-fidelity underwater image using the SuperPoint deep feature detection network, constructing local feature vectors using the R2D2 descriptor, calculating the joint similarity of color, texture, and gradient, and selecting the feature point pair with the highest confidence as a high-quality matching point; pose solving and optimization: using the weighted RANSAC algorithm to remove abnormal matching points, estimating the fundamental matrix based on the high-quality matching point pair using the eight-point method, solving the essential matrix in combination with the camera intrinsic parameter matrix, and optimizing the essential matrix. Singular value decomposition yields the rotation matrix and translation vector, and the reprojection error is minimized using a confidence matrix to obtain the optimal attitude parameters. Refraction-corrected triangulation: The light propagation path is corrected according to Snell's law, and a set of refraction-corrected triangulation equations is established. The depth value of the target point is calculated using the least squares method. Depth fusion and spatial localization: The depth value is fused pixel-level with the dense depth map output by the monocular depth estimation network MiDaS. The optimal attitude parameters and intrinsic parameter matrix are then combined to perform scale recovery and 3D coordinate inversion to obtain the target's spatial position in the world coordinate system. Confidence assessment and result output: The confidence value of the localization result is calculated based on the matching residual variance, depth stability, and image quality indicators. The localization result and confidence assessment report are output. The image quality indicators include peak signal-to-noise ratio (PSNR) and structural similarity index (SSIM).

[0005] In some embodiments, the preprocessing of the acquired images includes: environmental adaptive calibration: acquiring calibration plate images under different water depths and lighting conditions, calculating the refractive index and attenuation coefficient of the medium based on an underwater light propagation model, correcting the camera intrinsic parameter matrix and distortion coefficient, and establishing an environmental adaptive calibration parameter set; image restoration and enhancement: constructing an underwater imaging degradation model, estimating transmittance and background light based on dark channel priors, and using a convolutional neural network image enhancement model to perform detail compensation, color correction, and contrast enhancement on the restored image to obtain the high-fidelity underwater image.

[0006] In some implementations, establishing an environmental adaptive calibration parameter set includes: measuring environmental parameters in real time using a light sensor and a turbidity sensor, dynamically updating the camera focal length and principal point position based on interpolation calculations, and achieving automatic correction of the calibration parameters.

[0007] In some implementations, the convolutional neural network image enhancement model is an improved WaterNet network that uses attention-guided convolutional modules to extract channel features and utilizes residual block structures to enhance texture details and suppress over-enhancement.

[0008] In some implementations, the objective function for optimizing the weighted projection error in obtaining the optimal attitude parameters by weighting the reprojection error using the confidence matrix is: in, For the first The confidence weight of the matching points Let be the attitude projection matrix.

[0009] In some implementations, the step of performing singular value decomposition on the essential matrix to obtain the rotation matrix and translation vector includes: using the Levenberg-Marquardt algorithm to perform nonlinear least squares optimization on the weighted projection error, and outputting the rotation matrix and the translation vector.

[0010] In some implementations, the depth fusion and spatial localization includes: the depth output of the MiDaS network, after scale normalization and spatial alignment, is fused with the refraction-corrected depth using a confidence-weighted average.

[0011] In some implementations, the positioning results output by the confidence assessment step include the target's three-dimensional coordinates, pose matrix, root mean square depth error, matching success rate, and image sharpness index.

[0012] Secondly, embodiments of the present invention also provide an underwater target recognition and localization system based on a fusion of physical models and deep learning, comprising: a preprocessing module for acquiring underwater environmental parameters and performing detail compensation, color correction, and contrast enhancement on the acquired underwater images to obtain high-fidelity underwater images; a feature extraction and matching module for extracting multi-scale key points of the high-fidelity underwater images using the SuperPoint deep feature detection network, constructing local feature vectors using the R2D2 descriptor, calculating the joint similarity of color, texture, and gradient, and selecting the feature point pairs with the highest confidence as high-quality matching points; and a pose solving and optimization module for removing abnormal matching points using the weighted RANSAC algorithm, estimating the fundamental matrix based on the high-quality matching point pairs using the eight-point method, solving the essential matrix in conjunction with the camera intrinsic parameter matrix, and optimizing the essential matrix. The system performs singular value decomposition to obtain rotation and translation vectors, and uses a confidence matrix to minimize the reprojection error to obtain optimal attitude parameters. A refraction correction module corrects the light propagation path according to Snell's law, establishes a set of refraction correction triangulation equations, and calculates the depth value of the target point using the least squares method. A depth fusion and spatial positioning module performs pixel-level fusion of the depth value with the dense depth map output by the monocular depth estimation network MiDaS, and combines the optimal attitude parameters and intrinsic parameter matrix to perform scale recovery and 3D coordinate inversion to obtain the target's spatial position in the world coordinate system. A confidence assessment and result output module calculates the confidence value of the positioning result based on the matching residual variance, depth stability, and image quality indices (PSNR and SSIM), and outputs the positioning result and a confidence assessment report.

[0013] Thirdly, embodiments of the present invention also provide a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when run on a processor, executes all the steps of the underwater target identification and localization method based on the fusion of physical model and deep learning provided in the first aspect of the present invention.

[0014] Beneficial effects: This invention uses a physical imaging degradation model as a priori and cascades it with an improved WaterNet to significantly improve the color consistency and texture fidelity of degraded images. In the feature stage, SuperPoint+R2D2 and the joint similarity of color / texture / gradient together constitute a multimodal matching criterion, and confidence-weighted screening suppresses weak textures and false matches, significantly improving the matching stability of low-contrast scenes. In the pose stage, weighted RANSAC, the eight-point method and Levenberg-Marquardt optimization are combined, and the confidence weight of reprojection error is used to reduce the impact of abnormal matching on the solution and improve the convergence and stability of pose solution. In the depth stage, refraction-corrected triangulation based on Snell's law provides the metric depth, and then pixel-level, confidence-weighted fusion is performed with MiDaS dense depth after scale normalization and spatial alignment, effectively suppressing the systematic scale deviation caused by interface refraction and enhancing the accuracy of scale recovery. In summary, this invention achieves closed-loop collaboration in key aspects such as physical priors + deep learning enhancement, confidence-driven robust geometric estimation, and deep fusion of refraction error compensation and uncertainty perception. Under the premise of lightweight equipment and real-time performance, it significantly improves environmental adaptability and positioning accuracy, and is applicable to target recognition, tracking and high-precision positioning tasks of various underwater robot platforms in complex water conditions.

[0015] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of this application more obvious and understandable, the following are specific embodiments of this application. Attached Figure Description

[0016] To more clearly illustrate the technical solution of this application, the accompanying drawings used in this application will be briefly described below.

[0017] Figure 1 This is a flowchart illustrating an underwater target identification and localization method based on the fusion of physical models and deep learning, provided in an embodiment of the present invention. Figure 2 This is a schematic diagram of an adaptive calibration process and environmental parameter interpolation in an underwater target recognition and localization method based on the fusion of physical models and deep learning provided in an embodiment of the present invention. Figure 3 This is a flowchart of a dark channel and transmittance estimation and image enhancement method for underwater target recognition and localization based on physical model and deep learning provided in an embodiment of the present invention. Figure 4 This is a block diagram of an improved WaterNet network structure in an underwater target recognition and localization method based on the fusion of physical models and deep learning provided in an embodiment of the present invention. Figure 5 This is a schematic diagram of a feature extraction and matching process in an underwater target recognition and localization method based on the fusion of physical models and deep learning provided in an embodiment of the present invention. Figure 6 This is a schematic diagram of an attitude solving and optimization process in an underwater target recognition and localization method based on the fusion of physical models and deep learning provided in an embodiment of the present invention. Figure 7 This is a geometrical diagram of a refraction-corrected triangulation method for underwater target identification and localization based on the fusion of physical models and deep learning, provided in an embodiment of the present invention. Figure 8 This is a flowchart of a MiDaS depth alignment, confidence-weighted fusion, and 3D coordinate inversion method for underwater target recognition and localization based on physical models and deep learning provided in this embodiment of the invention. Figure 9 This is a schematic diagram of the structure of an underwater target recognition and localization system based on the fusion of physical model and deep learning, provided in an embodiment of the present invention. Detailed Implementation

[0018] The embodiments of the technical solution of this application will now be described in detail with reference to the accompanying drawings. These embodiments are only used to more clearly illustrate the technical solution of this application and are therefore merely examples, and should not be used to limit the scope of protection of this application.

[0019] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains; the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the application; the terms “comprising” and “having”, and any variations thereof, in the specification, claims, and foregoing description of the drawings are intended to cover non-exclusive inclusion.

[0020] In the description of the embodiments of this application, technical terms such as "first" and "second" are used only to distinguish different objects and should not be construed as indicating or implying relative importance or implicitly specifying the number, specific order, or primary and secondary relationship of the indicated technical features. In the description of the embodiments of this application, "multiple" means two or more, unless otherwise explicitly defined.

[0021] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0022] In the description of the embodiments in this application, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this document generally indicates that the preceding and following related objects have an "or" relationship.

[0023] In the description of the embodiments of this application, the term "multiple" refers to two or more (including two), similarly, "multiple sets" refers to two or more (including two sets), and "multiple pieces" refers to two or more (including two pieces).

[0024] In the description of the embodiments of this application, the technical terms "center," "longitudinal," "lateral," "length," "width," "thickness," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," "outer," "clockwise," "counterclockwise," "axial," "radial," and "circumferential" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing the embodiments of this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the embodiments of this application.

[0025] In the description of the embodiments of this application, unless otherwise expressly specified and limited, technical terms such as "installation," "connection," "joining," and "fixing" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components. For those skilled in the art, the specific meaning of the above terms in the embodiments of this application can be understood according to the specific circumstances.

[0026] Underwater environments commonly present challenges such as strong medium scattering, uneven illumination and color shifts, and degradation caused by suspended particles. These issues easily lead to decreased image contrast, loss of texture details, and accumulation of geometric distortions, placing higher demands on the robustness and real-time performance of vision-based recognition and localization. Given that underwater robot platforms often rely on monocular cameras as their primary sensing method, existing methods are sensitive to time-varying environmental parameters and interface refraction, and are easily affected by weak textures and low-contrast scenes. They suffer from insufficient stability in target recognition, and spatial localization exhibits scale bias and error drift, making it difficult to meet the accuracy and reliability requirements of operations in complex waters.

[0027] In view of the above problems, this application provides an adaptive monocular vision underwater target recognition and localization method based on the fusion of physical model and deep learning. The method first establishes an environment-adaptive calibration parameter set based on the underwater light propagation model and sensor data, and dynamically corrects the camera intrinsic parameter matrix and distortion coefficients. Then, it uses dark channel priors for transmittance estimation and physical restoration, and employs an improved WaterNet to achieve detail compensation and color correction, improving the quality of degraded images. In the feature stage, it uses SuperPoint to extract key points, combines R2D2 descriptors and joint similarity, and uses confidence weighting to filter high-quality matching point pairs. In the attitude stage, it estimates the fundamental matrix using weighted RANSAC and the eight-point method, solves the essential matrix, and uses Levenberg-Marquardt to optimize the weighted projection error, obtaining a stable camera relative attitude. In the depth stage, it performs refraction-corrected triangulation based on Snell's law to obtain the measured depth, and after scale normalization and spatial alignment, it performs pixel-level confidence fusion with the dense depth output by MiDaS to complete the target's 3D coordinate inversion. Finally, it outputs a confidence evaluation result including matching success rate, reprojection residual, depth error, and image quality indicators, thereby achieving stable recognition and high-precision positioning under complex water conditions. The specific steps of this underwater target recognition and localization method based on the fusion of physical models and deep learning are illustrated below with reference to various embodiments.

[0028] In some embodiments, such as Figure 1 As shown, the steps of the underwater target recognition and localization method based on the fusion of physical models and deep learning include: Step S101: Preprocess the acquired image.

[0029] This can be understood as the underwater environment affecting the image, requiring correction of the acquired image based on environmental parameters to improve the accuracy of subsequent recognition and positioning. Specifically, underwater environmental parameters are acquired, and detail compensation, color correction, and contrast enhancement are performed on the acquired underwater image to obtain a high-fidelity underwater image.

[0030] Step S102: Feature extraction and matching.

[0031] This can be understood as needing to identify the key points of the target object through key point extraction, and then determine the boundary contour of the target object through the key points. Specifically, the SuperPoint deep feature detection network is used to extract multi-scale key points, the R2D2 descriptor is used to construct local feature vectors, the joint similarity of color, texture and gradient is calculated, and the feature point pair with the highest confidence is selected as high-quality matching points.

[0032] Step S103: Attitude solution and optimization.

[0033] This can be understood as the pose of the target object affecting subsequent recognition and localization. It is necessary to determine the pose of the target object based on its key points and rotate and translate the essential matrix of the target object according to its pose, thereby projecting the coordinate system of the essential matrix onto the coordinate system under the current pose. Specifically, the weighted RANSAC algorithm is used to remove abnormal matching points, the fundamental matrix is ​​estimated using the eight-point method based on high-quality matching point pairs, the essential matrix is ​​solved in combination with the camera intrinsic parameter matrix, the essential matrix is ​​obtained by singular value decomposition of the essential matrix to obtain the rotation matrix and translation vector, and the reprojection error is minimized by weighting the confidence matrix to obtain the optimal pose parameters.

[0034] Step S104: Refraction Correction Triangulation.

[0035] This can be understood as follows: because refraction causes underwater images to deform, the depth of the target object underwater can be calculated based on this deformation. Specifically, the light propagation path is corrected according to Snell's law, a set of refraction-corrected trigonometric equations is established, and the depth value of the target point is calculated using the least squares method.

[0036] Step S105: Deep Fusion and Spatial Positioning.

[0037] This can be understood as follows: after acquiring depth information, it is necessary to reconstruct the three-dimensional space of the target object and perform depth fusion with the depth information to obtain the spatial position of the target object in the world coordinate system. Specifically, the depth obtained by refraction correction is fused with the dense depth map output by the monocular depth estimation network MiDaS at the pixel level, and scale recovery and three-dimensional coordinate inversion are performed by combining attitude parameters and intrinsic parameter matrices to obtain the spatial position of the target in the world coordinate system.

[0038] Step S106: Confidence assessment and result output.

[0039] This can be understood as follows: after outputting the results, it is also necessary to output the recognition and positioning quality of the target object, so that the user can determine whether the recognition and positioning results are reliable. Specifically, the confidence value of the positioning results is calculated based on the matching residual variance, depth stability and image quality indicators, and the positioning results and confidence evaluation report are output. The image quality indicators include peak signal-to-noise ratio PSNR and structural similarity index SSIM.

[0040] It should be noted that in this embodiment, firstly, the monocular camera is calibrated under different water depths and lighting conditions to obtain its intrinsic parameter matrix and distortion coefficients. Then, the background light and attenuation coefficients are estimated using an underwater light propagation model. An environment-adaptive calibration parameter set is established using interpolation updates from the illumination / turbidity sensors, establishing the geometric transformation relationship between the world coordinate system, camera coordinate system, and image coordinate system. Secondly, the acquired underwater images are first subjected to transmittance estimation and physical restoration based on dark channel priors. Then, attention-guided convolution and residual blocks using an improved WaterNet are used to complete detail compensation and color correction to improve image clarity and texture fidelity. Subsequently, SuperPoint is used to detect multi-scale keypoints and R2D2 is combined to construct local descriptors based on color / High-quality matching point pairs are selected using joint similarity of texture / gradient and a confidence threshold. Then, outlier matches are eliminated based on weighted RANSAC. The fundamental matrix is ​​estimated using the eight-point method, and the essential matrix is ​​derived from intrinsic parameters. Singular value decomposition is performed on the matrix, and the reprojection error is optimized using Levenberg-Marquardt with confidence weighting to recover stable camera relative attitude parameters. Finally, refraction correction triangulation is performed according to Snell's law to obtain the depth measurement. After scale normalization and spatial alignment, the depth measurement is fused with the dense depth output by MiDaS using pixel-level confidence weighting. Combined with attitude and intrinsic parameters, the 3D coordinates of target feature points are inverted and mapped to the world coordinate system, thereby achieving high-precision recognition and spatial positioning of underwater targets in complex aquatic environments. Compared with solutions relying on binoculars or structured light, this invention has synergistic advantages in terms of equipment simplification, environmental adaptability, matching robustness, and positioning accuracy. It also has good real-time performance and platform adaptability, making it suitable for target recognition and tracking positioning tasks in various underwater robot systems.

[0041] The following describes, in conjunction with different embodiments, Figure 1 The specific implementation methods of each step in the process are illustrated by example.

[0042] In some embodiments, Figure 1 Step S101 includes: The first step is to acquire calibration plate images under different water depths and lighting conditions, calculate the refractive index and attenuation coefficient of the medium based on the underwater light propagation model, correct the camera intrinsic parameter matrix and distortion coefficient, and establish an environment-adaptive calibration parameter set.

[0043] For example, acquiring calibration board images under different water depths and lighting conditions may include: importing calibration board images captured by a camera into the program and inputting the actual size of each grid cell on the calibration board (7×9 checkerboard, 20mm per cell); the system automatically identifies and extracts the corner coordinates of the checkerboard cells to establish the geometric correspondence between the image coordinate system and the world coordinate system. To improve numerical stability, the pixel coordinates are first subjected to Hartley normalization before being used in the estimation.

[0044] In this embodiment, the medium attenuation and background light are estimated based on the underwater light propagation model, and the irradiance is observed using a single scattering prior. Where I represents the observed pixel RGB, J represents the non-degradable irradiance, A represents the background light, β represents the attenuation coefficient, and d(x) represents the optical path. Using a close-range white / gray card sequence, the following least-squares equation is used to estimate β and A: in, This is the reference irradiance for the near-field calibration plate.

[0045] For example, in this embodiment, the intrinsic parameter matrix obtained from air calibration is based on the effect of underwater refraction. K With distortion coefficient D Compensation is performed to form an environmental adaptive calibration parameter set. The compensation parameters are obtained by minimizing the reprojection error: in , It is a perspective projection model with distortion.

[0046] For example, such as Figure 2 As shown, in this embodiment, the light sensor and turbidity sensor collect environmental parameters in real time. Dynamically update the focal length through sliding window interpolation. With the main point And update accordingly. K , D Automatic correction of calibration parameters is achieved.

[0047] The second step is image restoration and enhancement.

[0048] Specifically, an underwater imaging degradation model is constructed. Based on the prior estimation of transmittance and background light in the dark channel, a convolutional neural network image enhancement model is used to perform detail compensation, color correction, and contrast enhancement on the restored image to obtain a high-fidelity underwater image.

[0049] For example, in this embodiment, an underwater imaging degradation model is constructed; transmittance and background light are estimated based on the dark channel prior. The dark channel is defined as: Where 'c' represents the color channel. Indicates The local area centered on This represents the number of pixels in each color channel of the image.

[0050] A is selected from the high percentile pixels of the dark channel, and the coarse transfer rate is calculated: In the formula, Indicates coarse transmission rate. These are the values ​​of each color channel for pixel A, which is the highest percentile pixel in the dark channel. These are the preset weight parameters.

[0051] right Perform guided filtering refinement and physical restoration: In the formula, This represents the transmittance at pixel x. This represents the minimum lower limit of transmittance set to avoid numerical instability, with a value of 0.1.

[0052] Optionally, in this embodiment, the restored image is input into an improved WaterNet network to perform detail compensation, color correction, and contrast enhancement to obtain a high-fidelity underwater image. The improved WaterNet network includes: a focus-guided convolutional module (SE channel attention) cascaded after each convolutional block of the encoder to recalibrate channel weights; and a residual block cascaded after each upsampling stage of the decoder (3×3 convolution—BN—ReLU—3×3 convolution—BN, added with the identity mapping) to enhance texture and suppress over-enhancement (e.g., ...). Figure 3 , Figure 4 (As shown). The loss function during training is: In the formula, Indicates the loss value. This represents the enhanced image output by the network. This represents the reference ground truth image used for supervised training, and... Pixel-by-pixel alignment , , These represent the pixel values ​​of the R, G, and B channels of the enhanced image, respectively.

[0053] The entire forward pass during the inference period is enhanced.

[0054] In some embodiments, Figure 1 Step S102 specifically includes: The SuperPoint deep feature detection network is used to extract multi-scale key points; after non-maximum suppression, the output key points are sorted from high to low confidence, and only the top N key points are retained for matching, where N is any value from 100 to 1000, preferably 500.

[0055] Subsequently, R2D2 descriptors are used to construct local feature vectors. For example, in this embodiment, color, texture, and gradient similarity are calculated and weighted with descriptor similarity to form a joint similarity: in For L2 normalized descriptor dot product, To find the intersection of normalized color histograms, χ for LBP 2 Distance exponential mapping, For HOG gradient cosine similarity, the weights α, β, γ, The determination is optimized on the validation set using grid search.

[0056] like Figure 5 As shown, the feature point pairs with the highest confidence are selected as high-quality matching points based on the joint similarity S and the ratio test.

[0057] In some embodiments, step S103 includes the following specific steps: The weighted random sampling consensus (RANSAC) algorithm is used to remove outlier matching points; based on high-quality matching point pairs, the fundamental matrix F is estimated using the eight-point method, and the essential matrix is ​​solved by combining the camera intrinsic parameter matrix. In the formula, E For the essential matrix, K This is the intrinsic parameter matrix.

[0058] right E Perform singular value decomposition to obtain the rotation matrix R With translation vector T For example, in this embodiment, the confidence level of each matching pair is... Construct a diagonal confidence matrix And perform nonlinear minimization based on weighted projection error: This can be understood as follows: the optimization algorithm employs Levenberg-Marquardt nonlinear least squares; the rotation uses Lie algebra. Figure 6 Parameterization (Rodriguez formula), iteration termination threshold is 10 -6 Or 50 times; optimal output R , T (like Figure 1 As shown), in the formula, This represents the coordinates of the i-th observed pixel in the image. For the first The coordinates of a point in three-dimensional space in the world coordinate system Represented as an attitude projection matrix, This is the intrinsic parameter matrix.

[0059] In some embodiments, Figure 7 Step S104 specifically includes: like Figure 1 As shown, the propagation path of light at the air-glass-water interface is corrected according to Snell's law. The refraction direction in vector form is: Where d is the incident unit direction and n is the interface normal unit vector. This represents the unit vector of the normal at the air-glass interface. This represents the unit vector of the glass-water interface normal.

[0060] The ray beams after refraction correction for the two cameras (or two adjacent frames) are represented as follows: In the formula, and This represents the three-dimensional coordinates of the camera's optical center in a unified coordinate system corresponding to the two imaging operations. and This represents the scalar distance parameter originating from the camera's optical center along its respective ray. a and b This represents the unit ray direction vector after in-reference back projection and interface refraction correction according to Snell's law.

[0061] Solve using the closed-form solution with the shortest connection: Let the target point This solution is equivalent to the triangulation solution with the minimum distance between the two refracted correction rays in the least squares sense, and its z-component in camera coordinates is the depth.

[0062] For example, in this embodiment, the refractive index of the underwater medium ranges from 1.33 to 1.36, and the refractive compensation coefficient is obtained through experimental calibration.

[0063] In some embodiments, Figure 8 Step S105 specifically includes: like Figure 1 As shown, let the depth measured by refraction correction triangulation be D. r The dense depth output by the monocular depth estimation network MiDaS is D. m First, in the effective mask Perform scale normalization and spatial alignment within the internal space, and calculate the linear alignment parameter (s, a): The closed-form solution is obtained from the normal equation, and the aligned solution is obtained. .

[0064] For example, in this embodiment, pixel-level fusion is performed using a confidence-weighted average method: in Based on the geometric condition number and ray angle settings in S104, The parameters are set based on the response intensity and local stability estimation of MiDaS. Then, scale recovery and 3D coordinate inversion are performed by combining attitude parameters and intrinsic matrix. The inversion calculation formula is as follows: In the formula, R Represents the rotation matrix. Represents the column coordinates (horizontal) of a pixel on the image plane. Represents the row coordinates (vertical) of a pixel on the image plane.

[0065] In some embodiments, Figure 9 Step S106 includes: The confidence score of the localization result is calculated based on the matching residual variance, depth stability, and image quality metrics (PSNR, SSIM), and a report is output. The matching residual variance is defined as follows: Depth stability is measured by the mean of the fused depth variance within the sliding window: Image quality metrics PSNR and SSIM are calculated. Confidence values ​​are linearly weighted and then mapped to [0, 1] using a Sigmoid algorithm: The output localization results include the target's 3D coordinates, pose matrix, root mean square depth error (RMSE), matching success rate, and image sharpness index, and generate a confidence assessment report.

[0066] This invention also provides an underwater target recognition and localization system based on a fusion of physical models and deep learning, such as... ​As shown, the underwater target recognition and localization system based on physical model and deep learning fusion includes: a preprocessing module 100, a feature extraction and matching module 200, a pose solving and optimization module 300, a refraction correction module 400, a deep fusion and spatial localization module 500, and a confidence assessment and result output module 600. The preprocessing module 100 is used to acquire underwater environmental parameters and perform detail compensation, color correction, and contrast enhancement on the acquired underwater images to obtain high-fidelity underwater images. The feature extraction and matching module 200 is used to extract multi-scale key points of the high-fidelity underwater images using the SuperPoint deep feature detection network, construct local feature vectors using the R2D2 descriptor, calculate the joint similarity of color, texture, and gradient, and select the feature point pairs with the highest confidence as high-quality matching points. The pose solving and optimization module 300 is used to remove abnormal matching points using the weighted RANSAC algorithm, and based on the high-quality matching point pairs, it uses... The system employs an eight-point method to estimate the fundamental matrix, combines it with the camera intrinsic parameter matrix to solve for the essential matrix, performs singular value decomposition on the essential matrix to obtain the rotation matrix and translation vector, and minimizes the reprojection error using a confidence matrix to obtain the optimal attitude parameters. A refraction correction module 400 corrects the light propagation path according to Snell's law, establishes a set of refraction correction triangulation equations, and calculates the depth value of the target point using the least squares method. A depth fusion and spatial positioning module 500 performs pixel-level fusion of the depth value with the dense depth map output by the monocular depth estimation network MiDaS, combines the optimal attitude parameters and intrinsic parameter matrix to perform scale recovery and 3D coordinate inversion, and obtains the target's spatial position in the world coordinate system. A confidence assessment and result output module 600 calculates the confidence value of the positioning result based on the matching residual variance, depth stability, and image quality indices (PSNR and SSIM), and outputs the positioning result and a confidence assessment report.

[0067] This invention also provides a computer-readable storage medium, which is a non-volatile or non-transient storage medium, storing a computer program thereon. When the computer program is executed by a processor, it implements the steps of the adaptive monocular vision underwater target recognition and localization method based on the fusion of physical models and deep learning described above.

[0068] This invention also provides a device for underwater target identification and localization, including a memory and a processor. The memory stores a computer program that can run on the processor. When the processor runs the computer program, it executes the steps of the adaptive monocular vision underwater target identification and localization method based on the fusion of physical models and deep learning described above.

[0069] It should be noted that this application is not limited to the above-described embodiments. The above embodiments are merely examples, and any embodiments with the same structure and effect as the technical concept within the scope of this application are included in the technical scope of this application. Furthermore, various modifications that can be conceived by those skilled in the art to the embodiments, and other ways of constructing by combining some of the constituent elements of the embodiments, without departing from the spirit of this application, are also included in the scope of this application.

Claims

1. A method for underwater target identification and localization based on the fusion of physical models and deep learning, characterized in that, include: The acquired images are preprocessed: underwater environmental parameters are acquired, and detail compensation, color correction and contrast enhancement are performed on the acquired underwater images to obtain high-fidelity underwater images; Feature extraction and matching: The SuperPoint deep feature detection network is used to extract multi-scale key points of the high-fidelity underwater image. The R2D2 descriptor is used to construct local feature vectors, and the joint similarity of color, texture and gradient is calculated. The feature point pair with the highest confidence is selected as the high-quality matching point. Attitude solution and optimization: The weighted RANSAC algorithm is used to remove abnormal matching points. The fundamental matrix is ​​estimated using the eight-point method based on high-quality matching point pairs. The essential matrix is ​​solved by combining the camera intrinsic parameter matrix. The essential matrix is ​​decomposed by singular value decomposition to obtain the rotation matrix and translation vector. The reprojection error is minimized by weighting the confidence matrix to obtain the optimal attitude parameters. Refraction Corrected Triangulation: Based on Snell's Law, the light propagation path is corrected, a set of refraction corrected triangulation equations is established, and the depth value of the target point is calculated using the least squares method. Deep Fusion and Spatial Positioning: The depth value is fused with the dense depth map output by the monocular depth estimation network MiDaS at the pixel level. The optimal pose parameters and intrinsic parameter matrix are combined to perform scale recovery and three-dimensional coordinate inversion to obtain the spatial position of the target in the world coordinate system. Confidence assessment and result output: Calculate the confidence value of the localization result based on the matching residual variance, depth stability and image quality indicators, and output the localization result and confidence assessment report. The image quality indicators include peak signal-to-noise ratio PSNR and structural similarity index SSIM.

2. The underwater target identification and localization method based on the fusion of physical model and deep learning according to claim 1, characterized in that, The preprocessing of the acquired image includes: Environmental adaptive calibration: Acquire calibration plate images under different water depths and lighting conditions, calculate the refractive index and attenuation coefficient of the medium based on the underwater light propagation model, correct the camera intrinsic parameter matrix and distortion coefficient, and establish an environmental adaptive calibration parameter set; Imaging restoration and enhancement: An underwater imaging degradation model is constructed. Based on the prior estimation of transmittance and background light in the dark channel, a convolutional neural network image enhancement model is used to perform detail compensation, color correction and contrast enhancement on the restored image to obtain the high-fidelity underwater image.

3. The underwater target identification and localization method based on the fusion of physical model and deep learning according to claim 2, characterized in that, The establishment of the environmental adaptive calibration parameter set includes: By measuring environmental parameters in real time using light and turbidity sensors, and dynamically updating the camera's focal length and principal point position based on interpolation calculations, the calibration parameters can be automatically corrected.

4. The underwater target identification and localization method based on the fusion of physical model and deep learning according to claim 2, characterized in that, The convolutional neural network image enhancement model is an improved WaterNet network, which uses attention-guided convolutional modules to extract channel features and utilizes residual block structures to enhance texture details and suppress over-enhancement.

5. The underwater target identification and localization method based on the fusion of physical model and deep learning according to claim 1, characterized in that, In the process of minimizing the reprojection error using the confidence matrix to obtain the optimal attitude parameters, the objective function for optimizing the weighted projection error is: in, For the first The confidence weight of the matching points Let be the attitude projection matrix.

6. The underwater target identification and localization method based on the fusion of physical model and deep learning according to claim 1, characterized in that, The process of obtaining the rotation matrix and translation vector by performing singular value decomposition on the essential matrix includes: The Levenberg–Marquardt algorithm is used to perform nonlinear least squares optimization on the weighted projection error, and the rotation matrix and the translation vector are output.

7. The underwater target identification and localization method based on the fusion of physical model and deep learning according to claim 1, characterized in that, The deep fusion and spatial positioning include: The depth output of the MiDaS network, after scale normalization and spatial alignment, is fused with the refraction-corrected depth using a confidence-weighted average.

8. The underwater target identification and localization method based on the fusion of physical model and deep learning according to claim 1, characterized in that, The confidence assessment step outputs the localization results, including the target's three-dimensional coordinates, attitude matrix, root mean square value of depth error, matching success rate, and image sharpness index.

9. A system for underwater target identification and localization based on the fusion of physical models and deep learning, characterized in that, include: The preprocessing module is used to acquire underwater environmental parameters and perform detail compensation, color correction and contrast enhancement on the acquired underwater images to obtain high-fidelity underwater images. The feature extraction and matching module is used to extract multi-scale key points of the high-fidelity underwater image using the SuperPoint deep feature detection network, construct local feature vectors using R2D2 descriptors, calculate the joint similarity of color, texture and gradient, and select the feature point pair with the highest confidence as high-quality matching points. The attitude solution and optimization module is used to remove abnormal matching points using the weighted RANSAC algorithm, estimate the fundamental matrix using the eight-point method based on high-quality matching point pairs, solve the essential matrix by combining the camera intrinsic parameter matrix, obtain the rotation matrix and translation vector by performing singular value decomposition on the essential matrix, and obtain the optimal attitude parameters by weighting and minimizing the reprojection error through the confidence matrix. The refraction correction module is used to correct the light propagation path according to Snell's law, establish a set of refraction correction trigonometric equations, and calculate the depth value of the target point using the least squares method. The depth fusion and spatial positioning module is used to perform pixel-level fusion of the depth value with the dense depth map output by the monocular depth estimation network MiDaS, and to perform scale recovery and three-dimensional coordinate inversion by combining the optimal attitude parameters and intrinsic parameter matrix to obtain the spatial position of the target in the world coordinate system. The confidence assessment and result output module is used to calculate the confidence value of the localization result based on the matching residual variance, depth stability and image quality indicators (PSNR and SSIM), and output the localization result and confidence assessment report.

10. A computer-readable storage medium, characterized in that, It stores a computer program, which, when run on a processor, executes all the steps of the underwater target identification and localization method based on the fusion of physical model and deep learning as described in any one of claims 1 to 8.

Images