An image enhancement and pose estimation method, device, storage medium, product and apparatus for underwater closed space scenarios

Abstract

The application discloses an image enhancement and pose estimation method, equipment, storage medium, product and device for an underwater closed space scene, and belongs to the underwater visual image processing and robot vision technical field. Considering a dark environment or a slightly turbid water body scene of a closed underwater operation, color correction processing and contrast enhancement processing are performed on collected original underwater images; and visual perception fusion is performed to obtain a final enhanced image. A pose estimation network is used to detect two-dimensional coordinates and corresponding confidence of each joint of a mechanical arm in the final enhanced image under each visual angle. Then, a confidence weighted triangulation method is used to solve three-dimensional coordinates of each joint of the mechanical arm, so that three-dimensional pose estimation of the mechanical arm is completed. The underwater image enhancement algorithm is used to perform color correction and definition improvement on the underwater image, significantly improves the visual monitoring definition and accuracy of the underwater mechanical arm, and provides stable and reliable visual input for underwater operation.

Description

Technical Field

[0001] This invention relates to the fields of underwater visual image processing and robot vision technology, and in particular to a method, device, storage medium, product, and apparatus for image enhancement and attitude estimation in underwater confined space scenarios. Background Technology

[0002] With the development of marine resource development, underwater engineering operation and maintenance, and underwater special operation technologies, underwater robotic arms are widely used in underwater assembly, equipment maintenance, and hydraulic structure inspection. Visual monitoring systems are the core sensing units for remote control and autonomous operation of underwater robotic arms. In the closed underwater environment of a pressure vessel, the pressure can reach 30 MPa, and the water is slightly turbid with extremely low ambient light. Furthermore, the water exhibits spectral selective absorption and scattering effects on light, leading to severe color distortion in images captured by underwater cameras, such as a bluish or greenish tint, low contrast, blurred details, and significant noise. This directly affects the accuracy of subsequent target detection and attitude estimation. Simultaneously, the refraction, reflection, and partial occlusion of the underwater environment place extremely high demands on the real-time performance, accuracy, and robustness of the robotic arm's 3D attitude estimation.

[0003] For underwater image enhancement technology, existing solutions mainly fall into two categories: traditional image processing methods and deep learning-based methods. Traditional methods, such as white balance algorithms, linear contrast stretching, and histogram equalization, can only achieve simple color and brightness adjustments and cannot adapt to the complex light attenuation and scattering characteristics underwater. They are prone to problems such as image oversaturation, secondary color distortion, and loss of detail, and the enhancement effect is extremely poor in slightly murky and dark environments. If underwater image enhancement methods based on convolutional neural networks (CNNs) are adopted, relying on large-scale paired underwater degradation and clear image training data, it will also lead to the extreme difficulty and high cost of obtaining paired labeled data of real underwater scenes, resulting in weak model generalization ability and unstable enhancement effects across scenes. In addition, existing enhancement methods based on generative adversarial networks (GANs) do not fully integrate the physical model of underwater optical imaging and only achieve image enhancement through data-driven methods. This is prone to problems such as the enhanced image not conforming to the physical imaging law of underwater and content distortion. Moreover, the model has a large computational load and cannot meet the real-time processing requirements of multi-camera synchronous acquisition scenarios.

[0004] For the three-dimensional attitude estimation technology of underwater robotic arms, existing solutions are mainly divided into monocular vision method, multi-view stereo vision method and lidar method. LiDAR methods are costly and produce sparse point clouds, making it impossible to achieve fine-grained attitude estimation of robotic arm joints. Monocular vision methods rely on prior target models, resulting in extremely low accuracy in underwater occlusion and uneven lighting scenarios, failing to meet operational requirements. Multi-view stereo vision is currently the mainstream solution, but existing technologies have significant drawbacks: First, 2D joint detection often uses a static bin-assignment coordinate classification strategy, which is prone to missing large instance key points and having large quantization errors for small instance key points in dense prediction scenarios, leading to a significant drop in detection accuracy in low-quality underwater images. Second, 3D reconstruction often uses ordinary triangulation methods, assuming consistent weights for observation data from each viewpoint, without considering the low observation quality of some viewpoints caused by underwater refraction, reflection, and occlusion, easily introducing significant systematic errors and resulting in large deviations in 3D attitude estimation results. Third, existing algorithms struggle to balance real-time performance and accuracy; high-precision models have high inference latency, failing to meet the real-time control requirements of underwater robotic arms, while lightweight models cannot guarantee the accuracy of joint detection.

[0005] In summary, existing technologies cannot simultaneously meet the dual requirements of real-time performance, naturalness, and physical realism of underwater image enhancement in dark environments and slightly turbid waters, as well as the high precision, robustness, and low latency of 3D pose estimation for robotic arms. This severely restricts the engineering application of underwater robotic arm visual monitoring systems. Summary of the Invention

[0006] The present invention aims to solve the above-mentioned problems and thereby provides an image enhancement and attitude estimation method, device, storage medium, product and apparatus for underwater confined space scenarios.

[0007] The technical solution adopted by this invention to solve the above-mentioned technical problems is: an image enhancement and attitude estimation method, device, storage medium, product, and apparatus for underwater confined space scenarios, comprising the following steps:

[0008] S1: Perform adaptive channel statistical color correction processing on the acquired raw underwater images based on minimum color loss constraints;

[0009] S2: Perform LAB layered adaptive contrast enhancement processing with minimal color loss on the color-corrected image;

[0010] S3: Perform visual perception fusion based on gradient fusion, chromaticity similarity, and color loss constraints on the enhanced image to obtain the final enhanced image;

[0011] S4: Input the final enhanced image into the RTMO two-dimensional pose estimation network to detect the two-dimensional coordinates and corresponding confidence scores of each joint of the robotic arm under each viewpoint.

[0012] S5: Based on the intrinsic and extrinsic parameters of each camera obtained from the joint calibration of multiple cameras, combined with the two-dimensional coordinates and confidence scores of the joints at each viewpoint, the three-dimensional coordinates of each joint of the robotic arm are obtained by using the confidence-weighted triangulation method, thus completing the three-dimensional attitude estimation of the robotic arm.

[0013] Furthermore, the specific implementation process of the adaptive channel statistical color correction processing based on the minimum color loss constraint described in S1 is as follows:

[0014] S1.1: Calculate the mean values ​​of the red channel R, green channel G, and blue channel B of the original underwater image, respectively. and variance And construct a normalized correction model based on statistical features:

[0015]

[0016] in, This represents the pixel value of the corresponding color channel in the original image. This represents the channel image after initial color correction;

[0017] S1.2: Using the green channel G as a reference channel, construct adaptive compensation factors for the red channel R and the blue channel B;

[0018] S1.3: Based on the adaptive compensation factor, perform dynamic compensation on the red channel R and blue channel B, which have severe attenuation, to obtain the compensated red channel R and blue channel B;

[0019] S1.4: Convert the compensated image to linear RGB space, construct a color adaptation matrix, and further correct the image color under different lighting conditions. The expression is as follows:

[0020]

[0021]

[0022]

[0023]

[0024] in, This represents the compensated RGB image. Represents a linear spatial image. Represents the gamma coefficient. Represents the reference light source matrix. Represents the target light source matrix. This represents the output image after color correction.

[0025] Furthermore, the specific implementation process of the LAB layered adaptive contrast enhancement processing with minimal color loss described in S2 is as follows:

[0026] S2.1: The color-corrected image Convert to the LAB color space, which separates luminance and chrominance, to obtain the luminance component. and the first chromaticity component Second chromaticity component ;

[0027] S2.2: For the brightness component Local contrast enhancement is performed, and a multi-scale decomposition method is used to extract the base layer and detail layer. First, adaptive histogram equalization is applied to the luminance component, and then a smoothing filter operator is used to decompose the two base layers, resulting in:

[0028]

[0029]

[0030] in, As the first basic layer, As the second basic layer, and This indicates a smoothing and sharpening integrated filtering operation;

[0031] S2.3: Based on the first base layer and the second base layer, construct the detail layer information of the luminance component to obtain:

[0032]

[0033]

[0034] in, This represents the first level of detail. The second detail layer is represented by the separation of large-scale brightness information and small-scale texture information in the image by decomposing the brightness component into a base layer and a detail layer.

[0035] S2.4: Transfer the second base layer With the first detail layer Second layer of detail Reconstruction is performed; simultaneously, the chromaticity components are... and Adaptive balancing adjustments are performed using the following formulas:

[0036] when hour,

[0037]

[0038] when hour,

[0039]

[0040] in, and These represent the chromaticity components in the LAB color space. and These represent the adjusted chroma components. Finally, the enhanced luminance component and the adjusted chroma components are converted back to RGB space to obtain the image with enhanced contrast. .

[0041] Furthermore, the specific implementation process of the visual perception fusion processing constrained by gradient fusion, chromaticity similarity, and color loss described in S3 is as follows:

[0042] S3.1: Acquire the original image, the color-corrected image, and the contrast-enhanced image respectively, and convert them to a luminance and chrominance separation space;

[0043] S3.2: The Sobel operator is used to calculate the gradient magnitude of the luminance component to obtain the image gradient information;

[0044] S3.3: Calculate the gradient similarity between the color-corrected image and the original image, and between the contrast-enhanced image and the original image, respectively. The expression is as follows:

[0045]

[0046] When calculating the original image and the color-corrected image, The gradient of the original image. To correct the color gradient of the image; when calculating the original image and the contrast-enhanced image, The gradient of the original image. To enhance image gradient for contrast, Indicates the gradient similarity between two images;

[0047] S3.4: Calculate the color similarity of the image based on the chromaticity components. Its expression is:

[0048]

[0049] When calculating the original image and the color-corrected image, , These are the two-channel chroma values ​​of the original image. , To correct the dual-channel chromaticity values ​​of the color-corrected image; when calculating the original image and the contrast-enhanced image, , These are the two-channel chroma values ​​of the original image. , Enhance the dual-channel chroma values ​​of the image to improve contrast; , These represent the values ​​of the two images on the first chromaticity component. , These represent the values ​​of the two images on the second chromaticity component, respectively.

[0050] S3.5: Construct a comprehensive similarity score based on gradient similarity and color similarity:

[0051]

[0052] The softmax function is then used to calculate the fusion weights between the color-corrected image and the contrast-enhanced image. and ;

[0053] S3.6: Based on the fusion weights, the color correction result and the contrast enhancement result are weighted and fused to obtain the final enhanced image:

[0054]

[0055] in, This represents the final enhanced image output.

[0056] Furthermore, the specific implementation process of the three-dimensional pose estimation method for the robotic arm described in S5 is as follows:

[0057] S5.1: Obtain the intrinsic parameter matrix, distortion coefficient, extrinsic parameter matrix between cameras, and projection matrix P for each underwater camera;

[0058] S5.2: The feature map output by the backbone is processed by the hybrid encoder to obtain the sampled feature map; a score and pose feature are generated for each grid cell to predict the bounding box, key point coordinates and visibility;

[0059] S5.3: Expand the bounding box to cover all relevant nodes, and divide it evenly along the horizontal and vertical axes respectively. indivual, There are 10 bins, each corresponding to a coordinate interval. The x-coordinate of each horizontal bin is calculated as follows:

[0060]

[0061] in, and These represent the left and right sides of the bounding box, respectively, and the value of index i ranges from 1 to... The bin method on the y-axis is the same;

[0062] S5.4: Encode the coordinates of each bin to generate a bin-specific position representation, and optimize the position encoding by linear transformation through a fully connected layer;

[0063] S5.5: Extract key point features from pose features and optimize them. Multiply the optimized key point features with the position representation of each bin to calculate the probability heatmap of key point occurrence in each bin.

[0064] S5.6: Obtain the two-dimensional coordinates of each joint of the robotic arm through the probability heatmap decoding, and output the confidence score of each joint.

[0065] S5.7: Input the enhanced underwater images from each viewpoint into the trained RTMO network, set a score threshold of 0.1 and perform grid filtering with non-maximum suppression (NMS), and decode to obtain the two-dimensional coordinates of each joint of the robotic arm from each viewpoint. With the corresponding confidence level , where i is the camera number and j is the key point number.

[0066] Furthermore, the specific implementation process of the three-dimensional pose reconstruction of the robotic arm based on the confidence-weighted triangulation method described in S5 is as follows:

[0067] S5.8: Set a confidence threshold δ for each joint point j. For the j-th joint point, only retain the confidence scores from each viewpoint. Two-dimensional coordinate observations are used to eliminate invalid observation data with low confidence, obstruction, or interference;

[0068] S5.9: For the j-th joint point, let its homogeneous coordinates in three-dimensional space be... For each effective observation angle i, its constituent matrix is ​​a 3×4 camera projection matrix P. i The homogeneous coordinates of the two-dimensional observation are Based on the camera projection relationship, we have:

[0069]

[0070] The two linear equations corresponding to each effective viewpoint are combined to form a 2×4 coefficient matrix A. i The coefficient matrices of all valid viewpoints are stacked to form an over-constrained system of linear equations:

[0071] ;

[0072] S5.10: Assign weights to the linear equation for each effective viewpoint i That is, the weights are positively correlated with the confidence of the key points from this perspective. Applying the weights to the coefficient matrix A yields the weighted coefficient matrix A. wReduce the impact of low-quality observations on 3D reconstruction;

[0073] S5.11: For the weighted coefficient matrix A w Perform singular value decomposition (SVD) to obtain the homogeneous coordinates X. j Dividing the homogeneous coordinates by its fourth element yields the three-dimensional Euclidean coordinates of the joint point j. ;

[0074] S5.12: Traverse all nodes of the robotic arm and repeat steps 5.8 to 5.11 above to obtain the three-dimensional coordinates of all joints. Combined with the kinematic model of the robotic arm, complete the estimation and pose calculation of the three-dimensional posture of the robotic arm.

[0075] An image enhancement and attitude estimation device for underwater confined space scenarios includes a memory, a processor, and a computer optimization program stored in the memory, wherein the processor executes the computer program to implement the steps of an image enhancement and attitude estimation method for underwater confined space scenarios.

[0076] A medium comprising image enhancement and attitude estimation for underwater confined space scenes, wherein a computer program is stored thereon, which, when executed by a processor, implements the steps of an image enhancement and attitude estimation method for underwater confined space scenes.

[0077] An image enhancement and attitude estimation product for underwater confined space scenarios includes a computer program that, when executed by a processor, implements steps for image enhancement and attitude estimation methods for underwater confined space scenarios.

[0078] An image enhancement and attitude estimation device for underwater confined space scenarios is provided, comprising an underwater image enhancement submodule and a robotic arm attitude estimation submodule. When the underwater image enhancement submodule is executed, it implements steps S1 to S3 of the image enhancement and attitude estimation method for underwater confined space scenarios. When the robotic arm attitude estimation submodule is executed, it implements steps S4 to S5 of the method of claim 1.

[0079] Compared with the prior art, the present invention has the following advantages:

[0080] This invention employs an underwater image enhancement algorithm based on a three-stage architecture of color correction, contrast enhancement, and visual perception fusion to correct the color and improve the clarity of underwater images, effectively solving the problems of color distortion and blurred details in underwater images. Based on RTMO two-dimensional attitude estimation and multi-view confidence-weighted triangulation, it performs high-precision, low-latency three-dimensional attitude estimation for the robotic arm, addressing the high latency, low accuracy, and weak robustness issues of existing technologies in special underwater scenarios. This invention can output high-quality enhanced images and robotic arm attitude estimation results in confined underwater dark environments and slightly turbid water scenarios, significantly improving the clarity and accuracy of underwater robotic arm visual monitoring and providing stable and reliable visual input for underwater operations. Attached Figure Description

[0081] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an undue limitation of the invention. In the drawings:

[0082] Figure 1 This is a flowchart illustrating the principle and flowchart of the underwater image enhancement algorithm of this invention.

[0083] Figure 2 The image before enhancement is shown using the underwater image enhancement algorithm of this invention.

[0084] Figure 3 This is a comparison of the effects of enhancing images using the underwater image enhancement algorithm of this invention;

[0085] Figure 4 This is a flowchart illustrating the principle of the robotic arm posture estimation algorithm of the present invention.

[0086] Figure 5 This is a schematic diagram illustrating the input of the robotic arm pose estimation algorithm of the present invention to perform pose estimation on an image;

[0087] Figure 6 This is a comparison chart showing the output results of the robot arm pose estimation algorithm of the present invention for pose estimation of images;

[0088] Figure 7 This is a graph showing the average image quality metric after enhancement by the underwater image enhancement algorithm of this invention;

[0089] Figure 8 This is a diagram showing the average joint position error of the robotic arm after attitude estimation by the robotic arm attitude estimation algorithm of this invention. Detailed Implementation

[0090] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. It should be noted that, unless otherwise specified, the embodiments and features in the embodiments of the present invention can be combined with each other, and the described embodiments are only some embodiments of the present invention, not all embodiments.

[0091] See appendix Figure 1 and Figure 4 This embodiment describes an image enhancement and attitude estimation method, device, storage medium, product, and apparatus for underwater confined space scenarios, comprising the following steps:

[0092] S1: Perform adaptive channel statistical color correction processing on the acquired raw underwater images based on minimum color loss constraints. Construct an adaptive compensation model based on the statistical characteristics of each color channel of the image, and dynamically compensate for the color channels that are greatly affected by water absorption to correct the color distortion of the underwater images.

[0093] S2: Perform LAB layered adaptive contrast enhancement processing with minimal color loss on the color-corrected image. By converting the image to a color space that separates luminance and chrominance, multi-scale decomposition of the luminance component is performed to extract the base layer and detail layer information of the image. Based on local statistical characteristics, the detail information is enhanced to improve image clarity and sense of depth.

[0094] S3: Perform visual perception fusion of gradient, chromaticity similarity and color loss constraints on the enhanced image. Construct fusion weights based on the gradient information and color similarity of the image. Perform weighted fusion of color correction results and contrast enhancement results to obtain a final enhanced image that achieves a balance between color naturalness and detail representation.

[0095] S4: Input the enhanced underwater image into the RTMO two-dimensional attitude estimation network to detect the two-dimensional coordinates and corresponding confidence scores of each joint of the robotic arm from each viewpoint.

[0096] S5: Based on the intrinsic and extrinsic parameters of each camera obtained from the joint calibration of multiple cameras, combined with the two-dimensional coordinates and confidence scores of the joints at each viewpoint, the three-dimensional coordinates of each joint of the robotic arm are obtained by using the confidence-weighted triangulation method, thus completing the three-dimensional attitude estimation of the robotic arm.

[0097] The specific implementation process of the adaptive channel statistical color correction processing based on the minimum color loss constraint described in S1 is as follows:

[0098] S1.1: Calculate the mean values ​​of the red channel R, green channel G, and blue channel B of the original underwater image, respectively. and variance And construct a normalized correction model based on statistical features:

[0099]

[0100] in, This represents the pixel value of the corresponding color channel in the original image. This represents the channel image after initial color correction;

[0101] S1.2: Using the green channel G as the reference channel, construct adaptive compensation factors for the red channel R and the blue channel B, respectively:

[0102]

[0103]

[0104] in, , , These represent the red, green, and blue channels of the image after initial color correction, respectively. To prevent the stability constant from being zero; underwater water exhibits wavelength-selective absorption of visible light, with red light attenuating the fastest, followed by blue light, and green light experiencing the lowest transmission loss. The mean value of the green channel image is closest to the natural imaging benchmark without attenuation. Based on the minimum color loss theory, this invention selects the green channel with optimal transmission as the color benchmark. By constructing an adaptive compensation factor through the variance difference between the red and blue channels relative to the green channel, it can minimize the overall color deviation loss of the RGB three channels and avoid color oversaturation and color cast problems caused by global uniform compensation.

[0105] S1.3: Based on the aforementioned adaptive compensation factor, dynamic compensation is performed on the red channel R and blue channel B, which suffer from severe attenuation, to obtain the compensated red channel R and blue channel B:

[0106]

[0107]

[0108] in, and The red channel R and blue channel B are respectively represented after compensation. Through the above compensation, the color channel that is more affected by water absorption is brought closer to the green channel G, thereby improving the problem of the image being too blue or too green.

[0109] S1.4: Convert the compensated image to linear RGB space, construct a color adaptation matrix, and further correct the image color under different lighting conditions. The expression is as follows:

[0110]

[0111]

[0112]

[0113]

[0114] in, This represents the compensated RGB image. Represents a linear spatial image. Represents the gamma coefficient. Represents the reference light source matrix. Represents the target light source matrix. This represents the output image after color correction.

[0115] The specific implementation process of the LAB layered adaptive contrast enhancement processing with minimal color loss described in S2 is as follows:

[0116] S2.1: The color-corrected image Convert to the LAB color space, which separates luminance and chrominance, to obtain the luminance component. and the first chromaticity component Second chromaticity component Among them, the brightness component The first chromaticity component is used for subsequent detail enhancement processing. Second chromaticity component Used for color balance adjustment; existing global LAB enhancement schemes uniformly process brightness and chromaticity, without taking into account the residual color shift characteristics after minimum color loss correction; this invention separates brightness and chromaticity for independent optimization, performs layered detail enhancement on brightness separately, and adaptively balances chromaticity based on the color loss difference, avoiding the superposition of secondary color distortion during the enhancement process.

[0117] S2.2: For the brightness component Local contrast enhancement is performed, and a multi-scale decomposition method is used to extract the base layer and detail layer. First, adaptive histogram equalization is applied to the luminance component, and then a smoothing filter operator is used to decompose the two base layers, resulting in:

[0118]

[0119]

[0120] in, As the first basic layer, As the second basic layer, and This indicates a smoothing and sharpening integrated filtering operation; a dual-layer SSIF smoothing and sharpening integrated filtering is used to extract the base layer layer by layer, which can specifically suppress global noise caused by air bubbles and suspended particles in closed underwater environments under the constraint of minimal color loss, while preserving the fine joint texture of the robotic arm.

[0121] S2.3: Based on the first base layer and the second base layer, construct the detail layer information of the luminance component to obtain:

[0122]

[0123]

[0124] in, This represents the first level of detail. The second detail layer is represented by the separation of large-scale brightness information and small-scale texture information in the image by decomposing the brightness component into a base layer and a detail layer.

[0125] S2.4: Transfer the second base layer With the first detail layer Second layer of detail Reconstruction is performed; simultaneously, the chromaticity components are... and Adaptive balancing adjustments are performed using the following formulas:

[0126] when hour,

[0127]

[0128] when hour,

[0129]

[0130] in, and These represent the chromaticity components in the LAB color space. and These represent the adjusted chroma components. Finally, the enhanced luminance component and the adjusted chroma components are converted back to RGB space to obtain the image with enhanced contrast. .

[0131] The specific implementation process of the visual perception fusion processing constrained by gradient fusion, chromaticity similarity, and color loss described in S3 is as follows:

[0132] S3.1: Acquire the original image, the color-corrected image, and the contrast-enhanced image respectively, and convert them to a luminance and chrominance separation space; wherein, the luminance component is used to calculate structural and texture information, and the chrominance component is used to characterize color similarity;

[0133] S3.2: The Sobel operator is used to calculate the gradient magnitude of the luminance component to obtain the image gradient information.

[0134]

[0135]

[0136]

[0137] in, and These represent the response results obtained using the horizontal and vertical Sobel operators, respectively. Indicates the magnitude of the image gradient;

[0138] S3.3: Calculate the gradient similarity between the color-corrected image and the original image, and between the contrast-enhanced image and the original image, respectively. The expression is as follows:

[0139]

[0140] When calculating the original image and the color-corrected image, The gradient of the original image. To correct the color gradient of the image; when calculating the original image and the contrast-enhanced image, The gradient of the original image. To enhance image gradient for contrast, Indicates the gradient similarity between two images;

[0141] S3.4: Calculate the color similarity of the image based on the chromaticity components. Its expression is:

[0142]

[0143] When calculating the original image and the color-corrected image, , These are the two-channel chroma values ​​of the original image. , To correct the dual-channel chromaticity values ​​of the color-corrected image; when calculating the original image and the contrast-enhanced image, , These are the two-channel chroma values ​​of the original image. , Enhance the dual-channel chroma values ​​of the image to improve contrast; , These represent the values ​​of the two images on the first chromaticity component. , These represent the values ​​of the two images on the second chromaticity component, respectively.

[0144] S3.5: Construct a comprehensive similarity score based on gradient similarity and color similarity:

[0145]

[0146] The softmax function is then used to calculate the fusion weights between the color-corrected image and the contrast-enhanced image. and ;

[0147] S3.6: Based on the fusion weights, the color correction result and the contrast enhancement result are weighted and fused to obtain the final enhanced image:

[0148]

[0149] in, This represents the final enhanced image output.

[0150] The specific implementation process of the robotic arm three-dimensional pose estimation method described in S5 is as follows:

[0151] S5.1: Complete the multi-camera joint calibration. Use the Charuco chessboard-based multi-camera hierarchical joint calibration method to obtain the intrinsic parameter matrix, distortion coefficient, extrinsic parameter matrix and projection matrix P of each underwater camera, and ensure that the calibration reprojection error is ≤0.55 pixels.

[0152] S5.2: Adopts a YOLO-like architecture with CSPDarknet as the backbone network. The feature maps output by the backbone are processed by a hybrid encoder to obtain 16x downsampled feature maps and 32x downsampled feature maps. The network head generates scores and pose features for each grid cell through dual convolutional blocks, which are used to predict bounding boxes, key point coordinates and visibility.

[0153] S5.3: Regress the bounding box of the robotic arm instance using pointwise convolutional layers, expand the bounding box by a factor of n to cover all relevant nodes, and then uniformly divide the expanded bounding box along the horizontal and vertical axes into... indivual, There are 10 bins, each corresponding to a coordinate interval. The x-coordinate of each horizontal bin is calculated as follows:

[0154]

[0155] in, and These represent the left and right sides of the bounding box, respectively, and the value of index i ranges from 1 to... The bin method on the y-axis is similar;

[0156] S5.4: Sine position encoding is used to encode the coordinates of each bin, generating bin-specific position representations. The position encoding is then optimized using a fully connected layer through a linear transformation. The sine position encoding is defined as:

[0157]

[0158] Where t represents the scale, c is the index, and C represents the total number of dimensions of the position encoding;

[0159] S5.5: Extract key point features from pose features, optimize key point features through gated attention unit (GAU), multiply the optimized key point features with the position representation of each bin, and calculate the probability heatmap of key point occurrence in each bin using softmax.

[0160] S5.6: Integrate the probability heatmap, decode it to obtain the two-dimensional coordinates of each joint of the robotic arm, and output the confidence score of each joint.

[0161] S5.7: Input the enhanced underwater images from each viewpoint into the trained RTMO network, set a score threshold of 0.1 and perform grid filtering with non-maximum suppression (NMS), and decode to obtain the two-dimensional coordinates of each joint of the robotic arm from each viewpoint. With the corresponding confidence level , where i is the camera number and j is the key point number.

[0162] The specific implementation process of the robotic arm's 3D pose reconstruction based on the confidence-weighted triangulation method described in S5 is as follows:

[0163] S5.8: Set a confidence threshold δ for each joint point j. For the j-th joint point, only retain the confidence scores from each viewpoint. Two-dimensional coordinate observations are used to eliminate invalid observation data with low confidence, obstruction, or interference;

[0164] S5.9: For the j-th joint point, let its homogeneous coordinates in three-dimensional space be... For each effective observation angle i, its constituent matrix is ​​a 3×4 camera projection matrix P. i The homogeneous coordinates of the two-dimensional observation are Based on the camera projection relationship, we have:

[0165]

[0166] Expanding the cross product yields two linear equations for each effective viewpoint, forming a 2×4 coefficient matrix A. i The coefficient matrices of all valid viewpoints are stacked to form an over-constrained system of linear equations:

[0167] ;

[0168] S5.10: Assign weights to the linear equation for each effective viewpoint i That is, the weights are positively correlated with the confidence of the key points from this perspective. Applying the weights to the coefficient matrix A yields the weighted coefficient matrix A. w Reduce the impact of low-quality observations on 3D reconstruction;

[0169] S5.11: For the weighted coefficient matrix A w Perform singular value decomposition (SVD) to obtain the homogeneous coordinates X. j Dividing the homogeneous coordinates by its fourth element yields the three-dimensional Euclidean coordinates of the joint point j. ;

[0170] S5.12: Traverse all nodes of the robotic arm and repeat steps 5.1 to 5.4 above to obtain the three-dimensional coordinates of all joints. Combined with the kinematic model of the robotic arm, complete the estimation and pose calculation of the three-dimensional posture of the robotic arm.

[0171] An image enhancement and attitude estimation device for underwater confined space scenarios includes a memory, a processor, and a computer optimization program stored in the memory, wherein the processor executes the computer program to implement the steps of the method of claim 1.

[0172] A medium comprising image enhancement and attitude estimation for underwater confined space scenarios, wherein a computer program / instructions are stored thereon, which, when executed by a processor, implement the steps of the method of claim 1.

[0173] An image enhancement and attitude estimation product for underwater confined space scenarios includes a computer program / instructions that, when executed by a processor, implement the steps of the method of claim 1.

[0174] An image enhancement and attitude estimation device for underwater confined space scenarios is provided, comprising an underwater image enhancement submodule and a robotic arm attitude estimation submodule. When the underwater image enhancement submodule is executed, it implements steps S1 to S3 of the method of claim 1, and when the robotic arm attitude estimation submodule is executed, it implements steps S4 to S5 of the method of claim 1.

[0175] To verify the effectiveness of the underwater image enhancement and attitude estimation algorithm, the following experiment was designed. Multiple sets of original images collected in actual, enclosed underwater dark environments were selected as the experimental dataset, covering underwater images with varying water turbidity, light intensity, and viewing angles, providing a comprehensive verification basis for algorithm testing.

[0176] The test indicators and acceptance methods are set as follows. The image enhancement effect uses the Underwater Image Quality Metric (UIQM) as the core evaluation indicator. This indicator is a reference-free evaluation indicator specifically for underwater scenarios, capable of comprehensively quantifying image sharpness, color fidelity, and contrast even without a clear original reference image. The acceptance method involves calculating the UIQM values ​​of multiple enhanced images under actual underwater usage conditions and taking the average. The threshold for this indicator is set to UIQM ≥ 2.8. The test object attitude estimation effect uses the Mean Joint Position Error (MPJPE) as the core evaluation indicator. This indicator quantifies the accuracy of the 3D attitude estimation results by calculating the Euclidean distance between the true 3D coordinates and the estimated position of each joint of the test object. The acceptance method involves processing multiple enhanced viewpoint images using the attitude estimation algorithm, calculating the MPJPE values ​​of multiple samples, and taking the average. The threshold for this indicator is set to MPJPE ≤ 20mm.

[0177] like Figures 2-3 As shown, the acquired raw underwater images are input into the image enhancement submodule. The algorithm significantly improves the color distortion, uneven lighting, and blurring problems of underwater images, and effectively completes color correction and contrast enhancement. In multiple experiments with different water turbidity levels, low light, and reflection interference, the enhancement results maintain good consistency and robustness, without secondary color distortion or loss of detail. It has good adaptability to various complex underwater imaging conditions.

[0178] like Figures 5-6 As shown, the robot arm's pose estimation is performed on the enhanced multi-view images. The green box in the figure represents the target detection area, and the colored markers represent the identified key feature points. Experimental results show that even under complex environmental conditions such as uneven underwater illumination, scattering interference, and partial occlusion, the proposed method can still stably extract target key points and complete pose reconstruction without significant key point drift or loss. It demonstrates robustness and reliability in the underwater environment, providing effective data support for underwater robot arm operation monitoring and analysis.

[0179] like Figure 7 As shown, the UIQM values ​​of multiple enhanced image samples were calculated, and the actual measured average value was approximately 2.99, which met and exceeded the set technical indicator of UIQM≥2.8, and the enhancement effect met the expected target.

[0180] like Figure 8 As shown, the MPJPE values ​​of multiple sets of experimental samples after attitude estimation were calculated. The actual measured average value was approximately 12.40 mm, which met the set technical indicator of MPJPE ≤ 20 mm. The attitude estimation accuracy and effect met the actual application requirements of underwater visual monitoring.

[0181] This experiment verified the effectiveness of the underwater image enhancement and attitude estimation algorithms from three dimensions: functional effect, environmental adaptability, and quantitative indicators. Both algorithms can adapt to the complex imaging characteristics of dark and slightly turbid underwater environments. The quantitative indicators of the processing results meet the standards and are better than the preset thresholds. They can effectively improve the quality of underwater images and the effect of robotic arm attitude estimation, providing reliable technical support for visual monitoring of underwater operations.

[0182] In summary, this invention addresses the challenges of image enhancement and robotic arm pose estimation in enclosed underwater dark environments and slightly turbid waters for underwater visual monitoring. It covers the restoration of color distortion and detail blurring in underwater images, as well as robotic arm pose estimation in multi-view images under conditions of uneven underwater illumination, scattering interference, and partial occlusion. This invention designs an integrated image enhancement and robotic arm pose estimation method adapted to underwater scenarios, achieving dual optimization of high-quality underwater image restoration and robotic arm pose estimation. This method includes an underwater image enhancement strategy based on a three-stage architecture of color correction, contrast enhancement, and visual perception fusion, and a robotic arm pose estimation method using RTMO two-dimensional pose estimation and multi-view confidence-weighted triangulation.

[0183] Experimental results show that, on multiple underwater test samples with different water quality, lighting, and texture characteristics, the present invention can achieve high-quality underwater image enhancement and accurate robotic arm posture estimation. It outperforms the expected thresholds in the core indicators of underwater image quality metric UIQM and mean joint position error MPJPE, and demonstrates excellent performance in color reproduction, detail clarity, posture estimation accuracy, and visual effects.

[0184] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0185] Computer programs used to implement the methods of this application may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0186] In the context of this application, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium can be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0187] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0188] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), middleware components (e.g., application servers), or frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0189] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0190] Note that the above are merely preferred embodiments and technical principles applied in this application. Those skilled in the art will understand that this application is not limited to the specific embodiments described herein, and various obvious changes, readjustments, and substitutions can be made without departing from the scope of protection of this application. For example, those skilled in the art can use the various forms of processes shown above to reorder, add, or delete steps; the steps described in this application can be executed in parallel, sequentially, or in different orders, as long as the desired results of the technical solution of this application can be achieved, and no limitations are imposed herein.

[0191] The specific embodiments described above do not constitute a limitation on the scope of protection of this application. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A method for image enhancement and pose estimation in underwater confined space scenarios, characterized in that, The specific steps are as follows: S1: Perform adaptive channel statistical color correction processing on the acquired raw underwater images based on minimum color loss constraints; S2: Perform LAB layered adaptive contrast enhancement processing with minimal color loss on the color-corrected image; S3: Perform visual perception fusion based on gradient fusion, chromaticity similarity, and color loss constraints on the enhanced image to obtain the final enhanced image; S4: Input the final enhanced image into the RTMO two-dimensional pose estimation network to detect the two-dimensional coordinates and corresponding confidence scores of each joint of the robotic arm under each viewpoint. S5: Based on the intrinsic and extrinsic parameters of each camera obtained from the joint calibration of multiple cameras, combined with the two-dimensional coordinates and confidence scores of the joints at each viewpoint, the three-dimensional coordinates of each joint of the robotic arm are obtained by using the confidence-weighted triangulation method, thus completing the three-dimensional attitude estimation of the robotic arm.

2. The image enhancement and attitude estimation method for underwater confined space scenes according to claim 1, characterized in that, The specific implementation process of the adaptive channel statistical color correction processing based on the minimum color loss constraint described in S1 is as follows: S1.1: Calculate the mean values ​​of the red channel R, green channel G, and blue channel B of the original underwater image, respectively. and variance And construct a normalized correction model based on statistical features: in, This represents the pixel value of the corresponding color channel in the original image. This represents the channel image after initial color correction; S1.2: Using the green channel G as a reference channel, construct adaptive compensation factors for the red channel R and the blue channel B; S1.3: Based on the adaptive compensation factor, perform dynamic compensation on the red channel R and blue channel B, which have severe attenuation, to obtain the compensated red channel R and blue channel B; S1.4: Convert the compensated image to linear RGB space, construct a color adaptation matrix, and further correct the image color under different lighting conditions. The expression is as follows: in, This represents the compensated RGB image. Represents a linear spatial image. Represents the gamma coefficient. Represents the reference light source matrix. Represents the target light source matrix. This represents the output image after color correction.

3. The image enhancement and attitude estimation method for underwater confined space scenes according to claim 2, characterized in that, The specific implementation process of the LAB layered adaptive contrast enhancement processing with minimal color loss described in S2 is as follows: S2.1: The color-corrected image Convert to the LAB color space, which separates luminance and chrominance, to obtain the luminance component. and the first chromaticity component Second chromaticity component ; S2.2: For the brightness component Local contrast enhancement is performed, and a multi-scale decomposition method is used to extract the base layer and detail layer. First, adaptive histogram equalization is applied to the luminance component, and then a smoothing filter operator is used to decompose the two base layers, resulting in: in, As the first basic layer, As the second basic layer, and This indicates a smoothing and sharpening integrated filtering operation; S2.3: Based on the first base layer and the second base layer, construct the detail layer information of the luminance component to obtain: in, This represents the first level of detail. The second detail layer is represented by the separation of large-scale brightness information and small-scale texture information in the image by decomposing the brightness component into a base layer and a detail layer. S2.4: Transfer the second base layer With the first detail layer Second layer of detail Reconstruction is performed; simultaneously, the chromaticity components are... and Adaptive balancing adjustments are performed using the following formulas: when hour, when hour, in, and These represent the chromaticity components in the LAB color space. and These represent the adjusted chroma components. Finally, the enhanced luminance component and the adjusted chroma components are converted back to RGB space to obtain the image with enhanced contrast. .

4. The image enhancement and attitude estimation method for underwater confined space scenes according to claim 3, characterized in that, The specific implementation process of the visual perception fusion processing constrained by gradient fusion, chromaticity similarity, and color loss described in S3 is as follows: S3.1: Acquire the original image, the color-corrected image, and the contrast-enhanced image respectively, and convert them to a luminance and chrominance separation space; S3.2: The Sobel operator is used to calculate the gradient magnitude of the luminance component to obtain the image gradient information; S3.3: Calculate the gradient similarity between the color-corrected image and the original image, and between the contrast-enhanced image and the original image, respectively. The expression is as follows: When calculating the original image and the color-corrected image, The gradient of the original image. To correct the color gradient of the image; when calculating the original image and the contrast-enhanced image, The gradient of the original image. To enhance image gradient for contrast, Indicates the gradient similarity between two images; S3.4: Calculate the color similarity of the image based on the chromaticity components. Its expression is: When calculating the original image and the color-corrected image, , These are the two-channel chroma values ​​of the original image. , To correct the dual-channel chromaticity values ​​of the color-corrected image; when calculating the original image and the contrast-enhanced image, , These are the two-channel chroma values ​​of the original image. , Enhance the dual-channel chroma values ​​of the image to improve contrast; , These represent the values ​​of the two images on the first chromaticity component. , These represent the values ​​of the two images on the second chromaticity component, respectively. S3.5: Construct a comprehensive similarity score based on gradient similarity and color similarity: The softmax function is then used to calculate the fusion weights between the color-corrected image and the contrast-enhanced image. and ; S3.6: Based on the fusion weights, the color correction result and the contrast enhancement result are weighted and fused to obtain the final enhanced image: in, This represents the final enhanced image output.

5. The image enhancement and attitude estimation method for underwater confined space scenes according to claim 1, characterized in that, The specific implementation process of the robotic arm three-dimensional pose estimation method described in S5 is as follows: S5.1: Obtain the intrinsic parameter matrix, distortion coefficient, extrinsic parameter matrix between cameras, and projection matrix P for each underwater camera; S5.2: The feature map output by the backbone is processed by the hybrid encoder to obtain the sampled feature map; a score and pose feature are generated for each grid cell to predict the bounding box, key point coordinates and visibility; S5.3: Expand the bounding box to cover all relevant nodes, and divide it evenly along the horizontal and vertical axes respectively. indivual, There are 10 bins, each corresponding to a coordinate interval. The x-coordinate of each horizontal bin is calculated as follows: in, and These represent the left and right sides of the bounding box, respectively, and the value of index i ranges from 1 to... The bin method on the y-axis is the same; S5.4: Encode the coordinates of each bin to generate a bin-specific position representation, and optimize the position encoding by linear transformation through a fully connected layer; S5.5: Extract key point features from pose features and optimize them. Multiply the optimized key point features with the position representation of each bin to calculate the probability heatmap of key point occurrence in each bin. S5.6: Obtain the two-dimensional coordinates of each joint of the robotic arm through the probability heatmap decoding, and output the confidence score of each joint. S5.7: Input the enhanced underwater images from each viewpoint into the trained RTMO network, set a score threshold of 0.1 and perform grid filtering with non-maximum suppression (NMS), and decode to obtain the two-dimensional coordinates of each joint of the robotic arm from each viewpoint. With the corresponding confidence level , where i is the camera number and j is the key point number.

6. The image enhancement and attitude estimation method for underwater confined space scenes according to claim 5, characterized in that, The specific implementation process of the robotic arm's 3D pose reconstruction based on the confidence-weighted triangulation method described in S5 is as follows: S5.8: Set a confidence threshold δ for each joint point j. For the j-th joint point, only retain the confidence scores from each viewpoint. Two-dimensional coordinate observations are used to eliminate invalid observation data with low confidence, obstruction, or interference; S5.9: For the j-th joint point, let its homogeneous coordinates in three-dimensional space be... For each effective observation angle i, its constituent matrix is ​​a 3×4 camera projection matrix P. i The homogeneous coordinates of the two-dimensional observation are Based on the camera projection relationship, we have: The two linear equations corresponding to each effective viewpoint are combined to form a 2×4 coefficient matrix A. i The coefficient matrices of all valid viewpoints are stacked to form an over-constrained system of linear equations: ； S5.10: Assign weights to the linear equation for each effective viewpoint i That is, the weights are positively correlated with the confidence of the key points from this perspective. Applying the weights to the coefficient matrix A yields the weighted coefficient matrix A. w Reduce the impact of low-quality observations on 3D reconstruction; S5.11: For the weighted coefficient matrix A w Perform singular value decomposition (SVD) to obtain the homogeneous coordinates X. j Dividing the homogeneous coordinates by its fourth element yields the three-dimensional Euclidean coordinates of the joint point j. ; S5.12: Traverse all nodes of the robotic arm and repeat steps 5.8 to 5.11 above to obtain the three-dimensional coordinates of all joints. Combined with the kinematic model of the robotic arm, complete the estimation and pose calculation of the three-dimensional posture of the robotic arm.

7. An image enhancement and attitude estimation device for underwater confined space scenarios, comprising a memory, a processor, and a computer optimization program stored in the memory, characterized in that: The processor executes the computer program to implement the steps of the method according to any one of claims 1 to 6.

8. A medium comprising image enhancement and attitude estimation for underwater confined space scenarios, wherein a computer program is stored thereon, characterized in that: When the computer program / instructions are executed by the processor, they implement the steps of the method described in any one of claims 1 to 6.

9. A product comprising image enhancement and pose estimation for underwater confined space scenarios, including a computer program, characterized in that: When executed by a processor, the computer program implements the steps of the method according to any one of claims 1 to 6.

10. A device for image enhancement and attitude estimation in underwater confined space scenarios, characterized in that: It includes an underwater image enhancement submodule and a robotic arm attitude estimation submodule. When the underwater image enhancement submodule is executed, it implements steps S1 to S3 of the method described in claim 1. When the robotic arm attitude estimation submodule is executed, it implements steps S4 to S5 of the method described in claim 1.

Images