An underwater structure ranging system based on computer vision

Abstract

The invention discloses an underwater structure ranging system based on computer vision, comprising an image acquisition and preprocessing module, a camera calibration and image correction module, a stereo vision matching and depth estimation module, a depth map processing module, a three-dimensional reconstruction module and a target tracking and analysis module; the invention realizes the whole process from underwater image acquisition to three-dimensional structure ranging by combining computer vision technology with dual cameras to synchronously capture original images; a calibration plate is used to synchronously shoot and calibrate the cameras to eliminate lens distortion and improve the accuracy of stereo correspondence; a motion model and state prediction are used to improve the continuity and prediction performance of target tracking, solve the shortcomings of traditional target tracking algorithms in target state update and prediction, and use modules such as image preprocessing, stereo vision matching, depth estimation, three-dimensional reconstruction and target tracking to effectively cope with complex illumination and water quality conditions in underwater environments and improve the accuracy and reliability of underwater structure ranging.

Description

Technical Field

[0001] The invention relates to the field of underwater vision technology, in particular to an underwater structure ranging system based on computer vision. Background Art

[0002] In the underwater structure ranging system, the ranging method based on binocular stereo vision faces many technical difficulties. First, the lighting conditions in the underwater environment are complex and changeable. The attenuation, scattering and refraction of light will have an adverse effect on the image quality, resulting in low image contrast and loss of details, which increases the difficulty of stereo matching. Secondly, due to the fluidity and turbidity of water, there are a lot of noise and blurred areas in the image, which interfere with feature extraction and matching. Moreover, when the underwater shooting distance is far, the parallax between the foreground and the background is small, making it difficult to accurately estimate the depth. At the same time, since underwater targets generally lack texture features and their appearance is easily affected by changes in water flow and water pressure, stable feature tracking is difficult to achieve. In addition, in water bodies with high turbidity, the backscattering effect will cause haze in the image, resulting in blurred target edges, which increases the difficulty of target segmentation and recognition. Finally, the absorption and scattering characteristics of underwater lighting will cause image color distortion, which will affect subsequent feature matching and three-dimensional reconstruction. Summary of the invention

[0003] The purpose of the present invention is to provide an underwater structure ranging system based on computer vision, which comprehensively utilizes modules such as image preprocessing, stereo vision matching, depth estimation, three-dimensional reconstruction and target tracking to effectively cope with the complex lighting and water quality conditions in the underwater environment, improves the accuracy and reliability of underwater structure ranging, and provides a more complete solution for data acquisition and analysis of underwater scenes.

[0004] The purpose of the present invention can be achieved through the following technical solutions:

[0005] The present application provides an underwater structure ranging system based on computer vision, comprising

[0006] The image acquisition and preprocessing module uses dual cameras combined with computer vision technology to capture underwater images, and performs noise reduction and enhancement on the original images to obtain high-quality image data;

[0007] The camera calibration and image correction module uses Zhang Zhengyou's calibration method to obtain the internal and external parameters of the camera, correct image distortion, ensure that the dual cameras have an accurate stereo correspondence, and provide accurate geometric information for subsequent depth estimation;

[0008] The stereo vision matching and depth estimation module extracts feature points from the left and right views, calculates the disparity through the stereo vision matching algorithm, obtains the disparity map and converts it into a depth map to obtain the depth information of the image scene;

[0009] A depth map processing module, which filters and fills holes in the preliminary depth map, reduces matching errors and enhances the continuity and reliability of depth data, corrects depth errors and generates a processed depth map;

[0010] The 3D reconstruction module uses the calibrated depth map and camera parameters combined with the position information to reconstruct the 3D space point cloud and generate a high-precision 3D point cloud model;

[0011] The target tracking and analysis module uses a multi-target tracking algorithm to analyze dynamic targets in underwater scenes, and obtains the target's movement status and trajectory through data association and analysis.

[0012] The whole process from underwater image acquisition to three-dimensional structure ranging is completed, realizing the depth information acquisition and target tracking analysis of the underwater environment.

[0013] Furthermore, dual cameras are used to synchronously capture the underwater environment to obtain original visual information, and bilateral filtering is used to reduce the noise of the original image. The bilateral filtering technology can remove noise while maintaining edge information. Histogram equalization is performed on the denoised image to enhance image contrast. The histogram equalized image is then analyzed to extract feature points for subsequent matching and position estimation between images.

[0014] Furthermore, a calibration plate is used to synchronously shoot the dual cameras, obtain an image sequence for calibration, extract the corner points of the calibration plate in the image sequence, perform corner point detection, use the function in the OpenCV library to perform corner point detection, obtain the coordinates of the corner points on the calibration plate, match the corner points in the image sequence shot by the dual cameras, and make sure that the corner points corresponding to the left and right cameras can correctly correspond during matching. The correspondence between the world coordinates and the image coordinates of the corner points is used to calculate the internal parameters of the dual cameras using the Zhang Zhengyou calibration method. The dual camera images are distorted with precise internal and external parameters, and then the corrected images are used for stereo calibration. The imaging planes of the dual cameras are aligned, and the positions and postures of the dual cameras are adjusted according to the stereo calibration results to obtain accurate stereo correspondence.

[0015] Furthermore, through scale-invariant feature transformation, feature points are extracted from the left view and the right view captured by the dual cameras to ensure consistency in type and extraction criteria with the feature points in the left view. Then, the internal and external parameters of the left and right views are obtained, including the focal length, optical center, distortion coefficient of the camera, and the relative position and posture between the two cameras. The corresponding feature points between the left and right images are found according to the semi-global matching algorithm. The overall information of the image is used for matching, and the disparity between the corresponding feature points of the left and right images is calculated. The size of the disparity is directly related to the distance between the object and the camera. The larger the parallax, the closer the object is to the camera. By constructing a disparity map, the disparity value of each pixel in the scene is recorded.

[0016] Furthermore, the parallax is the horizontal offset between corresponding points of the left and right images, reflecting the depth information of the object relative to the camera. The parallax calculation formula is as follows:

[0017] Disparity = |x_left-x_right|, where x_left and x_right are the horizontal coordinates of the corresponding feature points in the left and right images respectively;

[0018] Through the geometric arrangement of dual cameras and the parallax calculation formula, a linear method is used to convert the parallax value into the physical depth value from the object to the camera. By obtaining the intrinsic and extrinsic information of the camera, a proportional factor L is calculated. The specific formula is expressed as L=f*T. Through the calculated proportional factor, the parallax value d is converted into the physical depth value Z by the specific formula: Z=f*T / d, where Z represents the real depth value from the object to the camera, that is, the physical depth, f is the focal length of the camera, T is the baseline distance between cameras, and d is the parallax value. A depth map is generated by converting the parallax value into a physical depth value. The depth map shows the physical distance of each point in the image scene. The depth map is then corrected and optimized to remove possible noise and errors, improve the accuracy and reliability of the depth map, and enable it to more realistically reflect the three-dimensional structure of the physical world.

[0019] Furthermore, a Gaussian filtering algorithm is used to smooth the initial depth map, and the image noise and details are reduced through the weight distribution of the Gaussian function. The smoothed depth map is enhanced by edge preservation, and the edge detection algorithm is used to identify the object boundaries. The empty areas in the depth map are filled through dilation and erosion operations. The filled depth map is divided into regions using connected region analysis, and each independent object is identified and marked. The refraction and scattering parameters of light in water are calculated according to the underwater imaging model, and the necessary physical values ​​are prepared for the correction of the depth map. The error of each pixel in the depth map is corrected through the refraction and scattering parameters, and the depth deviation caused by the influence of the underwater environment is corrected using the light propagation model.

[0020] Furthermore, Zhang Zhengyou's calibration method is used to obtain the internal and external parameters of the camera, and then the spatial position of each pixel in the world coordinate system is calculated based on the position information of the camera. The corrected depth map and internal and external parameters are used to generate initial three-dimensional point cloud data. The spatial filtering technology is used to reduce the noise of the initial point cloud data, and then the surface of the denoised point cloud data is reconstructed through the Poisson reconstruction algorithm to obtain a continuous three-dimensional model. According to the reconstructed three-dimensional model, the boundaries and feature points of the object are extracted, and spatial geometry analysis is performed on the extracted feature points to determine the specific coordinates of the object in the three-dimensional space.

[0021] Furthermore, the object boundaries and feature points extracted from the three-dimensional model are used for target detection and analysis, and then combined with the initial target information extracted from the scene by the joint probabilistic data association algorithm; the target state at the next moment is predicted based on the target's motion model, and then multiple frames of image data are collected, the target is detected in each frame, and target features including borders, color histograms and feature points are extracted; the correlation between the detected target features and the predicted target state is calculated, focusing on the matching of feature similarity and spatial proximity; the matching relationship of the target in consecutive frames is determined by maximizing the correlation; if the target is lost in some frames, the predicted trajectory information and previous observation data are used for data association to supplement the interrupted tracking; the motion trajectory of each target is smoothed, noise and errors are filtered, and the expression of the target's motion trajectory is optimized; the motion patterns of all targets are analyzed, abnormal motion or special events are identified, and the movement state of dynamic targets in the scene is judged based on the target's motion trajectory and behavioral characteristics to form a comprehensive target motion analysis report.

[0022] The method for implementing the underwater structure ranging system based on computer vision comprises the following steps:

[0023] S1. Use dual cameras combined with computer vision technology to capture underwater images, initially obtain original visual information of the underwater environment, and use image preprocessing technology to reduce noise and enhance the captured original images to obtain higher quality image data;

[0024] S2. According to Zhang Zhengyou's calibration method, the dual cameras obtain the internal and external parameters of the camera. The internal and external parameters obtained through calibration are used to correct image distortion and synchronize the cameras, establish the stereo correspondence between the images of the two cameras, and provide accurate geometric information for depth estimation;

[0025] S3, extracting feature points from the left and right views captured by the dual cameras through scale-invariant feature transformation according to the internal and external parameters of the cameras, finding corresponding feature points between the left and right images through a stereo vision matching algorithm, calculating the disparity and obtaining a disparity map, and then converting the disparity map into a depth map according to the geometric arrangement of the dual cameras and the disparity calculation formula, and converting the extracted disparity information into a physical distance metric;

[0026] S4, after obtaining the preliminary depth map, performing post-processing operations on the depth map, filtering the depth map, and using morphological operations to fill holes, then obtaining the filtered and filled depth map, and correcting the depth error using the underwater imaging model;

[0027] S5. Through depth map correction, the calibrated depth map and the internal and external parameters obtained by calibration are used in combination with the position information of the camera to reconstruct the three-dimensional space point cloud, and the coordinates of the object in the three-dimensional space are generated at the same time. According to the three-dimensional point cloud data, a three-dimensional reconstruction algorithm is used to obtain a high-precision three-dimensional point cloud model;

[0028] S6. Then use the multi-target tracking algorithm to analyze and judge the dynamic targets in the scene, and obtain the target's movement state and trajectory through the association and analysis of multi-frame data.

[0029] The beneficial effects of the present invention are:

[0030] (1) Computer vision technology combined with dual cameras to synchronously capture original images realizes the entire process from underwater image acquisition to 3D structure ranging, including target tracking and analysis, providing comprehensive support for the acquisition of underwater environment depth information and target dynamic monitoring;

[0031] (2) A calibration board is used to synchronously shoot and calibrate the camera to eliminate lens distortion and improve the accuracy of stereo correspondence. A Gaussian filter algorithm is used to smooth the depth map to maintain the continuity of depth data, improve the imaging quality and accuracy of the dual-camera system, ensure that accurate depth information is captured, and solve the shortcomings of traditional depth estimation algorithms in noise processing.

[0032] (3) Motion models and state prediction are used to improve the continuity and prediction performance of target tracking, address the shortcomings of traditional target tracking algorithms in target state update and prediction, achieve effective monitoring and analysis of underwater dynamic targets, and address the challenges of traditional data association algorithms in matching accuracy and stability. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] For better understanding and implementation, the technical solution of the present application is described in detail below with reference to the accompanying drawings.

[0034] Figure 1 This is a structural block diagram of a computer vision-based underwater structure ranging system provided in Example 1 of the present application. DETAILED DESCRIPTION

[0035] In order to further explain the technical means and effects taken by the present invention to achieve the predetermined invention purpose, exemplary embodiments will be described in detail here, and examples thereof are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are only examples of methods and systems consistent with some aspects of the present application as detailed in the attached claims.

[0036] The terms used in this application are for the purpose of describing specific embodiments only and are not intended to limit this application. The singular forms of "a", "said" and "the" used in this application and the appended claims are also intended to include plural forms unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0037] The specific implementation methods, features and effects of the present invention are described in detail below in conjunction with the accompanying drawings and preferred embodiments.

[0038] Example 1

[0039] See also Figure 1 ,This embodiment provides an underwater structure ranging system based on computer vision, which comprehensively utilizes modules such as image preprocessing, stereo vision matching, depth estimation, three-dimensional reconstruction and target tracking, effectively copes with the complex lighting and water quality conditions of the underwater environment, improves the accuracy and reliability of underwater structure ranging, and provides a more complete solution for data acquisition and analysis of underwater scenes.

[0040] The present invention provides an underwater structure ranging system based on computer vision, which includes image acquisition and preprocessing, camera calibration and image correction, stereo vision matching and depth estimation, depth map processing, three-dimensional reconstruction and target tracking and analysis, which jointly realize the acquisition, processing and analysis of image data, and finally realize the tracking, three-dimensional reconstruction and depth information estimation of the target, and construct a complete visual processing flow.

[0041] The image acquisition and preprocessing module uses dual cameras combined with computer vision technology to capture underwater images, and performs noise reduction and enhancement on the original images to obtain high-quality image data;

[0042] The camera calibration and image correction module uses Zhang Zhengyou's calibration method to obtain the internal and external parameters of the camera, correct image distortion, ensure that the dual cameras have an accurate stereo correspondence, and provide accurate geometric information for subsequent depth estimation;

[0043] The stereo vision matching and depth estimation module extracts feature points from the left and right views, calculates the disparity through the stereo vision matching algorithm, obtains the disparity map and converts it into a depth map to obtain the depth information of the image scene;

[0044] A depth map processing module, which filters and fills holes in the preliminary depth map, reduces matching errors and enhances the continuity and reliability of depth data, corrects depth errors and generates a processed depth map;

[0045] The 3D reconstruction module uses the calibrated depth map and camera parameters combined with the position information to reconstruct the 3D space point cloud and generate a high-precision 3D point cloud model;

[0046] The target tracking and analysis module uses a multi-target tracking algorithm to analyze dynamic targets in underwater scenes, and obtains the target's movement status and trajectory through data association and analysis.

[0047] The whole process from underwater image acquisition to three-dimensional structure ranging is completed, realizing the depth information acquisition and target tracking analysis of the underwater environment.

[0048] It realizes the whole process from underwater image acquisition to three-dimensional structure ranging, including image acquisition, camera calibration, stereo vision matching, depth estimation, three-dimensional reconstruction, target tracking and analysis, so as to achieve the depth information acquisition and target tracking analysis of the underwater environment, making the underwater structure ranging more accurate, stable and comprehensive, and providing strong support for the monitoring, detection and analysis of underwater scenes.

[0049] Furthermore, dual cameras are used to synchronously capture the underwater environment to obtain original visual information, and bilateral filtering is used to reduce the noise of the original image. The bilateral filtering technology can remove noise while maintaining edge information, thereby improving the visual quality of the image. Histogram equalization is performed on the denoised image to enhance image contrast and make the details of the underwater image clearer and more discernible. The histogram-equalized image is then analyzed to extract feature points for subsequent matching and position estimation between images.

[0050] Specifically, the denoised color image is converted into a grayscale image, and the RGB color image is converted into a grayscale image, where the pixel value of the grayscale image = 0.299×R+0.587×G+0.114×B;

[0051] Then calculate the histogram of the grayscale image, count the number of pixels at each pixel level (0-255) in the grayscale image, get the histogram, and calculate the cumulative distribution function (CDF). Normalize the histogram of the grayscale image to get the probability distribution at the pixel level, and then calculate the cumulative probability distribution function (CDF) at the pixel level, which represents the probability of pixels less than or equal to a certain grayscale level appearing in the image. According to the CDF calculation formula, equalize each pixel level and map the original pixel value to the new pixel value. The calculation formula for the equalized pixel value is:

[0052] Equalized pixel value = round((CDF(original pixel value)-CDF(min)) / (M*N-CDF(min))*(L-1))

[0053] Among them, CDF (original pixel value) represents the cumulative probability value corresponding to the original pixel value, min is the minimum cumulative probability value, M and N are the number of rows and columns of the image respectively, and L is the number of pixel levels (usually 256).

[0054] Furthermore, a calibration plate is used to synchronously shoot the dual cameras, an image sequence for calibration is obtained, the corner points of the calibration plate in the image sequence are extracted, and corner point detection is performed. Functions in libraries such as OpenCV are used for corner point detection, the coordinates of the corner points on the calibration plate are obtained, and the corner points in the image sequence shot by the dual cameras are matched to ensure that the corner points corresponding to the left and right cameras can correspond correctly during matching. The correspondence between the world coordinates and the image coordinates of the corner points is used to calculate the internal parameters of the dual cameras using the Zhang Zhengyou calibration method, including focal length, optical center, etc.; the dual camera images are distorted and corrected through precise internal and external parameters to eliminate the influence of lens distortion, and then the corrected images are used for stereo calibration to align the imaging planes of the dual cameras to ensure the coplanarity of the imaging planes, and the positions and postures of the dual cameras are adjusted according to the stereo calibration results to achieve accurate stereo correspondence.

[0055] The imaging quality and accuracy of the dual-camera system are improved, providing more reliable and precise data support for subsequent 3D reconstruction, depth estimation and target tracking. The effect of the entire process is to ensure that the dual-camera system can accurately capture the depth information of the scene, providing a reliable foundation for subsequent calculations and analysis.

[0056] Furthermore, through scale-invariant feature transformation, feature points are extracted from the left view and the right view captured by the dual cameras to ensure consistency in type and extraction criteria with the feature points in the left view, and then the internal and external parameters of the left and right views are obtained, including the focal length, optical center, distortion coefficient of the camera, and the relative position and posture between the two cameras. These parameters are used for subsequent disparity calculation and depth estimation. The corresponding feature points between the left and right images are found according to the semi-global matching algorithm, and the overall information of the image is used for matching to improve the accuracy and robustness of the matching, and the disparity between the corresponding feature points of the left and right images is calculated. The size of the disparity is directly related to the distance between the object and the camera. The larger the parallax, the closer the object is to the camera. By constructing a disparity map, the disparity value of each pixel in the scene is recorded.

[0057] By constructing a disparity map to display the depth differences between different pixels, we can help understand the spatial position relationship of objects in the scene. Through the disparity map, we can intuitively observe the changes in the depth direction of the object, thereby realizing the estimation of the three-dimensional structure and position of the objects in the scene.

[0058] Furthermore, the parallax is the horizontal offset of corresponding points of the left and right images, which directly reflects the depth information of the object relative to the camera. The parallax calculation formula is as follows:

[0059] Disparity = |x_left-x_right|, where x_left and x_right are the horizontal coordinates of the corresponding feature points in the left and right images respectively;

[0060] Through the geometric arrangement of dual cameras and the parallax calculation formula, a linear method is used to convert the parallax value into the physical depth value from the object to the camera. By obtaining the intrinsic and extrinsic information of the camera, a proportional factor L is calculated. The specific formula is expressed as L=f*T. Through the calculated proportional factor, the parallax value d is converted into the physical depth value Z by the specific formula: Z=f*T / d, where Z represents the real depth value from the object to the camera, that is, the physical depth, f is the focal length of the camera, T is the baseline distance between cameras, and d is the parallax value. A depth map is generated by converting the parallax value into a physical depth value. The depth map shows the physical distance of each point in the image scene. The depth map is then corrected and optimized to remove possible noise and errors, improve the accuracy and reliability of the depth map, and enable it to more realistically reflect the three-dimensional structure of the physical world.

[0061] Furthermore, a Gaussian filtering algorithm is used to smooth the initial depth map, and the weight distribution of the Gaussian function is used to reduce image noise and details, improve the overall continuity of the depth data, and perform edge-preserving enhancement on the smoothed depth map. An edge detection algorithm is used to identify object boundaries to prevent edge blurring during the Gaussian filtering process and ensure the clarity of the object contour. The missing areas in the depth map are filled through dilation and corrosion operations to enhance the continuity and integrity of the depth data inside the object. The filled depth map is divided into regions using connected region analysis, and each independent object is identified and marked. The refraction and scattering parameters of light in water are calculated based on the underwater imaging model to prepare the necessary physical values ​​for the correction of the depth map. The error of each pixel in the depth map is corrected through the refraction and scattering parameters, and the depth deviation caused by the influence of the underwater environment is corrected using the light propagation model.

[0062] Specifically, the underwater imaging model represents the refraction and scattering process in water. The refraction parameters of light in water are calculated by Snell's law. Snell's law describes the physical law of the refraction phenomenon of light at the interface between media. The formula is: n_1\sinφ=n_2\sinθ;

[0063] Among them, n_1 is the refractive index in air, which is 1, n_2 is the refractive index of water, which is 1.33, φ is the angle at which the light enters the water surface, and θ is the angle of refraction of the light in the water;

[0064] Scattering parameter calculation, by determining the scattering coefficient of water quality to light, use scattering theory to calculate the scattering parameters at each pixel point, considering the influence of factors such as water quality concentration and wavelength on scattering; then according to the calculated refraction and scattering parameters, the depth value of each pixel in the depth map is corrected.

[0065] The light propagation model is used to correct the depth deviation. The pixel position in the depth image is corrected according to the change of the refraction angle and the influence of scattering on the light propagation. The scattering error in the depth image is corrected to make it more accurately represent the actual depth information. The depth deviation caused by absorption in the depth image is corrected according to the light absorption phenomenon in the water, the wavelength of the light and the water quality.

[0066] Furthermore, Zhang Zhengyou's calibration method is used to obtain the internal and external parameters of the camera to ensure the accuracy of spatial transformation during 3D reconstruction. The spatial position of each pixel in the world coordinate system is calculated based on the camera's position information. The corrected depth map and internal and external parameters are used to generate initial 3D point cloud data. Spatial filtering technology is used to reduce the noise of the initial point cloud data to improve the clarity and availability of the point cloud data. The surface of the denoised point cloud data is reconstructed using the Poisson reconstruction algorithm to obtain a continuous 3D model. Based on the reconstructed 3D model, the boundaries and feature points of the object are extracted to provide data support for object recognition and classification. Spatial geometric analysis is performed on the extracted feature points to determine the specific coordinates of the object in the 3D space. By comparing and analyzing with a known object database, the type and attributes of the object are identified to provide accurate 3D model information for subsequent applications.

[0067] In an underwater environment, Zhang Zhengyou's calibration method is used to obtain the internal and external parameters of the camera, calculate the spatial position of each pixel in the world coordinate system, generate initial three-dimensional point cloud data, and perform noise reduction and surface reconstruction on it to finally obtain a continuous three-dimensional model. This paper describes the conversion and reconstruction process from image data to a three-dimensional model.

[0068] Furthermore, a joint probabilistic data association algorithm is used to extract the initial target information in the scene, including the target's location, speed, size, type and other attributes; the target state at the next moment is predicted based on the target's motion model, including related attributes such as the predicted position and predicted speed; multiple frames of image data are collected, the target is detected in each frame, and target features including borders, color histograms and feature points are extracted; the correlation between the detected target features and the predicted target state is calculated, focusing on the matching of feature similarity and spatial proximity; by maximizing the correlation, the matching relationship of the target in consecutive frames is determined to achieve the continuity of target tracking; if the target is lost in some frames, the predicted trajectory information and previous observation data are used for data association to supplement the interrupted tracking; the motion trajectory of each target is smoothed, noise and errors are filtered out, and the expression of the target's motion trajectory is optimized; the motion patterns of all targets are analyzed, and abnormal motion or special events, such as rapid lane changes, sudden stops and other behavioral characteristics, are identified; based on the target's motion trajectory and behavioral characteristics, the moving state of dynamic targets in the scene is judged to form a comprehensive target motion analysis report.

[0069] The specific contents include: using the joint probability data association algorithm to extract target information and track the target, analyzing the target's motion law and inferring the target's motion pattern through historical target trajectories and observation data, selecting the target's motion model according to the characteristics of the underwater environment and the target's motion characteristics, expressing the target's state as a state vector, including attributes such as position and speed, establishing a state space model, and using the state transfer equation to predict the target's state at the next moment based on the current target state and the target's motion model. After receiving new observation data, the predicted target state and actual observation data are combined to update the state to correct the prediction error and improve the prediction accuracy.

[0070] Through motion models and state prediction, the position, speed and other attributes of the target at future moments can be estimated, providing predictive performance for target tracking and analysis.

[0071] The joint probabilistic data association algorithm is used to extract the initial target information in the scene, including the target's location, speed, size, type and other attributes, and predict the target state at the next moment based on the target's motion model. In this stage, multiple frames of image data are collected, the target in each frame is detected, the target features are extracted, and the target state is predicted, matched and tracked. Finally, the target's motion trajectory and behavior characteristics are obtained to form a comprehensive target motion analysis report.

[0072] Furthermore, when calculating the correlation between the detected target feature and the predicted target state, the matching degree between the target feature and the predicted state is first evaluated by comparing their similarity and spatial proximity, and the Euclidean distance or similarity metric between the features is calculated, and the spatial position information is combined to make a trade-off, and the similarity score between the target feature and the predicted target state is calculated, and the spatial distance score is calculated by the position information, and then the similarity score and the spatial distance score are combined according to a certain weight to obtain a final correlation score.

[0073] Relevance score = similarity score * α + spatial distance score * β;

[0074] Among them, α and β are the weight coefficients of the similarity score and the spatial distance score, which are set according to the specific situation. The higher the final correlation score, the higher the matching degree between the target feature and the predicted target state.

[0075] Furthermore, the method for implementing the underwater structure ranging system based on computer vision comprises the following steps:

[0076] S1. Capture underwater images through dual cameras combined with computer vision technology to initially obtain original visual information of the underwater environment. Use image preprocessing techniques, such as bilateral filtering and histogram equalization, to reduce noise and enhance the captured original images to obtain higher quality image data.

[0077] S2. Obtain the internal and external parameters of the dual cameras according to Zhang Zhengyou's calibration method. These parameters describe the geometric relationship and distortion characteristics between the dual cameras. The internal and external parameters obtained through calibration are used to correct image distortion and synchronize the cameras to ensure that the images of the two cameras have an accurate stereo correspondence and provide accurate geometric information for depth estimation.

[0078] S3, according to the internal and external parameters of the camera, extract feature points from the left and right views captured by the dual cameras through scale-invariant feature transformation, and find the corresponding feature points between the left and right images through a stereo vision matching algorithm, such as a semi-global matching algorithm, and calculate the disparity to obtain a disparity map. The disparity map contains the depth information of each part of the image scene, and then according to the geometric arrangement of the dual cameras and the disparity calculation formula, the disparity map is converted into a depth map, and the extracted disparity information is converted into a physical distance metric;

[0079] S4. After obtaining the preliminary depth map, perform post-processing operations on the depth map, including filtering the depth map using a Gaussian filtering method and filling holes using morphological operations to reduce matching errors and enhance the continuity and reliability of the depth data, obtain the filtered and filled depth map, and correct the depth error using an underwater imaging model, which takes into account the refraction and scattering effects of light in the underwater environment;

[0080] S5. Through depth map correction, the corrected depth map and the internal and external parameters obtained by calibration are used in combination with the position information of the camera to reconstruct the three-dimensional space point cloud, and the coordinates of the object in the three-dimensional space are generated at the same time. According to the three-dimensional point cloud data, a three-dimensional reconstruction algorithm such as Poisson reconstruction is used to obtain a high-precision three-dimensional point cloud model;

[0081] S6. Then use a multi-target tracking algorithm, such as a joint probabilistic data association algorithm, to analyze and determine the dynamic targets in the scene, and obtain the target's movement state and trajectory through the association and analysis of multiple frames of data.

[0082] The above description is only a preferred embodiment of the present invention and does not limit the present invention in any form. Although the present invention has been disclosed as a preferred embodiment as above, it is not used to limit the present invention. Any technical personnel in this field can make some changes or modify the technical contents disclosed above into equivalent embodiments without departing from the scope of the technical solution of the present invention. However, any brief modifications, equivalent changes and modifications made to the above embodiments based on the technical essence of the present invention without departing from the content of the technical solution of the present invention are still within the scope of the technical solution of the present invention.

Claims

1. An underwater structure ranging system based on computer vision, characterized in that: It includes an image acquisition and preprocessing module, which uses dual cameras combined with computer vision technology to capture underwater images, and performs noise reduction and enhancement on the original images to obtain high-quality image data; The camera calibration and image correction module uses Zhang Zhengyou's calibration method to obtain the internal and external parameters of the camera, correct image distortion, ensure that the dual cameras have an accurate stereo correspondence, and provide accurate geometric information for subsequent depth estimation; The stereo vision matching and depth estimation module extracts feature points from the left and right views, calculates the disparity through the stereo vision matching algorithm, obtains the disparity map and converts it into a depth map to obtain the depth information of the image scene; A depth map processing module, which filters and fills holes in the preliminary depth map, reduces matching errors and enhances the continuity and reliability of depth data, corrects depth errors and generates a processed depth map; The 3D reconstruction module uses the calibrated depth map and camera parameters combined with the position information to reconstruct the 3D space point cloud and generate a high-precision 3D point cloud model; The target tracking and analysis module uses a multi-target tracking algorithm to analyze dynamic targets in underwater scenes, and obtains the target's movement status and trajectory through data association and analysis; The whole process from underwater image acquisition to three-dimensional structure ranging is completed, realizing the acquisition of depth information of underwater environment and target tracking analysis; The underwater environment is captured synchronously by dual cameras to obtain original visual information, and bilateral filtering is used to reduce noise on the original image. Bilateral filtering technology can remove noise while maintaining edge information. Histogram equalization is performed on the denoised image to enhance image contrast. The histogram-equalized image is then analyzed to extract feature points for subsequent image matching and position estimation. Use a calibration plate to shoot the dual cameras synchronously, obtain the image sequence for calibration, extract the corner points of the calibration plate in the image sequence, perform corner point detection, use the function in the OpenCV library to detect corner points, obtain the coordinates of the corner points on the calibration plate, match the corner points in the image sequence shot by the dual cameras, and make sure that the corner points corresponding to the left and right cameras can correspond correctly during matching. Use the correspondence between the world coordinates and image coordinates of the corner points, and then use the Zhang Zhengyou calibration method to calculate the internal parameters of the dual cameras. Use precise internal and external parameters to correct the distortion of the dual camera images, and then use the corrected images to perform stereo calibration, align the imaging planes of the dual cameras, adjust the position and posture of the dual cameras according to the stereo calibration results, and obtain accurate stereo correspondence. Through scale-invariant feature transformation, feature points are extracted from the left view and the right view captured by the dual cameras to ensure consistency in type and extraction criteria with the feature points in the left view. Then, the internal and external parameters of the left and right views are obtained, including the focal length, optical center, distortion coefficient of the camera, and the relative position and posture between the two cameras. The corresponding feature points between the left and right images are found according to the semi-global matching algorithm. The overall information of the image is used for matching, and the disparity between the corresponding feature points of the left and right images is calculated. The size of the disparity is directly related to the distance between the object and the camera. The larger the disparity, the closer the object is to the camera. By constructing a disparity map, the disparity value of each pixel in the scene is recorded. The parallax is the horizontal offset between corresponding points in the left and right images, reflecting the depth information of the object relative to the camera. The parallax calculation formula is as follows: , in and are the horizontal coordinates of the corresponding feature points in the left and right images respectively; Through the geometric arrangement of the dual cameras and the parallax calculation formula, a linear method is used to convert the parallax value into the physical depth value from the object to the camera. By obtaining the intrinsic and extrinsic information of the camera, a proportional factor L is calculated. The specific formula is expressed as , by calculating the scaling factor, the specific formula for converting the disparity value d into the physical depth value Z is: , where Z represents the real depth value from the object to the camera, i.e., the physical depth, f is the focal length of the camera, T is the baseline distance between cameras, and d is the disparity value. The disparity value is converted into a physical depth value to generate a depth map, which displays the physical distance of each point in the image scene. The depth map is then corrected and optimized to remove noise and errors, improve the accuracy and reliability of the depth map, and enable it to more realistically reflect the three-dimensional structure of the physical world; The initial depth map is smoothed using a Gaussian filtering algorithm, image noise and details are reduced through the weight distribution of the Gaussian function, edge-preserving enhancement is performed on the smoothed depth map, object boundaries are identified using an edge detection algorithm, and then the empty areas in the depth map are filled through dilation and corrosion operations. The filled depth map is divided into regions using connected region analysis, and each independent object is identified and marked. Then, based on the underwater imaging model, the refraction and scattering parameters of light in water are calculated to prepare physical values ​​for the correction of the depth map. Error correction is performed on each pixel of the depth map through the refraction and scattering parameters, and the light propagation model is used to correct the depth deviation caused by the underwater environment. The Zhang Zhengyou calibration method is used to obtain the internal and external parameters of the camera. Then, based on the position information of the camera, the spatial position of each pixel in the world coordinate system is calculated. The corrected depth map and internal and external parameters are used to generate the initial three-dimensional point cloud data. The spatial filtering technology is used to reduce the noise of the initial point cloud data. The surface of the denoised point cloud data is reconstructed using the Poisson reconstruction algorithm to obtain a continuous three-dimensional model. Based on the reconstructed three-dimensional model, the boundaries and feature points of the object are extracted. For the extracted feature points, spatial geometric analysis is performed to determine the specific coordinates of the object in the three-dimensional space. The object boundaries and feature points extracted from the 3D model are used for target detection and analysis, and then combined with the initial target information extracted from the scene by the joint probabilistic data association algorithm; the target state at the next moment is predicted based on the target's motion model, and then multiple frames of image data are collected to detect the target in each frame, and the target features including the border, color histogram and feature points are extracted; the correlation between the detected target features and the predicted target state is calculated, focusing on the matching of feature similarity and spatial proximity; the matching relationship of the target in consecutive frames is determined by maximizing the correlation; if the target is lost in some frames, the predicted trajectory information and previous observation data are used for data association to supplement the interrupted tracking; the motion trajectory of each target is smoothed, noise and errors are filtered, and the expression of the target's motion trajectory is optimized; the motion patterns of all targets are analyzed, abnormal motion or special events are identified, and the moving state of dynamic targets in the scene is judged based on the target's motion trajectory and behavioral characteristics to form a comprehensive target motion analysis report; The method for implementing the underwater structure ranging system based on computer vision comprises: S1. Use dual cameras combined with computer vision technology to capture underwater images, initially obtain original visual information of the underwater environment, and use image preprocessing technology to reduce noise and enhance the captured original images to obtain higher quality image data; S2. According to Zhang Zhengyou's calibration method, the dual cameras obtain the internal and external parameters of the camera. The internal and external parameters obtained through calibration are used to correct image distortion and synchronize the cameras, establish the stereo correspondence between the images of the two cameras, and provide accurate geometric information for depth estimation; S3, extracting feature points from the left and right views captured by the dual cameras through scale-invariant feature transformation according to the internal and external parameters of the cameras, finding corresponding feature points between the left and right images through a stereo vision matching algorithm, calculating the disparity and obtaining a disparity map, and then converting the disparity map into a depth map according to the geometric arrangement of the dual cameras and the disparity calculation formula, and converting the extracted disparity information into a physical distance metric; S4, after obtaining the preliminary depth map, performing post-processing operations on the depth map, filtering the depth map, and using morphological operations to fill holes, then obtaining the filtered and filled depth map, and correcting the depth error using the underwater imaging model; S5. Through depth map correction, the calibrated depth map and the internal and external parameters obtained by calibration are used in combination with the position information of the camera to reconstruct the three-dimensional space point cloud, and the coordinates of the object in the three-dimensional space are generated at the same time. According to the three-dimensional point cloud data, a three-dimensional reconstruction algorithm is used to obtain a high-precision three-dimensional point cloud model; S6. Then use the multi-target tracking algorithm to analyze and judge the dynamic targets in the scene, and obtain the target's movement state and trajectory through the association and analysis of multi-frame data.

Images