Underwater binocular vision distance estimation method based on key point matching and multilayer perceptron
By adopting an underwater binocular vision distance estimation method based on key point matching and multilayer perceptron, the problem of unstable accuracy of traditional underwater ranging methods in complex environments is solved, achieving higher robustness and accuracy, and is suitable for low-power underwater platforms and multi-agent collaborative perception.
Patent Information
- Application Number
- CN202511266563.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-09-05
- Publication Date
- 2025-12-23
AI Technical Summary
Traditional underwater ranging methods are unstable in accuracy and have poor robustness in complex environments. They rely on camera intrinsic parameters and are sensitive to the accuracy of feature point matching, making it difficult to maintain stable performance under conditions of image blur or occlusion.
An underwater binocular vision distance estimation method based on keypoint matching and multilayer perceptron is adopted. Video data is acquired by binocular cameras to obtain the disparity features of keypoint matching pairs. The multilayer perceptron model is used for feature extraction and fusion to output the predicted distance of the target. The model is optimized by training and optimizing the dataset to avoid dependence on camera parameters and improve robustness.
It improves the robustness and accuracy of underwater ranging, reduces system deployment costs, is suitable for low-power underwater platforms, and is applicable to applications such as underwater multi-agent collaborative perception and biomimetic robot navigation.
Smart Images
Figure CN121190960A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of underwater binocular vision ranging technology, and in particular to an underwater binocular vision distance estimation method based on key point matching and multilayer perceptron. Background Technology
[0002] Underwater multi-agent tracking technology based on target detection is a low-cost and easily deployed underwater perception method, widely used in tasks such as collaborative underwater robot swarm operations, underwater target monitoring, and biomimetic robot behavior recognition. Traditional underwater ranging methods typically rely on binocular vision systems. By detecting the target position in an image, they combine feature point matching algorithms and imaging geometry models to calculate the disparity between the binocular images, thereby estimating the target distance. However, in real underwater environments, due to complex factors such as water turbidity, uneven lighting, weakened target texture, and changes in appearance, traditional geometry-based ranging methods are prone to errors in the feature extraction and matching stages. Furthermore, they are highly dependent on camera intrinsic parameters, leading to unstable ranging accuracy and poor robustness.
[0003] Existing research has attempted to incorporate deep learning methods for object detection and distance estimation, but most approaches still rely on precise target bounding box localization and high-quality feature point matching, making it difficult to maintain stable performance in the presence of image blur, occlusion, or local loss. Furthermore, some methods are highly sensitive to camera intrinsic parameters; if the intrinsic parameter estimation is flawed, it can easily lead to errors in disparity calculation, thus affecting the accuracy of the final distance estimation.
[0004] Therefore, there is an urgent need for an underwater ranging method that does not rely on high-precision camera parameters, can make full use of stable key feature point information in images, and has good robustness and generalization ability, so as to improve the perception accuracy and system reliability of multi-agent targets in complex underwater environments.
[0005] It should be noted that this section is intended to provide background or context for the technical solutions of this disclosure as set forth in the claims. The description herein does not constitute an admission that it is prior art simply because it is included in this section. Summary of the Invention
[0006] The purpose of this invention is to provide an underwater binocular vision distance estimation method based on key point matching and multilayer perceptron, thereby overcoming at least to some extent one or more problems caused by the limitations and defects of related technologies.
[0007] This invention first provides an underwater binocular vision distance estimation method based on keypoint matching and multilayer perceptron, comprising the following steps: S100, at a preset distance, video data of the target is acquired using a binocular camera, and key point matching pairs in the target area in the left and right images are obtained based on the video data. The disparity features of the key point matching pairs and the corresponding actual distances are combined to form input features and label pairs to form a dataset. S200: The multilayer perceptron model is used to extract and fuse the input features and output the predicted distance of the target. The multilayer perceptron model includes an input layer, four hidden layers and an output layer. S300, the disparity features of the keypoint matching pairs are used as input, and their corresponding actual distances are used as labels. The multilayer perceptron model is trained and optimized using the dataset and loss function. The S400 uses an optimized multilayer perceptron model to perform actual distance testing on the target and obtain the final predicted distance of the target.
[0008] In this invention, the preset distances include: 1.0 m, 1.5 m, 2.0 m, 2.5 m, 3.0 m and 4.0 m.
[0009] In this invention, step S100 specifically includes the following steps: S101, at a preset distance, use a binocular camera to collect video data of the target, obtain left and right images based on the video data, and set them as left and right views respectively; S102, the left view and right view are processed by a pre-trained target detection algorithm to obtain the target detection boxes corresponding to the target in the left view and right view, and the target detection boxes corresponding to the left view and right view are used as key points and descriptors extracted in the corresponding image regions by the ORB algorithm. S103; Obtain key point matching pairs within the target area in the left and right views through brute-force matching, and combine the disparity features of the key point matching pairs with the corresponding actual distances to form input feature and label pairs, thus forming a dataset.
[0010] In this invention, the four hidden layers of the multilayer perceptron model include 64, 128, 128 and 32 neurons respectively, and the output layer includes 1 neuron.
[0011] In this invention, step S200 specifically includes the following steps: S201, use the standard deviation standardization method to normalize the input features and corresponding labels; S202, construct a multilayer perceptron model, extract and fuse features from the normalized features, and output the predicted distance of the target.
[0012] In this invention, the loss function is a smooth L1 loss function, the optimizer used for model optimization is the Adam optimizer, and the initial learning rate is 1e-4.
[0013] In this invention, the estimation method further includes: S500 calculates the error between the final predicted distance and the actual distance to the target, thereby evaluating the accuracy of the model's prediction.
[0014] The present invention further provides an underwater binocular vision distance estimation system based on key point matching and multilayer perceptron, comprising: The data acquisition module is used to acquire video data of the target using a binocular camera at a preset distance, acquire key point matching pairs in the target area in the left and right images based on the video data, and combine the disparity features of the key point matching pairs with the corresponding actual distances to form input feature and label pairs, thus forming a dataset. The multilayer perceptron processing module is used to extract and fuse input features using a multilayer perceptron model and output the predicted distance of the target. The multilayer perceptron model includes an input layer, four hidden layers and an output layer. The training optimization module is used to take the disparity features of the keypoint matching pairs as input, their corresponding actual distances as labels, and use the dataset and loss function to train and optimize the multilayer perceptron model. The distance estimation module is used to perform actual distance testing on the target using an optimized multilayer perceptron model to obtain the final predicted distance of the target.
[0015] In this invention, the estimation system further includes: The evaluation module is used to calculate the error between the final predicted distance and the actual distance to the target, thereby evaluating the accuracy of the model's prediction.
[0016] The technical solution provided by this invention may include the following beneficial effects: This invention presents an underwater binocular vision distance estimation method based on keypoint matching and multilayer perceptron. Using a multilayer perceptron model for distance measurement, it can handle errors in feature point extraction and matching, exhibiting stronger robustness. It avoids dependence on high-precision camera calibration and imaging models, reducing system deployment costs. The model is simple to train and has fast inference speed, making it suitable for low-power underwater platforms. It is applicable to scenarios such as underwater multi-agent collaborative perception, biomimetic robot navigation, and behavior recognition. Attached Figure Description
[0017] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure. It is obvious that the drawings described below are merely some embodiments of this disclosure, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort.
[0018] Figure 1 A flowchart illustrating an underwater binocular vision distance estimation method based on key point matching and multilayer perceptron in an exemplary embodiment of this disclosure is shown. Figure 2 This diagram illustrates a flowchart of S100 in an exemplary embodiment of this disclosure; Figure 3 This diagram illustrates a flowchart of S200 in an exemplary embodiment of this disclosure; Figure 4 A comparison diagram showing the conventional method and the method proposed in this invention is provided; Figure 5 This diagram illustrates the structure of a multilayer perceptron model in an exemplary embodiment of this disclosure. Figure 6 This illustrates the experimental site in an exemplary embodiment of this disclosure; Figure 7 This illustrates a binocular camera used for data acquisition in an exemplary embodiment of this disclosure; Figure 8 This illustration shows a biomimetic manta ray silicone prototype in an exemplary embodiment of the present disclosure; Figure 9 This illustration shows video samples acquired in an exemplary embodiment of this disclosure; Figure 10 The test results of the conventional method in the exemplary embodiments of this disclosure are shown; Figure 11 The test results of the method of the present invention in an exemplary embodiment of this disclosure are shown. Detailed Implementation
[0019] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, they are provided so that this disclosure will be more comprehensive and complete, and will fully convey the concept of the exemplary embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.
[0020] Furthermore, the accompanying drawings are merely illustrative diagrams of embodiments of this disclosure and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and therefore repeated descriptions of them will be omitted. Some block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities.
[0021] This example implementation first provides an underwater binocular vision distance estimation method based on keypoint matching and multilayer perceptron. Please refer to [reference needed]. Figure 1 This method may include: S101-S104. Specifically: S100, at a preset distance, video data of the target is acquired using a binocular camera, and key point matching pairs in the target area in the left and right images are obtained based on the video data. The disparity features of the key point matching pairs and the corresponding actual distances are combined to form input features and label pairs to form a dataset. S200: The multilayer perceptron model is used to extract and fuse the input features and output the predicted distance of the target. The multilayer perceptron model includes an input layer, four hidden layers and an output layer. S300, the disparity features of the keypoint matching pairs are used as input, and their corresponding actual distances are used as labels. The multilayer perceptron model is trained and optimized using the dataset and loss function. The S400 uses an optimized multilayer perceptron model to perform actual distance testing on the target and obtain the final predicted distance of the target.
[0022] In this embodiment, a multilayer perceptron model is used for distance measurement, which can handle errors in feature point extraction and matching and has stronger robustness; it avoids dependence on high-precision camera calibration and imaging models, reducing system deployment costs; the model is simple to train and has a fast inference speed, making it suitable for low-power underwater platforms; it is suitable for application scenarios such as underwater multi-agent collaborative perception, bionic robot navigation and behavior recognition.
[0023] The specific process of each step in the above embodiments will be described below.
[0024] In S100, please refer to Figure 8 This embodiment uses a biomimetic manta ray silicone prototype from Northwestern Polytechnical University as a carrier for data acquisition and algorithm verification. Please refer to... Figure 2 S100 includes the following steps: S101, at a preset distance, use a binocular camera to collect video data of the target, obtain left and right images based on the video data, and set them as left and right views respectively.
[0025] Specifically, a calibration device was first set up on the edge of the pool, aligning the target with the binocular camera (see reference). Figure 7 The distances between them are set to 1.0 m, 1.5 m, 2.0 m, 2.5 m, 3.0 m, and 4.0 m, for a total of six distances. Please refer to... Figure 6 and Figure 9Video data of the target was collected multiple times at each distance setting, and approximately 300 frames were sampled at each distance setting, for a total of 1400 samples. 1150 samples were used as the training set, and the remaining 250 as the validation set. Assume the input binocular images are the left view... and right view .
[0026] S102, the left view and right view are processed by a pre-trained target detection algorithm to obtain the target detection boxes corresponding to the target in the left view and right view. The target detection boxes corresponding to the left view and right view are used as key points and descriptors extracted by the ORB algorithm in the corresponding image regions.
[0027] Specifically, the left and right images are first processed using a pre-trained object detection algorithm (YOLOX is used in this invention) to obtain the bounding box coordinates of the object in the images. Let the object detection box in the left view be denoted as... The corresponding target detection box in the right view is Subsequently, based on the test results, and The ORB (Oriented FAST and Rotated BRIEF) algorithm is used as input to extract key points and descriptors within the corresponding image region.
[0028] S103; Obtain key point matching pairs within the target area in the left and right views through brute-force matching, and combine the disparity features of the key point matching pairs with the corresponding actual distances to form input feature and label pairs, thus forming a dataset.
[0029] Specifically, a set of matching keypoint pairs in the left and right images is obtained through brute-force matching (BFMatcher). To enhance robustness, the matching results are sorted by distance, and the top N pairs (N=20 in this invention) are selected as the input point set. The above is described by the following formula: (1) Feature extraction, where K and D are the set of key points and the descriptor subset, respectively.
[0030] (2) Key point matching
[0031] (3) Sort by distance and select the 20 points with the closest distance.
[0032] (4) Final set
[0033] (5) Calculate a set of disparities
[0034]
[0035] Finally, the disparity and corresponding true distance of all samples are collected into a single txt file for training and validation.
[0036] The above method fully preserves the information of parallax features in ranging, while reducing the dependence on the in-camera model involved in 3D reconstruction.
[0037] Please refer to Figure 3 S200 specifically includes the following steps: S201 uses the standard deviation standardization method to normalize the input features and their corresponding labels in order to eliminate the impact of differences in the dimensions of different features on model training.
[0038] S202, construct a multilayer perceptron model, extract and fuse features from the normalized features, and output the predicted distance of the target.
[0039] This invention constructs a multilayer perceptron (MLP) model to directly map a set of input disparity values to target distance. This MLP model comprises a six-layer network structure: one input layer, four hidden layers, and one output layer. The input layer receives a 20-dimensional disparity feature vector. The hidden layers contain 64, 128, 128, and 32 neurons respectively, all employing the ReLU activation function to introduce non-linear expressive power. The output layer consists of one neuron, outputting the estimated distance of the target in the camera coordinate system. Please refer to [reference needed]. Figure 5 The parameters of each layer of the model are as follows: Input layer: 5-dimensional disparity feature vectors after dimensionality reduction; First hidden layer: 64 neurons, fully connected layer; Second hidden layer: 128 neurons, ReLU activated; Third hidden layer: 128 neurons, ReLU activated; Fourth hidden layer: 32 neurons, ReLU activated; Output layer: 1 neuron, outputting an estimated distance to the target; Specifically, the network has Layer, number The input of the layer is The output is ,So:
[0040] in It is the input vector. It is the first The weight matrix of the layer, It is the first Layer bias vector, It is the first The activation function (ReLU) of the layer.
[0041] The parameters of each layer of the network are initialized using the Kaiming Normal initialization method to accelerate convergence and improve performance.
[0042] By learning the nonlinear mapping relationship between disparity features and actual distance, it is possible to effectively avoid ranging deviations caused by traditional matching errors and enhance stability and generalization ability under image quality fluctuations.
[0043] S300 is the process of training and optimizing the network model.
[0044] During the network training phase, the multilayer perceptron model is trained using the dataset constructed in step 1. The feature vector composed of keypoint disparities is used as input, with the corresponding ground truth distance as the label. SmoothL1Loss is used as the loss function to optimize the neural network parameters. During optimization, backpropagation combined with gradient descent (Adam optimizer) is used to iteratively update the model weights and minimize the error between the predicted and true values.
[0045] Specifically, the model training method is as follows: Training GPU: RTX 3090 24G; Loss function: Smooth L1 loss function is adopted, which combines the advantages of L1 and L2 loss and improves the robustness of the model; Optimizer: The Adam optimizer is used, with an initial learning rate of 1e-4; Learning rate scheduler: StepLR is used, and the learning rate decays to 0.1 of its original value every 10 training rounds; Training rounds: 200 rounds; Batch size: 64; Gradient clipping: Clips the gradient norm, setting the maximum value to 1.0 to prevent gradient explosion.
[0046] After the model training is completed, the trained distance estimation model is deployed to the biomimetic manta ray platform, and actual distance measurement tests are conducted on the target manta ray in the pool environment.
[0047] Based on the above embodiments, the estimation method of this application further includes the following steps: S500 calculates the error between the final predicted distance and the actual distance to the target, thereby evaluating the model's prediction accuracy. By statistically analyzing the error between the predicted result and the actual reference distance, the Mean Squared Error (MSE) is calculated to evaluate the model's estimation accuracy under different distance conditions. The MSE is defined as follows:
[0048] in, This represents the distance predicted by the model in the i-th iteration. This represents the corresponding true distance, and N is the number of test samples.
[0049] The mean squared error (MSE) index is calculated by collecting the error between the predicted distance and the true distance output by the model, thereby evaluating the prediction accuracy of the model. Simultaneously, the distance estimation method based on a multilayer perceptron proposed in this invention is compared with the traditional binocular ranging method based on 3D reconstruction to verify the improvement and superiority of this method in terms of accuracy, robustness, and dependence on camera parameters.
[0050] The underwater binocular vision distance estimation method based on key point matching and multilayer perceptron of this application will be further illustrated below through specific experimental examples.
[0051] (1) Model Deployment Import the trained model parameters (.pth format file) into the biomimetic manta ray main control system, and simultaneously load the standardized models (x_scale.pkl, y_scale.pkl) saved during the training phase. The deployed system can receive binocular image input and execute the following process: ① Use the ORB algorithm to extract matching point pairs in the target region; ② Calculate the horizontal disparity of the first 20 pairs of matching points; ③ Input the multilayer perceptron model for forward inference and output the target distance prediction value; ④ The predicted results are restored to the true distance values through inverse standardization.
[0052] (2) Performance testing environment In the experimental water tank environment, the target biomimetic manta ray was set at several known reference distances (1.0 m, 1.5 m, 2.0 m, 2.5 m, 3.0 m, 4.0 m), and multiple real-time ranging tests were conducted at each distance, and the model prediction results were recorded.
[0053] (3) Accuracy assessment By statistically analyzing the error between the predicted results and the actual reference distance, the mean squared error (MSE) index is calculated to evaluate the estimation accuracy of the model under different distance conditions.
[0054] (4) Comparative Experiment with Traditional Methods: To verify the superiority of the proposed method, a comparative experiment was conducted between the proposed multilayer perceptron-based distance estimation method and the traditional 3D reconstruction method (triangulation method based on binocular cameras). The comparison included: Prediction accuracy: As shown in Table 1, the method proposed in this invention outperforms traditional methods under various distance settings. Table 1 Test Results
[0055] Robustness: The stability of the prediction results is analyzed using a line graph of 100 consecutive frames of measurement results. The test results of traditional methods are as follows... Figure 10 As shown, the test results of the method in this application are as follows: Figure 11 As shown, traditional methods exhibit drastic fluctuations at multiple time points, while the method proposed in this application is more stable with less noticeable mutations.
[0056] The present invention also provides an underwater binocular vision distance estimation system based on keypoint matching and multilayer perceptron, comprising: The data acquisition module is used to acquire video data of the target using a binocular camera at a preset distance, acquire key point matching pairs in the target area in the left and right images based on the video data, and combine the disparity features of the key point matching pairs with the corresponding actual distances to form input feature and label pairs, thus forming a dataset. The multilayer perceptron processing module is used to extract and fuse input features using a multilayer perceptron model and output the predicted distance of the target. The multilayer perceptron model includes an input layer, four hidden layers and an output layer. The training optimization module is used to take the disparity features of the keypoint matching pairs as input, their corresponding actual distances as labels, and use the dataset and loss function to train and optimize the multilayer perceptron model. The distance estimation module is used to perform actual distance testing on the target using an optimized multilayer perceptron model to obtain the final predicted distance of the target.
[0057] Regarding the system in the above embodiments, the specific manner in which each module performs its operations has been described in detail in the embodiments related to the method, and will not be elaborated upon here.
[0058] It should be noted that although several modules of the system for executing actions are mentioned in the detailed description above, this division is not mandatory. In fact, according to embodiments of the present invention, the features and functions of two or more modules described above can be embodied in one module. Conversely, the features and functions of one module described above can be further divided into multiple modules for embodiment. Components shown as modules may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of the present invention according to actual needs. Those skilled in the art can understand and implement this without any inventive effort.
[0059] The embodiments of the present invention have been described above with reference to the accompanying drawings. However, the present invention is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of the present invention without departing from the spirit and scope of the claims. All of these forms are within the protection scope of the present invention.
Claims
1. An underwater binocular vision distance estimation method based on keypoint matching and multilayer perceptron, characterized in that, Includes the following steps: S100, at a preset distance, video data of the target is acquired using a binocular camera, and key point matching pairs in the target area in the left and right images are obtained based on the video data. The disparity features of the key point matching pairs and the corresponding actual distances are combined to form input features and label pairs to form a dataset. S200: The multilayer perceptron model is used to extract and fuse the input features and output the predicted distance of the target. The multilayer perceptron model includes an input layer, four hidden layers and an output layer. S300, the disparity features of the keypoint matching pairs are used as input, and their corresponding actual distances are used as labels. The multilayer perceptron model is trained and optimized using the dataset and loss function. The S400 uses an optimized multilayer perceptron model to perform actual distance testing on the target and obtain the final predicted distance of the target.
2. The underwater binocular vision distance estimation method based on keypoint matching and multilayer perceptron according to claim 1, characterized in that, The preset distances include: 1.0 m, 1.5 m, 2.0 m, 2.5 m, 3.0 m and 4.0 m.
3. The underwater binocular vision distance estimation method based on keypoint matching and multilayer perceptron as described in claim 1, characterized in that, S100 specifically includes the following steps: S101, at a preset distance, use a binocular camera to collect video data of the target, obtain left and right images based on the video data, and set them as left and right views respectively; S102, the left view and right view are processed by a pre-trained target detection algorithm to obtain the target detection boxes corresponding to the target in the left view and right view, and the target detection boxes corresponding to the left view and right view are used as key points and descriptors extracted in the corresponding image regions by the ORB algorithm. S103; Obtain key point matching pairs within the target area in the left and right views through brute-force matching, and combine the disparity features of the key point matching pairs with the corresponding actual distances to form input feature and label pairs, thus forming a dataset.
4. The underwater binocular vision distance estimation method based on keypoint matching and multilayer perceptron as described in claim 1, characterized in that, The four hidden layers of the multilayer perceptron model include 64, 128, 128 and 32 neurons respectively, and the output layer includes 1 neuron.
5. The underwater binocular vision distance estimation method based on keypoint matching and multilayer perceptron as described in claim 1, characterized in that, S200 specifically includes the following steps: S201, use the standard deviation standardization method to normalize the input features and corresponding labels; S202, construct a multilayer perceptron model, extract and fuse features from the normalized features, and output the predicted distance of the target.
6. The underwater binocular vision distance estimation method based on keypoint matching and multilayer perceptron according to claim 1, characterized in that, The loss function is a smooth L1 loss function, and the optimizer used for model optimization is the Adam optimizer with an initial learning rate of 1e-4.
7. The underwater binocular vision distance estimation method based on keypoint matching and multilayer perceptron according to any one of claims 1 to 6, characterized in that, The estimation method further includes: S500 calculates the error between the final predicted distance and the actual distance to the target, thereby evaluating the accuracy of the model's prediction.
8. An underwater binocular vision distance estimation system based on keypoint matching and multilayer perceptron, characterized in that, include: The data acquisition module is used to acquire video data of the target using a binocular camera at a preset distance, acquire key point matching pairs in the target area in the left and right images based on the video data, and combine the disparity features of the key point matching pairs with the corresponding actual distances to form input feature and label pairs, thus forming a dataset. The multilayer perceptron processing module is used to extract and fuse input features using a multilayer perceptron model and output the predicted distance of the target. The multilayer perceptron model includes an input layer, four hidden layers and an output layer. The training optimization module is used to take the disparity features of the keypoint matching pairs as input, their corresponding actual distances as labels, and use the dataset and loss function to train and optimize the multilayer perceptron model. The distance estimation module is used to perform actual distance testing on the target using an optimized multilayer perceptron model to obtain the final predicted distance of the target.
9. The underwater binocular vision distance estimation system based on key point matching and multilayer perceptron according to claim 8, characterized in that, The estimation system also includes: The evaluation module is used to calculate the error between the final predicted distance and the actual distance to the target, thereby evaluating the accuracy of the model's prediction.