Method for binocular vision positioning of unmanned ship in near-shore environment based on far and near scene segmentation

Abstract

The invention discloses an unmanned ship binocular vision positioning method in a near-shore environment based on far and near scene segmentation, and relates to the field of unmanned ship positioning, and the method comprises the steps: obtaining an image of the near-shore environment collected by an unmanned ship binocular camera; dividing the image into a long-shot area and a close-shot area by adopting an OTSU algorithm; extracting feature points of a long-shot area and a close-shot area in the image by adopting an ORB algorithm; performing feature point matching on the feature points of the two adjacent frames of images to obtain an initial pose of the unmanned ship; and screening and inserting a new key frame, and optimizing the initial pose and the local map to obtain the real-time pose of the unmanned ship so as to realize the positioning of the unmanned ship. According to the invention, accurate positioning of the unmanned ship can be realized.

Description

Technical Field

[0001] The present application relates to the field of unmanned ship positioning, and in particular to a binocular vision positioning method for unmanned ships in a nearshore environment based on long and short scene segmentation. Background Art

[0002] Modern unmanned surface vessels (USVs) have been widely used in various water projects, especially in nearshore waters. Unmanned vessels can complete a variety of complex tasks, including maritime search and rescue, depth detection in shallow waters, and three-dimensional modeling of underwater terrain, and have great application prospects.

[0003] The key to autonomous operation of unmanned ships is to accurately locate themselves and perceive the surrounding environment. The traditional method uses the Automatic Identification System (AIS) of ships and uses satellites for positioning. However, problems such as environmental shielding and electromagnetic interference in the nearshore environment affect the accuracy of satellite positioning methods. Therefore, researchers have proposed using measurement methods to perceive the surrounding environment and achieve self-positioning. The measurement methods mainly use lidar ranging and satellite image fusion, but such methods still have problems such as low cost performance and poor positioning accuracy. Therefore, for the intelligent navigation of unmanned ships in nearshore environments, it is an urgent problem to explore low-cost and high-precision positioning methods.

[0004] Cost-effective visual positioning technology has become increasingly popular in recent years. Compared with the widespread application of visual odometers in indoor robot projects, visual positioning in nearshore environments has many challenges. There are reflections and ripples in nearshore environment images, which poses a challenge to feature extraction; the perspective of unmanned ships frequently shakes when running on the water, which makes feature tracking difficult. At the same time, the nearshore environment is relatively open, and depth information is difficult to measure. Summary of the invention

[0005] In order to solve the above problems, the present application provides a binocular vision positioning method for unmanned ships in a nearshore environment based on long and short scene segmentation.

[0006] To achieve the above objectives, this application provides the following solutions:

[0007] A binocular vision positioning method for an unmanned ship in a nearshore environment based on long and short scene segmentation, comprising:

[0008] Acquire images of the nearshore environment collected by the unmanned ship's binocular camera;

[0009] The image is divided into a distant view area and a near view area by using an OTSU algorithm;

[0010] The ORB algorithm is used to extract feature points of the distant view area and the near view area in the image;

[0011] Match the feature points of two adjacent frames of images to obtain the initial position and posture of the unmanned ship;

[0012] Filter and insert new key frames, optimize the initial pose and local map, obtain the real-time pose of the unmanned ship, and realize the positioning of the unmanned ship.

[0013] According to the specific embodiments provided in this application, this application discloses the following technical effects:

[0014] The present application provides a binocular vision positioning method for unmanned boats in a nearshore environment based on long-range and short-range segmentation, and uses the ORB algorithm to extract feature points of the long-range and short-range areas in the image; because the ORB algorithm uses image pyramids and centroid direction angle detection, each feature has scale invariance and rotation invariance, which can solve the problem of difficulty in feature extraction due to reflections, ripples, etc. in nearshore environment images. In addition, the present application optimizes the screening conditions of key frames, predicts the tracking quality, and creates key frames in time for local optimization, which solves the problem of frequent visual jitter of unmanned boats when running on the water, which leads to difficulty in feature tracking. Therefore, the present application can achieve accurate positioning of unmanned boats. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0016] Figure 1 This is a flow chart of a method for binocular vision positioning of an unmanned ship in a nearshore environment based on long and short scene segmentation in one embodiment of the present application;

[0017] Figure 2 Schematic diagram of the dilation principle; (a) is the original image, (b) is the structural element, and (c) is the repaired image;

[0018] Figure 3 Schematic diagram of splitting; the image block in (a) is 1, the image block in (b) is 4, the image block in (c) is 14; the image block in (d) is 26;

[0019] Figure 4 Schematic diagram of unmanned ship trajectory drawn using different algorithms;

[0020] Figure 5 A schematic diagram of using different structural elements to segment images of different weather conditions and different pyramid levels;

[0021] Figure 6Schematic diagram of feature points in distant view area and near view area. DETAILED DESCRIPTION

[0022] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0023] The visual positioning method designed in this application is: perform preliminary feature tracking on the binocular images obtained by the binocular camera to obtain preliminary inter-frame posture changes; then determine whether the current frame meets the conditions to become a key frame, and if so, insert a new key frame for local optimization, which will output the optimized key frame posture and map point coordinates; finally, determine whether the current key frame overlaps with the previous one, and decide whether to make the final correction in turn. Finally, output the posture changes of the unmanned ship at each moment to obtain the real-time posture of the unmanned ship.

[0024] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0025] In an exemplary embodiment, Figure 1 As shown, a binocular vision positioning method for an unmanned ship in a nearshore environment based on long and short scene segmentation is provided. The method is executed by a computer device, and can be specifically executed by a computer device such as a terminal or a server alone, or by a terminal and a server together. In an embodiment of the present application, the method is applied to a server as an example for explanation, including the following steps S1 to S5.

[0026] in:

[0027] S1: Obtain images of the nearshore environment collected by the unmanned ship's binocular camera.

[0028] S2: Using the OTSU algorithm to divide the image into a distant view area and a near view area.

[0029] S3: extracting feature points of the distant view area and the near view area in the image using the ORB algorithm.

[0030] S4: Match the feature points of two adjacent frames of images to obtain the initial position and posture of the unmanned ship.

[0031] S5: Filter and insert new key frames, and optimize the initial pose and local map to obtain the real-time pose of the unmanned ship and realize the positioning of the unmanned ship.

[0032] Through the above steps S1 to S5, the present application can achieve accurate positioning of the unmanned ship.

[0033] In a specific embodiment, step S2 specifically includes: using the OTSU algorithm to determine a division threshold, and iteratively calculating the division threshold; based on the iterative division threshold, using a binarization method to determine the distant view area and the near view area.

[0034] Specifically, in order to reduce the impact of the erroneous depth of long-distance feature points on inter-frame feature tracking, the present application divides the image of the nearshore environment captured by the unmanned ship binocular camera into a distant view area and a near view area. Since there are areas such as the sky in the image, the pixel intensity of different areas is specific, so a suitable threshold is calculated for preliminary division. The specific threshold is selected using the OTSU algorithm, which adopts the principle of maximum inter-class variance and is suitable for the case where the grayscale distribution of the image presents a bimodal state. It will adaptively find a suitable threshold, as shown in formula (1).

[0035] g=w0(u0-u) 2 +w1(u1-u) 2 (1)

[0036] In the formula, g is the segmentation threshold, w0 and w1 are the proportions of near view and far view respectively, u0 and u1 are the average grayscale values ​​of foreground and background respectively, and u is the global image average threshold. Facing the drastic lighting changes in the nearshore environment, the image may experience sudden exposure or low illumination. This will lead to insufficient segmentation, so the segmentation threshold is iteratively calculated, and a sliding window is used to monitor the pixel intensity histogram over a period of time to prevent the occurrence of sudden changes in the segmentation threshold. Finally, a binarization method (such as formula (2)) is used to obtain a rough far view area.

[0037]

[0038] After segmenting the distant area in the image, the morphological dilation operation is used to smooth the area near the boundary. The dilation operation detects pixel changes in the target area to expand the target boundary outward and fill small gaps and breaks. The dilation principle is as follows: Figure 2 As shown in (a)-(c) in .

[0039] In a specific embodiment, step S3 specifically includes:

[0040] Since the nearshore environment is relatively open and the average depth is large, there is great uncertainty in determining the depth of distant feature points. At the same time, the unmanned ship is constantly impacted by the water flow, and its own perspective frequently shakes, which has a great impact on feature tracking. This application uses the ORB algorithm as a means of feature extraction. This method uses image pyramid and centroid direction angle detection to make each feature scale invariant and rotation invariant.

[0041] In order to solve the problem of uneven distribution of image features, the present application uses a quadtree algorithm to perform equalization processing on the feature points after step S3.

[0042] The quadtree data structure is used to recursively divide the image into smaller regions and the minimum unit of image division is controlled according to the number of feature points in each region. The specific method is as follows:

[0043] The center of the image is taken as the root node, and the image is divided into four image blocks: upper left, lower left, upper right, and lower right, with the root node as the center. The center of each image block is taken as the four child nodes of the first root node, and the first-level quadtree is constructed first.

[0044] The number of feature points contained in these four image blocks is counted, and different operations are performed according to the number: the image block with the number of feature points of 1 is no longer split; the image block with the number of feature points of 0 directly deletes its corresponding node; the node with the number of feature points greater than 1 continues to be split into four nodes.

[0045] The above operation is repeated iteratively, and the splitting stops when the total number of nodes reaches the set value.

[0046] The total number of nodes is set according to the size of the image itself. In the general splitting process, there will be multiple nodes to be split at the same time. At this time, they are sorted according to the number of feature points contained in the node image block. The image block nodes with more feature points are split first. The specific splitting diagram is as follows Figure 3 shown.

[0047] After extracting the feature points, the feature points in different areas are marked according to the area where the feature points are located and the area information divided in the near and far scene segmentation.

[0048] In a specific embodiment, step S4 specifically includes: matching feature points of two adjacent frames of images, estimating the motion increment of the two adjacent frames of images by minimizing the projection error; and obtaining the initial position and posture of the unmanned ship based on the motion increment. Specifically as follows:

[0049] (1) Binocular registration depth measurement

[0050] Binocular matching depth measurement uses the left and right eye images of the same stereo frame for matching. The matching point pairs are feature points in the left and right eye images at the same time. The purpose is to measure the depth of the feature point at the current moment, thereby obtaining the relative position of the camera and the feature point at the current moment.

[0051] Feature tracking estimates the motion increment between two frames by minimizing the projection error. This process requires obtaining the depth value of the feature point relative to the binocular camera (hereinafter referred to as the camera). In terms of binocular registration depth measurement calculation, this application adopts the "coarse matching-fine matching-sub-pixel interpolation" process. In the coarse matching stage, the search is performed along the camera's polar line direction, and the Hamming distance (as shown in formula (3)) is used to match all feature points in turn to find the best matching point.

[0052]

[0053] Among them, a and b are the descriptors of two feature points. is a bitwise XOR operator. After obtaining the rough matching coordinates, a square sliding window is set near the coordinates obtained by the rough matching, and the absolute sum of differences (SAD) method is used to calculate the best matching neighborhood to obtain the precise matching correction value, as shown in formula (4).

[0054]

[0055] Where (x, y) is the coordinate of the upper left corner of the search window. I(x+i, y+j)| is the SDA value of image I at position (x+i, y+j). (x * ,y * ) is the coordinate of the upper left corner of the search window that minimizes SAD. A(i,j) represents the pixel value of the image block to be matched. Since the position of the best matching point does not necessarily happen to be located at a certain pixel, the disparity may be a sub-pixel result. In the precise matching process, the similarity between each image block in the sliding window and the image block near the feature point to be matched is mathematically a parabola structure. The establishment of this parabola model only requires five sets of parameters, and the best five sets of data in the precise matching are used to construct the parabola model. And calculate the minimum value of the parabola, which represents the local extreme value in the sliding window, that is, the optimal matching point.

[0056] Finally, the sub-pixel disparity is obtained from the matching points, and the depth value of the feature point is calculated using formula (5).

[0057]

[0058] (2) Feature point matching between frames

[0059] The left-eye images between consecutive frames are used for matching. The purpose of this step is to obtain matching point pairs at different times, and to calculate the camera's posture changes based on the positions between the feature points and the camera at different times to achieve positioning.

[0060] This application uses a fast matching strategy based on a grid method. This strategy is used when the system is just started and the camera motion state is not yet determined. If brute force matching is used, the amount of calculation will be greatly increased, so the quadtree grid in the feature extraction stage is used for fast matching.

[0061] After experiments, it was found that the matching feature points between two adjacent frames in the initial stage would appear in a circular range. Therefore, all grids in this circular range were searched and matched using the Hamming distance.

[0062] During the operation of the system, the posture is optimized by minimizing the reprojection error. At this time, the matching relationship between the projection points of the previous frame and the feature points of the current frame is required. Assuming that the camera moves at a uniform speed, the motion increment of the previous frame is applied to the camera extrinsic parameters of the current frame, where the camera extrinsic parameters are composed of the rotation matrix R and the translation vector t.

[0063] During the reprojection calculation process, the initial motion pose R, t is first applied to the three-dimensional feature point set X in the world coordinate system extracted from the previous frame. t-1 (w) , converted to the camera coordinate system of the current frame. Then use the camera intrinsic parameter matrix K to project the 3D feature point set in the camera coordinate system onto the image plane, and normalize the 3D feature point set according to the depth information. The projection process is shown in formula (6):

[0064]

[0065] After reprojection, it is necessary to select different levels of pyramids for feature matching according to the camera's forward or backward movement. The direction of travel is determined based on the assumed posture transformation, and matching points are searched in adjacent pyramid levels according to the principle of "near is big, far is small". The specific search still follows the search strategy based on grid division. For different levels of pyramids, the size of the grid is also scaled accordingly. In the case of multiple approximate matching points, they are directly filtered to ensure that all matching points are correct enough.

[0066] (3) Deep information outlier detection

[0067] In the feature matching process of consecutive frames, the same feature point is continuously tracked. The depth information of the feature point is generated in each stereo frame. A sliding window is used to detect the changes in the depth information of the tracked feature points in consecutive frames, and the RANSAC method is used to filter outliers. A probability distribution model is established based on the frequency of outliers to determine the uncertainty of the depth information of the current frame. When the total projection error is taken into account, the weighting coefficient is set according to the uncertainty as shown in formula (7).

[0068]

[0069] (4) Inter-frame motion estimation

[0070] In the feature matching process, the reprojected matching point pairs are obtained, and the projection point set X t-1 (proj) The difference is made with the actual observation point set X_t of the matched current frame to obtain the reprojection error of all feature points. The difference calculation method uses the Euclidean distance, as shown in formula (8).

[0071]

[0072] The reprojection error set of the current frame is obtained. According to the previous division of distant view and near view, the reprojection error set is divided into two parts: For near-view areas with high reliability, they are directly included in the total projection error. For far-view areas with erroneous depth information, the confidence of the error is adjusted using the weight calculated in outlier detection.

[0073] After the total reprojection error is calculated, the objective function is established based on it, as shown in formula (9), T * The approximate pose obtained after minimizing the total projection error is the initial pose of the unmanned ship. The camera's self-motion is iteratively estimated using the Levenberg-Marquardt method. Since the camera and the unmanned ship are rigidly connected, the pose estimation result of the camera is similar to the pose estimation result of the unmanned ship.

[0074]

[0075] Finally, the motion estimate (i.e., motion increment) between two consecutive frames and its uncertainty are obtained. Specifically, it is expressed in the form of a normal distribution: in The six-degree-of-freedom vector representing the camera motion between the previous frame and the current frame, Represents the uncertainty covariance of this estimate, which is approximated by calculating the Hessian matrix from the error (Jacobian matrix) in the last iteration.

[0076] In a specific embodiment, step S5 specifically includes: screening and inserting key frames based on the number of frames from the current frame image to the previous key frame, or the number of tracking losses of feature points of the current frame image relative to the feature points of the previous key frame, the number of tracking map points, or the tracking stability of feature points.

[0077] Specifically, in order to solve the problem of tracking failure caused by frequent viewpoint jitter in nearshore environments, this application optimizes the keyframe screening conditions, predicts the tracking quality, and creates keyframes in time for local optimization. At the same time, it also detects the depth of incorrect feature points and updates them, and uses triangulation to correct the depth value through new matching relationships. Finally, the matching relationship between keyframes is used to perform local pose optimization and map point optimization to build a complete map.

[0078] ()Key frame filtering

[0079] During the feature tracking process, the current frame will be continuously screened to see if it can be used as a key frame. When a new key frame is confirmed, it will be inserted into the local mapping thread of SP-SLAM for further optimization. There are three main conditions for the selection of key frames. A key frame can be inserted if any one of them is met:

[0080] 1) It has been a while (interval frames) since the last key frame was generated, or enough motion has occurred, or 70% of the feature point tracking for the last key frame has been lost.

[0081] 2) When 50% of the feature points in the previous frame have been lost.

[0082] 3) Use formula (10) to convert the uncertainty of each motion estimation from the Hessian matrix to a scalar, and use a sliding window to monitor the change of uncertainty in the pose estimation of several frames. When the change rate is greater than 30%, it means that the feature tracking has entered an unstable state and a key frame needs to be inserted. The calculation formula for the tracking stability of the feature point is:

[0083]

[0084] (2) Key frame insertion and matching

[0085] After a new keyframe is added to the local mapping thread, the camera pose of the new keyframe, the 3D position of the feature points under the keyframe in the world coordinate system and their matching relationship will be obtained. A new match will be made in the previous keyframe database to optimize the pose and map points. The specific updated data are:

[0086] 1) The position, common view relationship and optimal descriptor of the new map point in three-dimensional space;

[0087] 2) The common view group, observation range, and average observation direction of the new key frame.

[0088] In the local mapping thread, projection matching is still used to build new matching relationships, projecting map points in past keyframes into new keyframes to build matching relationships. If the current keyframe has more than 15 matching relationships with the previous keyframe, it is considered a common view keyframe. The establishment of these common view keyframes facilitates a new match after the subsequent keyframes are inserted.

[0089] (3) Local BA

[0090] Local bundle adjustment (BA) is a common method for improving the accuracy of local maps. This method is still based on minimizing the reprojection error. By optimizing the camera pose and the position of map points in the local area, more constraints are used to iteratively calculate more accurate values.

[0091] Different from the feature tracking stage, the goal of local BA is to minimize the reprojection error of all observation points by adjusting the camera pose and map point position simultaneously. The reprojection error is defined as follows:

[0092] e ij =z ij -π(T j ,P i ) (11)

[0093] Where: z ij It is the map point P i At key frame K j The observation position (pixel coordinate) on π(T j ,P i ) is the camera projection model that transforms the map point P i According to the key frame pose T j The result of the calculation projected onto the image plane.

[0094] Since the positions of map points are remeasured when the key frame is initialized, all errors are assigned a uniform weight when constructing the local BA objective function. The specific objective function is expressed as follows:

[0095]

[0096] This application still uses the Levenberg-Marquardt method for iterative optimization and solution, and uses the pseudo-Huber loss function to handle outliers to ensure data reliability.

[0097] After step S5, the method further includes: performing closed-loop detection on the real-time position and posture of the unmanned ship; the closed-loop detection includes loop detection and global correction.

[0098] The goal of loop closure detection is to identify that the camera has returned to a previously visited scene during a long SLAM operation. By detecting and processing loop closures, the cumulative error can be significantly reduced and the consistency and accuracy of the global map can be improved. When USVs are operating in near-shore environments, loop closure detection is still a necessary task.

[0099] (1) Loop detection

[0100] Loop detection is mainly divided into two parts: the search for the initial closed-loop keyframe and the verification of the continuous closed-loop keyframe. After a new keyframe is inserted, the method searches for the best matching frame in the keyframe database through the bag-of-words model matching, and determines whether a closed loop occurs through conditions. If the detection passes, the best matching frame is set as a candidate closed-loop keyframe.

[0101] When closed loops are detected continuously, a similarity transformation is performed on the new keyframe and the candidate closed loop keyframe, specifically using Sim(3). This method uses two sets of 3D matching points to construct an optimization equation, and can directly solve the rotation and translation relationship between the two sets of 3D matching points without iteration. Based on this method, the verification of continuous closed loop keyframes is performed, and the verification mainly follows the following three principles:

[0102] 1) Whether the closed loop is detected in continuous key frames of sufficient length;

[0103] 2) Whether the projection matching between the feature points in the new keyframe and the map points in the previous keyframe is accurate enough;

[0104] 3) The pose change between the new keyframe and the previous keyframe obtained by similarity transformation through Sim (3) is less than 1.2m or 5°

[0105] (2) Global correction

[0106] Closed-loop correction actually applies the result of the Sim(3) transformation calculated in the previous section to all keyframe poses and map points. Errors accumulate during continuous feature tracking and local mapping. After the system is closed-loop, the accumulated error is manifested as the pose error between the closed-loop frame and the previous keyframe. The Sim(3) transformation calculates the specific value of this error and propagates the pose to all keyframe poses and map points for global correction.

[0107] In order to evaluate the performance of the method proposed in this application, relevant experiments were conducted on the USVInland Dataset, a dataset of water surface environments. USVInland is the first multi-sensor dataset for unmanned surface vehicles in inland waterways. This application obtained 6 sequences in USVInland through scientific research application.

[0108] This application uses the data collected by RTK in the USVInland dataset as the real motion trajectory, and plots the output trajectories of the ORB-SLAM2 algorithm based on the feature point method, the binocular DSO algorithm based on the direct method, and the algorithm of this application, and plots these trajectories together for visual qualitative comparison. The six sequences of the experiment have certain differences in weather conditions, travel distances, and round trip conditions. The specific trajectories are as follows: Figure 4 shown.

[0109] Figure 4 (a) in the figure is the N02-4 sequence, which includes a large rotation movement and a bridge tunnel scene. The method of the present application maintains better accuracy; while ORB-SLAM2 and stereo DSO are affected by various factors and incorrectly estimate the distance traveled. Figure 4 (b) is the N03-3 sequence, which is a movement process of sailing along the shore vegetation. At the same time, the distant view contains information such as buildings, which is difficult to determine the depth. The method of the present application accurately restores this trajectory. Figure 4 (c) in the figure is the N03-4 sequence, which is moving along a river. The surface of the river contains some synchronously moving hulls, which poses a great challenge to feature tracking and causes certain degradation during the movement. The estimation of the scale and rotation of the binocular DSO is affected, but the method of the present application still accurately restores the movement trajectory. Figure 4 (d)-(f) in the figure are N03-5 sequence, H05-6 sequence, and W06-2 sequence respectively. In these sequences, the traditional ORB-SLAM2 algorithm has mis-estimated the depth. The performance of using stereo DSO in these sequences is poor, and even tracking loss occurs. Thanks to the distance segmentation method used in this application, the impact of this part of the wrong depth on the final result is reduced, and a more accurate result is obtained.

[0110] From the motion trajectory comparison diagram of the above sequence, it can be seen that the traditional ORB-SLAM2 method can track well in rotational motion, and restores the small turns and large rotations contained in the real trajectory well. However, in the face of translational motion in posture tracking, it is impossible to restore the normal scale. The binocular DSO using the direct method is affected by the complex lighting changes on the water surface, and it is difficult to restore the posture. The method proposed in this application has better completed the scale restoration work and obtained a more accurate trajectory.

[0111] This application also conducted separate experiments on the segmentation effect of distant and near scenes, using different structural elements to segment images of different weather conditions and different pyramid levels. The specific effects are as follows: Figure 5 shown.

[0112] The experiment tested nearshore environment images in different weather conditions and at different scales. Figure 5 The first column of sub-images is a cloudy day, the second column of sub-images is a rainy day; the third column of sub-images is a sunny day, and the fourth and fifth columns of sub-images are sunny days reduced by 2 times and 4.2 times. Use different structural elements and analyze their effects on the segmentation results. Figure 5 The third row of sub-images uses a circular structure with a radius of 15 pixels, the fourth row of sub-images uses a circular structure with a radius of 25 pixels, and the fifth row of sub-images uses a square structure with a length of 25 pixels and a width of 15 pixels.

[0113] The experimental results show that the edges of the near and far scenes segmented by circular structural parts are smoother, and the square structural parts are more advantageous in the expansion and supplement of the connected area. Combined with the basic requirements of near and far scene segmentation in nearshore visual positioning, a circular structural part with a radius of 25 pixels was finally selected.

[0114] After distance segmentation, feature extraction of the image can effectively determine the distant points, such as Figure 6 As shown, the red one is the far point marked. Figure 6 (a) in Figure 1 extracts most of the far points very close to the shore. Figure 6 In the open large-scale scene (b), the distant area is also successfully divided; facing the white sky and wall, the feature points at the junction of the distant and sky parts are successfully marked, avoiding the large light-colored area near the white wall.

[0115] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM may be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0116] The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. The non-relational database may include a distributed database based on blockchain, etc., but is not limited thereto. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but is not limited thereto.

[0117] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0118] This article uses specific examples to illustrate the principles and implementation methods of this application. The description of the above embodiments is only used to help understand the method and core ideas of this application. At the same time, for those skilled in the art, according to the ideas of this application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.

Claims

1. A binocular vision positioning method for unmanned ships in a nearshore environment based on long and short scene segmentation, characterized in that: include: Acquire images of the nearshore environment collected by the unmanned ship's binocular camera; The image is divided into a distant view area and a near view area by using an OTSU algorithm; The ORB algorithm is used to extract feature points of the distant view area and the near view area in the image; Match the feature points of two adjacent frames of images to obtain the initial position and posture of the unmanned ship; Filter and insert new key frames, optimize the initial pose and local map, obtain the real-time pose of the unmanned ship, and realize the positioning of the unmanned ship.

2. The binocular vision positioning method for unmanned ships in a nearshore environment based on long and short scene segmentation according to claim 1 is characterized in that: The image is divided into a distant view area and a near view area using the OTSU algorithm, specifically including: The OTSU algorithm is used to determine the partition threshold, and the partition threshold is iteratively calculated; Based on the iterative segmentation threshold, the binarization method is used to determine the distant view area and the near view area.

3. The binocular vision positioning method for unmanned ships in a nearshore environment based on long and short scene segmentation according to claim 1 is characterized in that: After the ORB algorithm is used to extract the feature points of the distant view area and the near view area in the image, the method further includes: using a quadtree algorithm to perform equalization processing on the feature points.

4. The binocular vision positioning method for unmanned ships in a nearshore environment based on long and short scene segmentation according to claim 1 is characterized in that: The feature points of two adjacent frames are matched to obtain the initial position of the unmanned ship, including: The feature points of two adjacent frames are matched, and the motion increment of the two adjacent frames is estimated by minimizing the projection error. The initial position and posture of the unmanned ship is obtained based on the motion increment.

5. The binocular vision positioning method for unmanned ships in a nearshore environment based on long and short scene segmentation according to claim 4 is characterized in that: The calculation formula of the initial position of the unmanned ship is: Among them, T * is the initial position of the unmanned ship, is the reprojection error set of the near-field area of ​​the current frame image, w t is the reprojection error weighting coefficient of the distant area of ​​the current frame image, is the reprojection error set of the distant area of ​​the current frame image, T is the pose in the last optimization, i is the sequence number of iterations, and n is the total number of iterations.

6. The binocular vision positioning method for unmanned ships in a nearshore environment based on long and short scene segmentation according to claim 1 is characterized in that: Filter and insert keyframes, including: The keyframes are filtered and inserted based on the number of frames from the current frame image to the previous keyframe, or the number of tracking losses of feature points of the current frame image relative to feature points of the previous keyframe, the number of tracked map points, or the tracking stability of feature points.

7. The binocular vision positioning method for unmanned ships in a nearshore environment based on long and short scene segmentation according to claim 6 is characterized in that: The calculation formula for the tracking stability of feature points is: Where h(ξ) is the scalar result of tracking stability ξ, is the six-degree-of-freedom vector of the camera motion between the previous frame image and the current frame image, Represents the uncertainty covariance of this estimate.

8. The binocular vision positioning method for unmanned ships in a nearshore environment based on long and short scene segmentation according to claim 1 is characterized in that: After filtering and inserting new keyframes, optimizing the initial pose and local map, obtaining the real-time pose of the unmanned ship, and realizing the positioning of the unmanned ship, it also includes: A closed-loop detection is performed on the real-time position and posture of the unmanned ship; the closed-loop detection includes loop closure detection and global correction.

Images