A target tracking and positioning method based on non-parametric classification and spatial credibility model

Abstract

The application discloses a target tracking positioning method based on non-parametric classification and composite reliability, relates to the field of target tracking, and comprises the following steps: step 1, designing an underwater video target tracking positioning system based on an imaging sonar and a binocular camera; step 2, establishing an underwater moving target multi-mode detection method based on non-parametric pixel classification; step 3, constructing a correlation filter learning model based on composite reliability, containing spatial reliability and multi-feature multi-channel reliability; step 4, solving a correlation filter learning optimization problem based on composite reliability; and step 5, completing joint tracking positioning based on the binocular camera and the imaging sonar. The application can improve the reliability and effectiveness of underwater robots in unstructured environments, and provides theoretical and technical support for the development of underwater robots, deep sea exploration and space robots in China.

Description

Technical Field

[0001] This invention relates to the field of video target tracking, and in particular to an underwater target tracking and positioning system and method based on non-parametric classification and composite confidence. Background Technology

[0002] Underwater robots can be defined as devices that move on or underwater, have perception systems such as vision and acoustics, and use robotic arms or other tools to replace or assist humans in performing certain surface and underwater tasks through remote control or autonomous operation.

[0003] However, most current underwater robots or autonomous underwater operation systems do not fully consider the joint tracking and localization of binocular cameras and imaging sonar in dynamic flow fields. This problem poses a significant challenge to subsequent autonomous operations, such as docking and maneuvering. In other words, the lack of joint tracking and localization information from binocular cameras and imaging sonar makes underwater autonomous control extremely challenging.

[0004] A literature review of existing technologies revealed that current underwater robot operation and control platforms have not addressed the joint tracking and positioning problem of binocular cameras and imaging sonar in dynamic flow fields. Specifically, existing tracking and positioning methods only target single sensors and do not consider challenges such as underwater target occlusion, apparent nonlinear high-dynamic changes in underwater targets, and rapid movement in dynamic flow fields.

[0005] Therefore, those skilled in the art are dedicated to developing an underwater target tracking and positioning system and method based on non-parametric classification and composite reliability, to provide stable and effective positioning information for underwater unmanned platforms. Summary of the Invention

[0006] In view of the above-mentioned deficiencies of the prior art, the technical problem to be solved by the present invention is an underwater target tracking and positioning system and method in a dynamic flow field.

[0007] To achieve the above objectives, this invention provides a target tracking and localization method based on a non-parametric classification and spatial credibility model, characterized in that the method includes the following steps:

[0008] Step 1: Design an underwater video target tracking and positioning system based on imaging sonar and underwater binocular camera;

[0009] Step 2: Establish a multi-mode underwater moving target detection method based on non-parametric pixel classification;

[0010] Step 3: Construct a correlation filter learning model based on composite credibility;

[0011] Step 4: Solve the correlation filter learning optimization problem based on composite confidence.

[0012] Step 5: Complete the joint tracking and positioning based on underwater binocular camera and imaging sonar.

[0013] Furthermore, step 1 also includes the following steps:

[0014] Step 1.1: Construct an underwater binocular camera, including selecting underwater light-transmitting materials, designing the camera's watertight structure, and then completing the underwater binocular camera calibration to obtain the camera's intrinsic and extrinsic parameter matrix and baseline parameters;

[0015] Step 1.2: Construct an imaging sonar, set its maximum detection range, signal gain, etc., and integrate the underwater binocular camera and imaging sonar through structural design.

[0016] Furthermore, step 2 includes selecting the number of samples, modeling the image background, and updating the background model.

[0017] Furthermore, step 2 also includes the following steps:

[0018] Step 2.1: Construct an image pixel sample set. Store a sample set for each pixel of the input image and set the number of samples, for example, between 15 and 60.

[0019] Step 2.2, Image Background Modeling, characterized by using the first frame image for initialization, then comparing the new input pixel values ​​with the sample set and performing classification;

[0020] Step 2.3: Update the background model using a time sampling update strategy or a spatial domain update strategy; in addition, this invention introduces an update factor to adaptively update the background model.

[0021] Further, step 2.2 includes calculating the distance between the pixel value and each sample in the sample set. If the distance value is less than a threshold, the number of relevant sample points is increased. When the background changes suddenly, the method uses the changed first frame image to initialize the background model.

[0022] Furthermore, the method given in step 3 includes spatial credibility and multi-feature multi-channel credibility for underwater targets.

[0023] Further, in step 3, a set of target features is given as It includes features such as histogram of oriented gradients and color names; the correlation filter variable is the target template. To improve computational efficiency, this invention proposes a correlation filter learning model based on composite confidence, whose Fourier domain representation is as follows:

[0024] (1)

[0025] Among them, symbols Represents Fourier transform, It is a column vector with length . , and These represent the width and height of the relative filter training or search region, respectively. Represents Formed Symmetric matrix It is a regularization parameter greater than zero, symbol This represents the complex conjugate operation;

[0026] Then the first Each filter channel is related to the filter variables. Calculation

[0027] (2)

[0028] in, This represents dividing by each element of the matrix.

[0029] Furthermore, step 3 also includes the following steps:

[0030] Step 3.1: Calculate the multi-feature, multi-channel confidence coefficient. Assuming that the feature variations of different channels are uncorrelated, the multi-feature multi-channel confidence coefficient Calculation process

[0031] (3)

[0032] Step 3.2, calculate the spatial credibility coefficient, given... This refers to the spatial credibility coefficient matrix for underwater targets, where each element is either 0 or 1. Based on the learned correlation filter and the spatial credibility matrix, this invention provides the following formula for calculating the coefficients of the spatial credibility matrix.

[0033] (4)

[0034] in, Representing the Hadamard product, the spatial confidence coefficient is mainly used to adjust the coefficients of the learned correlation filter and to identify target components that can be tracked.

[0035] Step 3.3: Based on the aforementioned composite credibility, namely the spatial credibility and the multi-feature multi-channel credibility, this invention proposes the following correlation filter learning model based on composite credibility:

[0036] (5)

[0037] in, It is a complex Lagrange multiplier; The corresponding objective function.

[0038] Furthermore, in step 4, the correlation filter is calculated based on the augmented Lagrange algorithm framework.

[0039] (6)

[0040] (7)

[0041] in, This represents the inverse Fourier transform.

[0042] Furthermore, in step 5, based on the external parameter calibration, the data after tracking and positioning is filtered using a non-sensitive Kalman filtering algorithm to complete the joint tracking and positioning of underwater targets in the binocular camera and imaging sonar.

[0043] This invention addresses the joint tracking and positioning problem of underwater targets, addressing the urgent need for autonomous underwater operations. Based on correlation filtering theory and methods, it proposes an underwater target tracking and positioning system and method based on nonparametric classification and composite confidence. A correlation filter learning model based on composite confidence is constructed, including spatial confidence and multi-feature multi-channel confidence. The optimization problem of correlation filter learning based on composite confidence is solved, enabling underwater unmanned platforms to perform underwater target tracking and positioning tasks in dynamic flow fields, laying the foundation for precise control of underwater robots.

[0044] The following will further explain the concept, specific structure, and technical effects of the present invention in conjunction with the accompanying drawings, so as to fully understand the purpose, features, and effects of the present invention. Attached Figure Description

[0045] Figure 1 This is a preferred embodiment of the underwater target tracking and localization system and method based on non-parametric classification and composite confidence, wherein: (a) an underwater video target tracking and localization system based on imaging sonar and binocular camera; (b) an underwater moving target multi-mode detection method based on non-parametric pixel classification; (c) a correlation filter learning model based on composite confidence; and (d) a joint tracking and localization result based on binocular camera and imaging sonar.

[0046] Figure 2 This is a flowchart of a preferred embodiment of the present invention for multi-mode underwater moving target detection based on non-parametric pixel classification;

[0047] Figure 3 This is a preferred embodiment of the present invention: a correlation filter based on composite confidence for underwater targets. Detailed Implementation

[0048] The following description, with reference to the accompanying drawings, illustrates several preferred embodiments of the present invention to make its technical content clearer and easier to understand. The present invention can be embodied in many different forms, and the scope of protection of the present invention is not limited to the embodiments mentioned herein.

[0049] In the accompanying drawings, components with the same structure are indicated by the same numerical designation, and components with similar structures or functions are indicated by similar numerical designations. The dimensions and thicknesses of each component shown in the drawings are arbitrary, and the present invention does not limit the dimensions and thicknesses of each component. To make the illustrations clearer, the thickness of some components has been appropriately exaggerated in the drawings.

[0050] This invention provides an underwater target tracking and localization system and method based on non-parametric classification and composite confidence, characterized in that:

[0051] Step 1: Design an underwater video target tracking and positioning system based on imaging sonar and binocular camera;

[0052] Step 2: Establish a multi-mode detection method for underwater moving targets based on non-parametric pixel classification;

[0053] Step 3: Construct a correlation filter learning model based on composite credibility, which includes spatial credibility and multi-feature multi-channel credibility;

[0054] Step 4: Solve the correlation filter learning optimization problem based on composite confidence.

[0055] Step 5: Complete the joint tracking and positioning based on the binocular camera and imaging sonar.

[0056] An underwater target tracking and localization system and method based on non-parametric classification and composite confidence, as described in the embodiments of the present invention, is as follows: Figure 1 As shown, step 1 specifically includes:

[0057] Step 1.1: Construct an underwater binocular camera, including selecting underwater light-transmitting materials, designing the camera's watertight structure, and then calibrating the underwater binocular camera to obtain the camera's intrinsic and extrinsic parameter matrix and baseline parameters.

[0058] Step 1.2: Construct an imaging sonar, set its maximum detection range, signal gain, etc., and integrate the underwater binocular camera and imaging sonar through structural design.

[0059] An underwater target tracking and localization system and method based on non-parametric classification and composite confidence, as described in the embodiments of the present invention, is as follows: Figure 2 As shown, step 2 specifically includes: establishing a multi-mode underwater moving target detection method based on non-parametric pixel classification, which includes selecting the number of samples, image background modeling, and updating the background model. The specific details are as follows:

[0060] Step 2.1: Construct an image pixel sample set. Store a sample set for each pixel of the input image, and set the number of samples, for example, between 15 and 60.

[0061] Step 2.2, Image Background Modeling. The method is characterized by initialization using the first frame image, followed by comparison of the new input pixel values ​​with the sample set and classification. Specifically, the distance between the pixel value and each sample in the sample set is calculated; if the distance is less than a threshold, the number of relevant sample points is increased. When the background changes abruptly, the method initializes the background model using the changed first frame image.

[0062] Step 2.3: Update the background model. This can be achieved using a temporal sampling update strategy or a spatial domain update strategy. Additionally, this invention introduces an update factor to adaptively update the background model.

[0063] An underwater target tracking and localization system and method based on non-parametric classification and composite confidence, as described in the embodiments of the present invention, is as follows: Figure 3 As shown, step 3 specifically includes spatial credibility and multi-feature multi-channel credibility for underwater targets.

[0064] Given a set of target features, denoted as It includes features such as Histogram of Oriented Gradient (HOG) and Color Names; and correlation filtering variables (i.e., the target template). This invention calculates the target's position in an image sequence by measuring the maximum value of the relative response. To improve computational efficiency, this invention proposes a correlation filter learning model based on composite confidence, whose Fourier domain representation is as follows:

[0065] (1)

[0066] Among them, symbols Represents The Fourier transform of . Where, It is a column vector with length . ( and These represent the width and height of the relative filter training or search region, respectively. Represents Formed Symmetric matrix. It is a regularization parameter greater than zero. (Symbol) This represents the complex conjugate operation.

[0067] So, the first Each filter channel is related to the filter variables. Calculation

[0068] (2)

[0069] in, This represents dividing by each element of the matrix.

[0070] Step 3.1: Calculate the multi-feature, multi-channel confidence coefficient. This invention addresses the problem caused by changes in the scale of underwater targets by assuming that the feature changes in different channels are uncorrelated. Then, the multi-feature, multi-channel confidence coefficient of this invention...

[0071] Calculation process

[0072] (3)

[0073] Step 3.2, calculate the spatial credibility coefficient. Given This is a spatial credibility coefficient matrix for underwater targets, where each element is either 0 or 1. Based on a learned correlation filter and the spatial credibility matrix, this invention provides the following formula for calculating the coefficients of the spatial credibility matrix.

[0074] (4)

[0075] in, This represents the Hadamard product. Spatial confidence coefficients are primarily used to adjust the coefficients of the learned correlation filter and to identify target components that can be used for tracking.

[0076] Step 3.3: Based on the aforementioned composite credibility, namely spatial credibility and multi-feature multi-channel credibility, this invention proposes the following correlation filter learning model based on composite credibility:

[0077] (5)

[0078] in, It is a complex Lagrange multiplier; The corresponding objective function.

[0079] An underwater target tracking and localization system and method based on non-parametric classification and composite confidence, as described in the embodiments of the present invention, is as follows: Figure 1 As shown, step 4 specifically includes solving the correlation filter learning optimization problem based on composite confidence. Its key feature is that it calculates the correlation filter based on the Augmented Lagrangian algorithm framework.

[0080] (6)

[0081] (7)

[0082] in, Represents the inverse Fourier transform, such as Figure 3 As shown.

[0083] According to the embodiments of the present invention, an underwater target tracking and positioning system and method based on non-parametric classification and composite confidence is described. Step 5 specifically includes filtering the tracking and positioning data using a non-sensitive Kalman filtering algorithm to complete the joint tracking and positioning based on a binocular camera and imaging sonar.

[0084] The underwater video target tracking and positioning system based on imaging sonar and binocular cameras features an interference-free configuration, consisting of four standard working units and two locking mechanisms.

[0085] The preferred embodiments of the present invention have been described in detail above. It should be understood that those skilled in the art can make numerous modifications and variations based on the concept of the present invention without creative effort. Therefore, all technical solutions that can be obtained by those skilled in the art based on the concept of the present invention through logical analysis, reasoning, or limited experimentation on the basis of existing technology should be within the scope of protection defined by the claims.

Claims

1. A target tracking and localization method based on nonparametric classification and spatial credibility model, characterized in that, The method includes the following steps: Step 1: Design an underwater video target tracking and positioning system based on imaging sonar and underwater binocular camera; Step 2: Establish a multi-mode underwater moving target detection method based on non-parametric pixel classification; Step 3: Construct a correlation filter learning model based on composite credibility; Step 4: Solve the correlation filter learning optimization problem based on composite confidence. Step 5: Complete the joint tracking and positioning based on underwater binocular camera and imaging sonar; Step 1 also includes the following steps: Step 1.1: Construct an underwater binocular camera, including selecting underwater light-transmitting materials, designing the camera's watertight structure, and then completing the underwater binocular camera calibration to obtain the camera's intrinsic and extrinsic parameter matrix and baseline parameters; Step 1.2: Construct an imaging sonar, set its maximum detection range and signal gain, and integrate the underwater binocular camera and imaging sonar through structural design; The method given in step 3 includes spatial reliability and multi-feature multi-channel reliability for underwater targets; In step 3, a set of target features is given as follows: It includes directional gradient histogram features, ColorNames features, and correlation filter variables, which are the target templates. To improve computational efficiency, a correlation filter learning model based on composite confidence is proposed, whose Fourier domain representation is as follows: (1) Among them, symbols Represents Fourier transform, It is a column vector with length . , and These represent the width and height of the relative filter training or search region, respectively. Represents Formed Symmetric matrix It is a regularization parameter greater than zero, symbol This represents the complex conjugate operation; Then the first Each filter channel is related to the filter variables. Calculation , (2) in, This represents dividing by each element of the matrix; Step 3 also includes the following steps: Step 3.1: Calculate the multi-feature, multi-channel confidence coefficient. Assuming that the feature variations of different channels are uncorrelated, the multi-feature multi-channel confidence coefficient Calculation process (3) Step 3.2, calculate the spatial credibility coefficient matrix, given... The spatial confidence coefficient matrix for underwater targets consists of elements that are either 0 or 1. Based on the learned correlation filter and the spatial confidence coefficient matrix, the following formula for calculating the spatial confidence coefficient matrix is ​​given. (4) in, Representing the Hadamard product, the spatial confidence coefficient matrix is ​​mainly used to adjust the coefficients of the learned correlation filter and to identify target components that can be tracked. Step 3.3: Based on the aforementioned composite credibility, namely the spatial credibility coefficient matrix and the multi-feature multi-channel credibility coefficients, the following correlation filter learning model based on composite credibility is proposed: (5) in, It is a complex Lagrange multiplier; The corresponding objective function.

2. The target tracking and localization method based on non-parametric classification and spatial credibility model as described in claim 1, characterized in that, Step 2 includes selecting the number of samples, modeling the image background, and updating the background model.

3. The target tracking and localization method based on non-parametric classification and spatial reliability model as described in claim 2, characterized in that, Step 2 also includes the following steps: Step 2.1: Construct an image pixel sample set. Store a sample set for each pixel of the input image and set the number of samples. Step 2.2, Image Background Modeling, characterized by using the first frame image for initialization, then comparing the new input pixel values ​​with the sample set and performing classification; Step 2.3: Update the background model using a time sampling update strategy or a spatial domain update strategy; in addition, introduce an update factor to adaptively update the background model.

4. The target tracking and localization method based on non-parametric classification and spatial credibility model as described in claim 3, characterized in that, Step 2.2 includes calculating the distance between the pixel value and each sample in the sample set. If the distance value is less than a threshold, the number of relevant sample points is increased. When the background changes suddenly, the method uses the changed first frame image to initialize the background model.

5. The target tracking and localization method based on non-parametric classification and spatial credibility model as described in claim 4, characterized in that, In step 4, the correlation filter is calculated based on the augmented Lagrange algorithm framework. (6) (7) in, This represents the inverse Fourier transform.

6. The target tracking and localization method based on non-parametric classification and spatial credibility model as described in claim 5, characterized in that, In step 5, based on the external parameter calibration, the Kalman filter algorithm is used to filter the data after tracking and positioning, thereby completing the joint tracking and positioning of underwater targets in the binocular camera and imaging sonar.

Images