An Imaging Sonar Odometry Method Based on SIFT Features

Abstract

The present invention proposes a method for an imaging sonar odometer based on SIFT features. The method obtains an acoustic image of an underwater environment through an imaging sonar, then extracts SIFT features from the obtained acoustic image, then performs SIFT feature matching, then uses the least squares method for motion estimation, and finally integrates to obtain the mileage of the sonar. The imaging sonar odometer method provided by the present invention solves the positioning problem of an underwater platform and can achieve the positioning problem of the underwater platform without relying on devices such as GPS.

Description

Technical Field

[0001] The present invention belongs to the technical field of sonar image processing, and particularly relates to an imaging sonar odometer method based on SIFT features. Background Art

[0002] How to achieve GPS-independent navigation underwater has always been a research hotspot. Since sound waves are currently known media that can propagate long distances underwater, there have emerged positioning methods based on acoustic means such as USBL, SBL, LBL, and inertial navigation fusion DVL. Among them, methods such as USBL, SBL, and LBL are local positioning systems with low positioning frequencies, and although the method of inertial navigation fusion DVL is a global positioning method with a high positioning frequency, it is costly.

[0003] In recent years, more and more research has been carried out on using image sonar for navigation. In these studies, the imaging sonar often outputs images as a sonar image output unit for use by unmanned platforms, with a single function and unable to provide other information further. On underwater unmanned platforms, in order to obtain accurate navigation information, a set of speed measurement sensors (such as DVL) is usually equipped, which undoubtedly increases its cost and limits its application scenarios. The present invention will utilize the technology of imaging sonar to implement an odometer, enabling the existing image sonar to estimate the platform's movement trajectory in addition to outputting sonar images, reducing the number of sensors used on the unmanned platform and lowering the system complexity and cost. Summary of the Invention

[0004] The purpose of the present invention is to solve the problems in the prior art and propose an imaging sonar odometer method based on SIFT features. The method of the present invention can achieve the purpose of providing a positioning function for the carrier platform underwater.

[0005] The present invention is realized through the following technical solutions. The present invention proposes an imaging sonar odometer method based on SIFT features, and the method includes the following steps:

[0006] Step 1: Extract SIFT features from the obtained sonar image;

[0007] Step 2: Match the SIFT features V A of the current moment image obtained and the features V B of the previous moment;

[0008] Step 3: Use the least squares method to estimate the translation amount (t x , t y ) k and the rotation amount

[0009] Step 4: Integrate the obtained translation amount (t x , t y ) k and the rotation amount after coordinate transformation to obtain the sonar odometry.

[0010] Furthermore, matching the SIFT features V A of the current moment image obtained and the features V B of the previous moment specifically includes the following steps:

[0011] SII1: First, take out a SIFT feature V A from the current image, and its position information is Using the constraint condition where dr is the maximum possible displacement amount and dθ is the maximum possible rotation amount, obtain the position range of the corresponding feature in the previous moment image B, and filter out these points;

[0012] SII2: From the SIFT feature points filtered out in step SII1, respectively take out their SIFT feature vectors and perform dot product operations with V A ;

[0013] SII3: Screen out the maximum value of the dot product result and compare it with the set threshold. If it is less, it means there is no point in image B that matches V A , otherwise, there is a paired point, which is denoted as V B ;

[0014] SII4: Repeat steps SII1 to SII3 until all SIFT features in image A are matched.

[0015] Furthermore, use the least squares method to estimate the translation amount (t x , t y ) k and the rotation amount specifically including the following steps:

[0016] SIII1: Estimate the position in the current moment image A through the formula using the SIFT feature position information (x Bi , y Bi ) of the previous moment image B

[0017] SIII2: Construct the error function

[0018] SIII3: Solve the error function The first-order and second-order partial derivatives are obtained to get the gradient vector g and the Hessian matrix H;

[0019] SIII4: Calculate the iteration step size Δp using the formula;

[0020] SIII5: Calculate the new iteration result from the iteration step size Δp through the formula

[0021] SIII6: Repeat the process from SIII1 to SIII5 a total of N times to obtain the final estimated translation amount (t x , t y ) k and the rotation amount

[0022] Furthermore, the calculation formula in SIII1 is:

[0023]

[0024] Furthermore, the error function is:

[0025]

[0026] Furthermore, the calculation formula in SIII4 is:

[0027] Δp = -H -1 g.

[0028] Furthermore, the calculation formula in SIII5 is:

[0029]

[0030] The present invention proposes an imaging sonar odometer system based on SIFT features, and the system includes:

[0031] Extraction module: Extract SIFT features from the obtained sonar images;

[0032] Matching module: Match the SIFT features V A of the current moment image obtained and the features V B of the previous moment;

[0033] Estimation module: Use the least squares method to estimate the translation amount (t x , t y ) k and the rotation amount

[0034] Calculation module: The obtained translation amount (t x , t y )k and rotation amount Perform coordinate system transformation and then integrate to obtain the sonar odometry.

[0035] The present invention provides an electronic device, including a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, the steps of the imaging sonar odometer method based on SIFT features are implemented.

[0036] The present invention provides a computer-readable storage medium for storing computer instructions. When the computer instructions are executed by a processor, the steps of the imaging sonar odometer method based on SIFT features are implemented.

[0037] Advantages of the present invention:

[0038] The present invention provides an imaging sonar odometer method based on SIFT features. The method uses an imaging sonar to implement the odometer technology, enabling the existing image sonar to estimate the platform movement trajectory in addition to outputting sonar images, reducing the usage of sensors on the unmanned platform, and lowering the system complexity and cost. Description of the Drawings

[0039] To more clearly illustrate the technical solutions in the embodiments of the present invention or in the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are only the embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained according to the provided drawings without creative efforts.

[0040] Figure 1 It is a flowchart for implementing the imaging sonar odometer method based on SIFT features provided by the present invention. Detailed Embodiments

[0041] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present invention.

[0042] The embodiments of the present invention disclose an imaging sonar odometer method based on SIFT features, including the following steps:

[0043] Step 1: Extract SIFT features from the obtained sonar image.

[0044] Step 2: The SIFT features V of the image at the current moment obtained Aand the feature V at the previous moment B Perform matching.

[0045] Step 3: Use the least squares method to estimate the translation amount (t x , t y ) k and the rotation amount

[0046] Step 4: Convert the obtained translation amount (t x , t y ) k and the rotation amount Perform coordinate transformation and then integrate to obtain the sonar odometry.

[0047] See Appendix Figure 1 , which is a graphical description of the method described in the present invention. The process of obtaining the SIFT feature V of the current moment image A and the feature V at the previous moment B and performing matching, the specific steps include:

[0048] SII1: First, take out a SIFT feature V of the current image A , and its position information is Using the constraint condition where dr is the maximum possible displacement amount and dθ is the maximum possible rotation amount, obtain the position range of the corresponding feature in the previous moment image B, and filter out these points.

[0049] SII2: From the SIFT feature points filtered out in step SII1, take out their respective SIFT feature vectors and perform dot product operations with V A .

[0050] SII3: Screen to obtain the maximum value of the dot product result and compare it with the set threshold. If it is less, it means that there is no point in image B that matches V A . Otherwise, there is a paired point, which is denoted as V B .

[0051] SII4: Repeat steps SII1 to SII3 until all SIFT features in image A are matched.

[0052] See Appendix Figure 1 , where the least squares method is used to estimate the translation amount (t x , t y ) k and the rotation amount The specific steps include:

[0053] SIII1: Estimate the position in the current image A of the SIFT feature position information (x Bi , y Bi ) of the previous image B obtained through the formula

[0054] SIII2: Construct the error function

[0055] SIII3: Solve the first-order and second-order partial derivatives of the error function to obtain the gradient vector g and the Hessian matrix H.

[0056] SIII4: Calculate the iteration step size Δp using the formula Δp = -H -1 g.

[0057] SIII5: Calculate the new iteration result from the iteration step size Δp through the formula

[0058] SIII6: Repeat the processes SIII1 to SIII5 a total of N times to obtain the final estimated translation amount (t x , t y ) k and the rotation amount

[0059] The present invention discloses a method for an imaging sonar odometer based on SIFT features. By processing sonar images, the motion trajectory of the sonar itself is calculated to achieve the positioning function of an underwater platform, which is of great significance for underwater application scenarios without GPS.

[0060] The method proposed by the present invention is further described below in conjunction with specific embodiments. The sonar used is the M750d imaging sonar of BP Company, with an output image size of 512x450 and a range of 20 meters. When the sonar image is obtained, the first step is to extract SIFT features, and the second step is to match the SIFT features V A of the current image and V B of the previous moment, with the screening parameters being dr = 0.2 m and dθ = 2°, and the screening threshold for the dot product result of the SIFT feature vectors being 0.95. The third step is to estimate the translation amount (t x , t y ) k of the current image frame relative to the previous image frame and the rotation amount using the least squares method, where the number of iterations is set to 1000 times. The fourth step is to use the obtained translation amount (t x , t y ) kand rotation amount Coordinate system transformation and then integration to obtain the sonar odometry.

[0061] The present invention provides an imaging sonar odometer system based on SIFT features, and the system includes:

[0062] Extraction module: Extract SIFT features from the obtained sonar images;

[0063] Matching module: Match the SIFT features V of the current moment image A and the features V of the previous moment B for matching;

[0064] Estimation module: Use the least squares method to estimate the translation amount (t x , t y ) k and rotation amount

[0065] Calculation module: Coordinate system transformation and then integration to obtain the sonar odometry with the obtained translation amount (t x , t y ) k and rotation amount Coordinate system transformation and then integration to obtain the sonar odometry.

[0066] The present invention provides an electronic device, including a memory and a processor, where the memory stores a computer program, and when the processor executes the computer program, the steps of the method for an imaging sonar odometer based on SIFT features are implemented.

[0067] The present invention provides a computer-readable storage medium for storing computer instructions, and when the computer instructions are executed by a processor, the steps of the method for an imaging sonar odometer based on SIFT features are implemented.

[0068] The memory in the embodiments of the present application may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read only memory (ROM), a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example but not limitation, many forms of RAM are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchlink DRAM (SLDRAM), and direct rambus RAM (DRRAM). It should be noted that the memory of the method described in the present invention is intended to include but not limited to these and any other suitable types of memory.

[0069] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on a computer, the processes or functions described in the embodiments of the present application are generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center in a wired manner (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wirelessly (such as infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that includes one or more integrated available media. The available medium can be a magnetic medium (such as a floppy disk, hard disk, magnetic tape), an optical medium (such as a high-density digital video disc (DVD)), or a semiconductor medium (such as a solid state disc (SSD)), etc.

[0070] In the implementation process, the steps of the above method can be completed by the integrated logic circuit of the hardware in the processor or the instructions in the form of software. The steps of the method disclosed in combination with the embodiments of the present application can be directly embodied as being executed and completed by the hardware processor, or executed and completed by a combination of the hardware and software modules in the processor. The software module can be located in a mature storage medium in the art such as a random access memory, flash memory, read-only memory, programmable read-only memory, or electrically erasable programmable memory, register, etc. This storage medium is located in the memory, and the processor reads the information in the memory and combines its hardware to complete the steps of the above method. To avoid repetition, it will not be described in detail here.

[0071] It should be noted that the processor in the embodiments of this application can be an integrated circuit chip with signal processing capabilities. During implementation, the steps of the above method embodiments can be completed through the integrated logic circuit in the hardware of the processor or instructions in software form. The above-mentioned processor can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. It can implement or execute the various methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in combination with the embodiments of this application can be directly embodied as being executed and completed by a hardware decoding processor, or executed and completed by a combination of the hardware and software modules in the decoding processor. The software module can be located in a mature storage medium in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. This storage medium is located in the memory, and the processor reads the information in the memory and combines its hardware to complete the steps of the above method.

[0072] The above has introduced in detail a method for an imaging sonar odometer based on SIFT features proposed by the present invention. Specific examples are used in this article to elaborate on the principle and implementation manner of the present invention. The description of the above embodiments is only used to help understand the method and its core idea of the present invention; at the same time, for those of ordinary skill in the art, according to the idea of the present invention, there will be changes in the specific implementation manner and application scope. In summary, the content of this specification should not be construed as a limitation to the present invention.

Claims

1. An imaging sonar odometer method based on SIFT features, characterized in that, the method comprises the following steps: Step 1: Extract SIFT features from the obtained sonar image; Step 2: Match the SIFT features V of the current moment image obtained A and the features V of the previous moment B to each other; Step 3: Use the least squares method to estimate the translation amount (t x , t y ) k and the rotation amount Step 4: Integrate the obtained translation amounts (t x , t y ) k and rotation amounts after coordinate system transformation to obtain the sonar odometry; Estimate the translation amount (t x , t y ) k and the rotation amount from the image frame at the current moment relative to the image frame at the previous moment by using the least squares method. The specific steps are as follows: SIII1: Estimate the position in the current image A of the SIFT feature position information (x Bi , y Bi ) of the previous image B obtained through the formula SIII2: Construct the error function SIII3: Solve for the first-order and second-order partial derivatives of the error function to obtain the gradient vector and the Hessian matrix H; g ​ SIII4: Calculate the iteration step size using the formula Δp ; SIII5: Obtained from the iteration step Δp A new iteration result is calculated through the formula SIII6: Repeat the processes SIII1 to SIII5 a total of N times to obtain the final estimated translation amount (t x , t y ). k and the rotation amount The calculation formula in SIII1 is as follows: The error function is: The calculation formula in SIII4 is: Δp = -H -1 g; The calculation formula in SIII5 is:

2. The method according to claim 1, characterized in that, The SIFT features V of the currently obtained image A and the features V of the previous moment B are matched. The specific steps include: SII1: First, extract a SIFT feature V of the current image A , and its position information is Using the constraint condition where dr is the maximum possible displacement and dθ is the maximum possible rotation amount, obtain the position range of the corresponding feature in the previous image B, and filter out these points; SII2: From the SIFT feature points selected in step SII1, respectively extract their respective SIFT feature vectors and perform dot product operations with V A ; SII3: Select the maximum value of the dot product result and compare it with the set threshold. If it is less than the threshold, it means that there is no point in Image B that matches V A ; otherwise, there is a paired point, which is denoted as V B ; SII4: Repeat steps SII1 to SII3 until all SIFT features in image A are matched.

3. An imaging sonar odometer system based on SIFT features, characterized in that, the system comprises: Extraction module: Extract SIFT features from the obtained sonar image; Matching module: Match the SIFT features V of the current moment image obtained A and the features V of the previous moment B to perform matching; Estimation module: Estimate the translation amount (t x , t y ) k and rotation amount Calculation module: The obtained translation amounts (t x , t y ) k and the rotation amount are subjected to coordinate system conversion and then integrated to obtain the sonar odometry; Estimate the translation amount (t x , t y ) k and the rotation amount The specific steps are as follows: SIII1: Estimate the position in the current image A of the SIFT feature position information (x Bi , y Bi ) of the previous image B obtained through the formula SIII2: Construct the error function SIII3: Solve the error function to obtain the first-order and second-order partial derivatives, and get the gradient vector g and the Hessian matrix H; SIII4: Calculate the iteration step Δp using the formula; SIII5: Obtained from the iteration step Δp Calculate a new iteration result through the formula SIII6: Repeat the processes SIII1 to SIII5 a total of N times to obtain the final estimated translation amount (t x , t y ). k and the rotation amount The calculation formula in SIII1 is as follows: The error function is: The calculation formula in SIII4 is: Δp = -H -1 g; The calculation formula in SIII5 is:

4. An electronic device, comprising a memory and a processor, the memory stores a computer program, characterized in that, when the processor executes the computer program, the steps of the method according to any one of claims 1-2 are implemented.

5. A computer-readable storage medium for storing computer instructions, characterized in that, when the computer instructions are executed by a processor, the steps of the method according to any one of claims 1-2 are implemented.

Images