Image matching correction method and device and positioning method for submarine exploration operation

Abstract

The invention provides an image matching correction method and device and a positioning method for submarine exploration operation of a deep-sea robot, and solves the problem that the quality of an image obtained by an underwater robot in a marine scene is not high and the positioning precision in deep sea is influenced. The method comprises the following steps: carrying out preliminary matching on image pairs to obtain an initial matching point pair set; performing grid division on the initial matching point pair set to obtain an image grid; preliminary correction is carried out on wrong matching in initial matching, outliers are eliminated, and a corresponding initial matching point pair set is updated; combining the image grids after the preliminary correction, and selecting the image grids uniformly distributed in the scene based on the maximum variance between the grids; constructing a transformation matrix based on the updated initial matching point pair set, and determining the transformation matrix by adopting an overall loss function combining global constraint and local constraint; and correcting the merged image grids based on the transformation matrix to obtain a final matching point pair set so as to complete image matching correction.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to an image matching correction method and device, and a positioning method for submerged sea exploration operations. Background Art

[0002] The exploration and development of marine oil and gas has great development prospects. Due to the complex and changeable environment of deep-sea diving exploration operations and many uncertain factors, marine exploration and development faces higher risks and difficulties. Deep-sea robots are extreme operation robots used in deep-water environments. Due to the complex and changeable deep-sea environment, underwater manual operations bring many troubles such as personal safety. Deep-sea robots have gradually become a powerful tool to replace manual operations with their strong adaptability to the deep-sea environment, good flexibility and economy.

[0003] The deep-sea robots that are frequently used internationally can be roughly divided into two types: one is ROV (Remote Operated Vehicle), which is mainly used for local exploration of the marine environment. ROV can be further divided into three types: mobile, floating and towed; the other is AUV (Autonomous Underwater Vehicle), which is more flexible and can carry out large-scale ocean cruises. There are nearly a thousand deep-sea robots in the field of marine oil and gas exploration and development and operations, such as the laying, installation and maintenance of seabed resource equipment.

[0004] Deep-sea robots need to perform a series of positioning, tracking and navigation when operating underwater, among which positioning is the prerequisite for all underwater operations. The accuracy of image matching affects the positioning performance of deep-sea robots in underwater operations. However, underwater light scattering, low visibility, blurred focus, interference from floating objects or sediments in the water caused by the marine environment will affect the image quality obtained by underwater robots in marine scenes, thereby interfering with the positioning accuracy in the deep sea, and further having an adverse effect on underwater operations. Summary of the invention

[0005] In view of this, the embodiments of the present invention provide an image matching correction method, device and positioning method for underwater exploration operations, which solve the problem that the image quality obtained by the underwater robot in the ocean scene is not high, thereby affecting the positioning accuracy in the deep sea.

[0006] In a first aspect, an image matching correction method provided by an embodiment of the present invention includes:

[0007] Perform preliminary matching on the image pairs to obtain an initial set of matching point pairs;

[0008] Gridding the initial matching point pair set to obtain an image grid;

[0009] Perform preliminary correction on the erroneous matches in the initial matching, remove the outliers and update the corresponding set of initial matching point pairs;

[0010] The image grids after preliminary correction are merged, and the image grids uniformly distributed in the scene are selected based on the maximum inter-grid variance;

[0011] A transformation matrix is ​​constructed based on the updated initial matching point pair set, and the transformation matrix is ​​determined by using an overall loss function combining global constraints and local constraints;

[0012] The merged image grid is corrected based on the transformation matrix to obtain a final set of matching point pairs to complete image matching correction.

[0013] In one embodiment, said gridding the initial matching point pair set to obtain an image grid includes: gridding the initial matching point pair set using a loose quadtree structure.

[0014] In one implementation, it also includes: determining a domain set of the initial matching point pair set by constructing a hash function.

[0015] In one embodiment, the preliminarily correcting the mismatch in the initial matching includes: preliminarily correcting the mismatch in the initial matching based on multiple local constraint loss functions and a domain set of the initial matching point pair set.

[0016] In one embodiment, the multiple local constraint loss function includes: a local neighborhood motion constraint and a local neighborhood distance constraint.

[0017] In one embodiment, merging the image grids after preliminary correction includes: merging the image grids containing less than a preset number of feature points in the image grids after preliminary correction with adjacent grids, sorting all image grids based on confidence, and retaining image grids with a confidence threshold value.

[0018] In a second aspect, an embodiment of the present invention provides an image matching correction device, comprising:

[0019] An initial matching module, used to perform preliminary matching on the image pairs to obtain an initial set of matching point pairs;

[0020] A grid division module, used for performing grid division on the initial matching point pair set to obtain an image grid;

[0021] A first correction module, used to perform preliminary correction on the erroneous matches in the initial matching, remove the outliers and update the corresponding initial matching point pair set;

[0022] An image grid processing module is used to merge the image grids after preliminary correction and select image grids that are evenly distributed in the scene based on the maximum inter-grid variance;

[0023] The transformation matrix module is used to construct the transformation matrix based on the updated initial matching point pair set, and determine the transformation matrix using the overall loss function combining global constraints and local constraints;

[0024] The second correction module is used to correct the merged image grid based on the transformation matrix to obtain a final set of matching point pairs to complete image matching correction.

[0025] In a third aspect, an embodiment of the present invention provides a positioning method for a deep-sea robot for submerged exploration operations, which uses any of the above-mentioned image matching correction methods to perform image matching correction, and performs underwater environment positioning based on the image after image matching correction.

[0026] In a fourth aspect, an embodiment of the present invention provides an electronic device, comprising a memory and a processor, wherein the memory is used to store one or more computer instructions, wherein the one or more computer instructions, when executed by the processor, implement an image matching correction method as described in any one of the above.

[0027] In a fifth aspect, an embodiment of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, it is used to implement the image matching correction method as described in any one of the above.

[0028] An image matching correction method, device and positioning method for submerged exploration operations provided in an embodiment of the present invention can be applied to deep-sea robots operating underwater, and an image matching correction technology is provided. This method can efficiently and robustly correct image matching association information by correcting image matching, thereby further improving the positioning accuracy of the robot and enhancing the applicability of the deep-sea robot in the field of marine oil and gas exploration and development and operations. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 FIG. 4 is a flow chart of an image matching correction method provided by an embodiment of the present invention.

[0030] Figure 2 FIG. 4 is a flow chart of another image matching correction method provided by an embodiment of the present invention.

[0031] Figure 3Shown is a schematic structural diagram of an image matching correction device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0032] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0033] The present invention proposes a method for improving positioning accuracy for the application of deep-sea robots in the field of marine oil and gas exploration, development and operation. The completion of the observation and collection of underwater geographic coordinates is a prerequisite for the deep-sea robot to be applied to marine oil and gas exploration operations. In addition, a series of operations performed by deep-sea robots underwater require tracking and positioning of the operation target, so the positioning ability of deep-sea robots in marine environments is crucial. The positioning of deep-sea robots depends to a certain extent on the continuous image sequence obtained by itself, and the extraction of useful information and feature matching are performed on it to help realize the positioning function of the underwater robot. Therefore, the matching accuracy of the image sequence will have a profound impact on the positioning accuracy of the underwater robot. The method provided by the present invention can effectively improve the positioning accuracy of the deep-sea robot in the underwater environment by correcting the image matching; and further can improve the work efficiency and performance of the deep-sea robot in underwater environments such as the field of marine oil and gas exploration. The specific implementation method is described in the following embodiments.

[0034] Embodiment 1:

[0035] This embodiment provides an image matching correction method, such as Figure 1 and Figure 2 The image matching correction method shown includes:

[0036] Step 01: Perform preliminary matching on the image pairs to obtain an initial set of matching point pairs.

[0037] Step 02: Grid the initial matching point pair set to obtain an image grid.

[0038] Furthermore, the gridding of the initial matching point pair set to obtain the image grid includes: gridding the initial matching point pair set using a loose quadtree structure. Use a loose quadtree structure to match the initial point pair set Divide the grid to obtain an image grid.

[0039] Step 03: Perform preliminary correction on the erroneous matches in the initial matching, remove the outliers and update the corresponding initial matching point pair set.

[0040] In addition, the image matching correction method also includes: determining a domain set of the initial matching point pair set by constructing a hash function.

[0041] Specifically, construct a hash function to determine the initial matching set Neighborhood information Among them, N i is the neighborhood set of the i-th initial matching point in image I, For images The neighborhood set of the i-th initial matching point in . Since the preliminary matching has many outliers, the divided neighborhood may include both dense and sparse cases.

[0042] The preliminary correction of the mismatch in the initial matching includes: preliminary correction of the mismatch in the initial matching based on multiple local constraint loss functions and a domain set of the initial matching point pair set.

[0043] Multiple local constraints are used to preliminarily correct the mismatches in the initial matching and update the matching set S in , and its overall loss function with multiple local constraints is as follows:

[0044]

[0045] in, is the number of reliable correspondences in image matching, and θ is a parameter that balances multiple local constraints.

[0046] The multiple local constraints include: local neighborhood motion constraints and local neighborhood distance constraints.

[0047] 1. Local neighborhood motion constraints:

[0048] If the matching results within the neighborhood are accurate, then in most cases, the difference between the motion vectors of the feature point pairs will be small. Therefore, the difference in the motion vectors within the neighborhood is used to verify the preservation of the local structure, and then filter out the outlier matching correspondences. The loss function of the local neighborhood motion constraint is as follows:

[0049]

[0050] in, Represents the neighborhood set N of the i-th initial matching point in image I i The number of feature points in v i and v jThey represent the motion vectors of the initial matching point pair and its neighborhood matching point pair in the image pair, and the difference between the two is expressed as Dist(v i , v j ) is used to measure the motion of the local neighborhood. The larger the value calculated, the more consistent the motion of the local neighborhood. i , v j ) can be expressed as:

[0051]

[0052] 2. Local neighborhood distance constraint

[0053] The camera's motion process does not affect the distribution of neighboring point pairs for reliable feature matching, that is, the neighborhood corresponding to the correct match can better maintain a smaller distance relationship than the neighborhood corresponding to the incorrect match. Therefore, a local neighborhood distance constraint is set to map the feature points in the first image to the matched second image, and compare the distance with all feature points in the neighborhood in the second image to help filter out obviously wrong feature matches. The loss function of the local neighborhood distance constraint is as follows:

[0054]

[0055] in, is the i-th initial matching point p in image I i The matching image The matching in To match the image The i-th matching point in .

[0056] Step 04: Merge the preliminarily rectified image grids and select image grids that are evenly distributed in the scene based on the maximum inter-grid variance.

[0057] Furthermore, the merging of the preliminarily corrected image grids includes: merging image grids containing less than a preset number of feature points in the preliminarily corrected image grids with adjacent grids, sorting all image grids based on confidence, and retaining image grids with a confidence threshold value.

[0058] In order to reduce the impact of the ocean environment with obvious motion scenes on the positioning accuracy, starting from the scene assumption of rigid structure, the initially corrected image grids are merged with high confidence, and then the maximum inter-grid variance is performed to select image grids that are evenly distributed in the scene.

[0059] Step 05: Construct a transformation matrix based on the updated initial matching point pair set, and determine the transformation matrix using an overall loss function that combines global constraints with local constraints.

[0060] Only focusing on the local information in the image for image matching correction, while ignoring the global features hidden in the visual task, will mislead the final correction result; therefore, in step 5, a combination of global constraints and local constraints is used to perform the final matching correction on the image after merging the grids to obtain the final matching correspondence set S in * :

[0061]

[0062] Among them, θ and θ′ are parameters that balance local constraints and global constraints, Cost dis Cost mot They are the local neighborhood distance constraint and the local neighborhood motion constraint in the multiple local constraints in step 4, respectively. gl is the loss function of the global constraint:

[0063]

[0064] Where T(p) is the image I to its matching corresponding image The global transformation mapping function, T(p) corresponds to the final matching set S in * The solution is a complementary process.

[0065] Step 06: Correct the merged image grid based on the transformation matrix to obtain a final set of matching point pairs to complete image matching correction.

[0066] The image matching correction method provided in this embodiment is based on a deep-sea robot that can be applied to underwater operations, and provides an image matching correction technology. By correcting image matching, this method can efficiently and robustly correct image matching association information, thereby further improving the positioning accuracy of the robot and enhancing the applicability of the deep-sea robot in marine oil and gas exploration, development and operations.

[0067] Embodiment 2:

[0068] This embodiment provides an image matching correction device 100, such as Figure 3 As shown, the image matching correction device includes: an initial matching module 10, a grid division module 20, a first correction module 30, an image grid processing module 40, a transformation matrix module 50 and a second correction module 60.

[0069] The initial matching module 10 is used to perform preliminary matching on the image pairs to obtain an initial matching point pair set;

[0070] The grid division module 20 is used to grid the initial matching point pair set to obtain an image grid;

[0071] The first correction module 30 is used to perform preliminary correction on the erroneous matches in the initial matching, remove the outliers and update the corresponding initial matching point pair set;

[0072] The image grid processing module 40 is used to merge the image grids after preliminary correction and select image grids uniformly distributed in the scene based on the maximum inter-grid variance;

[0073] The transformation matrix module 50 is used to construct a transformation matrix based on the updated initial matching point pair set, and determine the transformation matrix using an overall loss function combining global constraints and local constraints;

[0074] The second correction module 60 is used to correct the merged image grid based on the transformation matrix to obtain a final set of matching point pairs to complete image matching correction.

[0075] The initial matching module 10 performs preliminary matching on the image pairs to obtain an initial matching point pair set; then, the grid division module 20 performs grid division on the initial matching point pair set to obtain an image grid; the first correction module 30 performs preliminary correction on the erroneous matches in the initial matching, removes outliers and updates the corresponding initial matching point pair set; the image grid processing module 40 merges the image grids preliminarily corrected by the first correction module 30, and selects image grids uniformly distributed in the scene based on the maximum inter-grid variance; the transformation matrix module 50 constructs a transformation matrix based on the initial matching point pair set updated by the first correction module 30, and determines the transformation matrix using an overall loss function that combines global constraints with local constraints; finally, the second correction module 60 corrects the merged image grid based on the transformation matrix to obtain a final matching point pair set to complete the image matching correction.

[0076] Furthermore, the grid division module 20 is further used to grid the initial matching point pair set using a loose quadtree structure. The grid division of the initial matching point pair set to obtain the image grid includes: grid division of the initial matching point pair set using a loose quadtree structure. Optionally, for the image pair that has been preliminarily matched in step 01, Use a loose quadtree structure to match the initial point pair set Divide the grid to obtain an image grid.

[0077] Furthermore, the first correction module 30 is also used to determine the domain set of the initial matching point pair set by constructing a hash function. Preliminary correction is performed on the mismatch in the initial matching based on the multiple local constraint loss function and the domain set of the initial matching point pair set. The multiple local constraint loss function includes a local neighborhood motion constraint and a local neighborhood distance constraint. Constructing a hash function to determine the preliminary matching set Neighborhood information Among them, N i is the neighborhood set of the i-th initial matching point in image I, For images The neighborhood set of the i-th initial matching point in . Since the preliminary matching has many outliers, the divided neighborhood may include both dense and sparse cases.

[0078] Multiple local constraints are used to preliminarily correct the mismatches in the initial matching and update the matching set S in , and its overall loss function with multiple local constraints is as follows:

[0079]

[0080] in, is the number of reliable correspondences in image matching, and θ is a parameter that balances multiple local constraints.

[0081] The multiple local constraints include: local neighborhood motion constraints and local neighborhood distance constraints.

[0082] 1. Local neighborhood motion constraints:

[0083] If the matching results within the neighborhood are accurate, then in most cases, the difference between the motion vectors of the feature point pairs will be small. Therefore, the difference in the motion vectors within the neighborhood is used to verify the preservation of the local structure, and then filter out the outlier matching correspondences. The loss function of the local neighborhood motion constraint is as follows:

[0084]

[0085] in, Represents the neighborhood set N of the i-th initial matching point in image I i The number of feature points in v i and v j They represent the motion vectors of the initial matching point pair and its neighborhood matching point pair in the image pair, and the difference between the two is expressed as Dist(v i , v j ) is used to measure the motion of the local neighborhood. The larger the value calculated, the more consistent the motion of the local neighborhood. i , vj ) can be expressed as:

[0086]

[0087] 2. Local neighborhood distance constraint

[0088] The camera's motion process does not affect the distribution of neighboring point pairs for reliable feature matching, that is, the neighborhood corresponding to the correct match can better maintain a smaller distance relationship than the neighborhood corresponding to the incorrect match. Therefore, a local neighborhood distance constraint is set to map the feature points in the first image to the matched second image, and compare the distance with all feature points in the neighborhood in the second image to help filter out obviously wrong feature matches. The loss function of the local neighborhood distance constraint is as follows:

[0089]

[0090] in, is the i-th initial matching point p in image I i The matching image The matching in To match the image The i-th matching point in .

[0091] Furthermore, the image grid processing module 40 is also used to merge the image grids containing less than a preset number of feature points in the image grids after preliminary correction with adjacent grids, sort all image grids based on confidence, and retain image grids greater than the confidence threshold. The merging of the image grids after preliminary correction includes: merging the image grids containing less than a preset number of feature points in the image grids after preliminary correction with adjacent grids, sorting all image grids based on confidence, and retaining image grids greater than the confidence threshold. In order to reduce the impact of the ocean environment with obvious motion scenes on the positioning accuracy, starting from the scene assumption of rigid structure, the image grids after preliminary correction are merged with high confidence, and then the maximum inter-grid variance is performed to select image grids evenly distributed in the scene.

[0092] Furthermore, the transformation matrix module 50 only focuses on the local information in the image to perform image matching correction, while ignoring the global features hidden in the visual task, which will mislead the final correction result; therefore, in step 5, a combination of global constraints and local constraints is used to perform the final matching correction on the image after merging the grids to obtain the final matching correspondence set S in * :

[0093]

[0094] Among them, θ and θ′ are parameters that balance local constraints and global constraints, Cost dis Cost mot They are the local neighborhood distance constraint and the local neighborhood motion constraint in the multiple local constraints in step 4, respectively. gl is the loss function of the global constraint:

[0095]

[0096] Where T(p) is the image I to its matching corresponding image The global transformation mapping function, T(p) corresponds to the final matching set S in * The solution is a complementary process.

[0097] The image matching correction device provided in this embodiment is based on a deep-sea robot that can be used for underwater operations, and provides an image matching correction technology. By correcting image matching, the device can efficiently and robustly correct image matching association information, thereby further improving the positioning accuracy of the robot and enhancing the applicability of the deep-sea robot in marine oil and gas exploration, development and operations.

[0098] Embodiment three:

[0099] This embodiment provides a positioning method for a deep-sea robot submersible exploration operation, which performs image matching correction based on the image matching correction method described in the above embodiment, and performs underwater environment positioning based on the image after image matching correction. The positioning method for a deep-sea robot submersible exploration operation includes:

[0100] Step 01: Perform preliminary matching on the image pairs to obtain an initial set of matching point pairs.

[0101] Step 02: Grid the initial matching point pair set to obtain an image grid.

[0102] Furthermore, the gridding of the initial matching point pair set to obtain the image grid includes: gridding the initial matching point pair set using a loose quadtree structure. Use a loose quadtree structure to match the initial point pair set Divide the grid to obtain an image grid.

[0103] Step 03: Perform preliminary correction on the erroneous matches in the initial matching, remove the outliers and update the corresponding initial matching point pair set.

[0104] In addition, the image matching correction method also includes: determining a domain set of the initial matching point pair set by constructing a hash function.

[0105] Specifically, construct a hash function to determine the initial matching set Neighborhood information Among them, N i is the neighborhood set of the i-th initial matching point in image I, For images The neighborhood set of the i-th initial matching point in . Since the preliminary matching has many outliers, the divided neighborhood may include both dense and sparse cases.

[0106] The preliminary correction of the mismatch in the initial matching includes: preliminary correction of the mismatch in the initial matching based on multiple local constraint loss functions and a domain set of the initial matching point pair set.

[0107] Multiple local constraints are used to preliminarily correct the mismatches in the initial matching and update the matching set S in , and its overall loss function with multiple local constraints is as follows:

[0108]

[0109] in, is the number of reliable correspondences in image matching, and θ is a parameter that balances multiple local constraints.

[0110] The multiple local constraints include: local neighborhood motion constraints and local neighborhood distance constraints.

[0111] 1. Local neighborhood motion constraints:

[0112] If the matching results within the neighborhood are accurate, then in most cases, the difference between the motion vectors of the feature point pairs will be small. Therefore, the difference in the motion vectors within the neighborhood is used to verify the preservation of the local structure, and then filter out the outlier matching correspondences. The loss function of the local neighborhood motion constraint is as follows:

[0113]

[0114] in, Represents the neighborhood set N of the i-th initial matching point in image I i The number of feature points in v i and v j They represent the motion vectors of the initial matching point pair and its neighborhood matching point pair in the image pair, and the difference between the two is expressed as Dist(v i , v j) is used to measure the motion of the local neighborhood. The larger the value calculated, the more consistent the motion of the local neighborhood. i , v j ) can be expressed as:

[0115]

[0116] 2. Local neighborhood distance constraint

[0117] The camera's motion process does not affect the distribution of neighboring point pairs for reliable feature matching, that is, the neighborhood corresponding to the correct match can better maintain a smaller distance relationship than the neighborhood corresponding to the incorrect match. Therefore, a local neighborhood distance constraint is set to map the feature points in the first image to the matched second image, and compare the distance with all feature points in the neighborhood in the second image to help filter out obviously wrong feature matches. The loss function of the local neighborhood distance constraint is as follows:

[0118]

[0119] in, is the i-th initial matching point p in image I i The matching image The matching in To match the image The i-th matching point in .

[0120] Step 04: Merge the preliminarily rectified image grids and select image grids that are evenly distributed in the scene based on the maximum inter-grid variance.

[0121] Furthermore, the merging of the preliminarily corrected image grids includes: merging image grids containing less than a preset number of feature points in the preliminarily corrected image grids with adjacent grids, sorting all image grids based on confidence, and retaining image grids with a confidence threshold value.

[0122] In order to reduce the impact of the ocean environment with obvious motion scenes on the positioning accuracy, starting from the scene assumption of rigid structure, the initially corrected image grids are merged with high confidence, and then the maximum inter-grid variance is performed to select image grids that are evenly distributed in the scene.

[0123] Step 05: Construct a transformation matrix based on the updated initial matching point pair set, and determine the transformation matrix using an overall loss function that combines global constraints with local constraints.

[0124] Only focusing on the local information in the image for image matching correction, while ignoring the global features hidden in the visual task, will mislead the final correction result; therefore, in step 5, a combination of global constraints and local constraints is used to perform the final matching correction on the image after merging the grids to obtain the final matching correspondence set S in * :

[0125]

[0126] Among them, θ and θ′ are parameters that balance local constraints and global constraints, Cost dis Cost mot They are the local neighborhood distance constraint and the local neighborhood motion constraint in the multiple local constraints in step 4, respectively. gl is the loss function of the global constraint:

[0127]

[0128] Where T(p) is the image I to its matching corresponding image The global transformation mapping function, T(p) corresponds to the final matching set S in * The solution is a complementary process.

[0129] Step 06: Correct the merged image grid based on the transformation matrix to obtain a final set of matching point pairs to complete image matching correction.

[0130] The image matching correction method provided in this embodiment is based on a deep-sea robot that can be applied to underwater operations, and provides an image matching correction technology. By correcting image matching, this method can efficiently and robustly correct image matching association information, thereby further improving the positioning accuracy of the robot and enhancing the applicability of the deep-sea robot in marine oil and gas exploration, development and operations.

[0131] Embodiment 4:

[0132] This embodiment provides an electronic device, which may be a mobile phone, a computer or a tablet computer, etc., including a memory and a processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the image matching correction method described in Embodiment 1 is implemented. It is understood that the electronic device may also include an input / output (I / O) interface and a communication component.

[0133] The processor is used to execute all or part of the steps in the image matching correction method in Embodiment 1. The memory is used to store various types of data, which may include instructions of any application or method in the electronic device, and data related to the application.

[0134] The processor may be an application specific integrated circuit (ASIC), a digital signal processor (DSP), a programmable logic device (PLD), a field programmable gate array (FPGA), a controller, a microcontroller, a microprocessor or other electronic components, and is used to execute the image matching correction method in the first embodiment. The image matching correction method includes:

[0135] Step 01: Perform preliminary matching on the image pairs to obtain an initial set of matching point pairs.

[0136] Step 02: Grid the initial matching point pair set to obtain an image grid.

[0137] Furthermore, the gridding of the initial matching point pair set to obtain the image grid includes: gridding the initial matching point pair set using a loose quadtree structure. Use a loose quadtree structure to match the initial point pair set Divide the grid to obtain an image grid.

[0138] Step 03: Perform preliminary correction on the erroneous matches in the initial matching, remove the outliers and update the corresponding initial matching point pair set.

[0139] In addition, the image matching correction method also includes: determining a domain set of the initial matching point pair set by constructing a hash function.

[0140] Specifically, construct a hash function to determine the initial matching set Neighborhood information Among them, N i is the neighborhood set of the i-th initial matching point in image I, For images The neighborhood set of the i-th initial matching point in . Since the preliminary matching has many outliers, the divided neighborhood may include both dense and sparse cases.

[0141] The preliminary correction of the mismatch in the initial matching includes: preliminary correction of the mismatch in the initial matching based on multiple local constraint loss functions and a domain set of the initial matching point pair set.

[0142] Multiple local constraints are used to preliminarily correct the mismatches in the initial matching and update the matching set S in , and its overall loss function with multiple local constraints is as follows:

[0143]

[0144] in, is the number of reliable correspondences in image matching, and θ is a parameter that balances multiple local constraints.

[0145] The multiple local constraints include: local neighborhood motion constraints and local neighborhood distance constraints.

[0146] 1. Local neighborhood motion constraints:

[0147] If the matching results within the neighborhood are accurate, then in most cases, the difference between the motion vectors of the feature point pairs will be small. Therefore, the difference in the motion vectors within the neighborhood is used to verify the preservation of the local structure, and then filter out the outlier matching correspondences. The loss function of the local neighborhood motion constraint is as follows:

[0148]

[0149] in, Represents the neighborhood set N of the i-th initial matching point in image I i The number of feature points in v i and v j They represent the motion vectors of the initial matching point pair and its neighborhood matching point pair in the image pair, and the difference between the two is expressed as Dist(v i , v j ) is used to measure the motion of the local neighborhood. The larger the value calculated, the more consistent the motion of the local neighborhood. i , v j ) can be expressed as:

[0150]

[0151] 2. Local neighborhood distance constraint

[0152] The camera's motion process does not affect the distribution of neighboring point pairs for reliable feature matching, that is, the neighborhood corresponding to the correct match can better maintain a smaller distance relationship than the neighborhood corresponding to the incorrect match. Therefore, a local neighborhood distance constraint is set to map the feature points in the first image to the matched second image, and compare the distance with all feature points in the neighborhood in the second image to help filter out obviously wrong feature matches. The loss function of the local neighborhood distance constraint is as follows:

[0153]

[0154] in, is the i-th initial matching point p in image I i The matching image The matching in To match the image The i-th matching point in .

[0155] Step 04: Merge the preliminarily rectified image grids and select image grids that are evenly distributed in the scene based on the maximum inter-grid variance.

[0156] Furthermore, the merging of the preliminarily corrected image grids includes: merging image grids containing less than a preset number of feature points in the preliminarily corrected image grids with adjacent grids, sorting all image grids based on confidence, and retaining image grids with a confidence threshold value.

[0157] In order to reduce the impact of the ocean environment with obvious motion scenes on the positioning accuracy, starting from the scene assumption of rigid structure, the initially corrected image grids are merged with high confidence, and then the maximum inter-grid variance is performed to select image grids that are evenly distributed in the scene.

[0158] Step 05: Construct a transformation matrix based on the updated initial matching point pair set, and determine the transformation matrix using an overall loss function that combines global constraints with local constraints.

[0159] Only focusing on the local information in the image for image matching correction, while ignoring the global features hidden in the visual task, will mislead the final correction result; therefore, in step 5, a combination of global constraints and local constraints is used to perform the final matching correction on the image after merging the grids to obtain the final matching correspondence set S in * :

[0160]

[0161] Among them, θ and θ′ are parameters that balance local constraints and global constraints, Cost disCost mot They are the local neighborhood distance constraint and the local neighborhood motion constraint in the multiple local constraints in step 4, respectively. gl is the loss function of the global constraint:

[0162]

[0163] Where T(p) is the image I to its matching corresponding image The global transformation mapping function, T(p) corresponds to the final matching set S in * The solution is a complementary process.

[0164] Step 06: Correct the merged image grid based on the transformation matrix to obtain a final set of matching point pairs to complete image matching correction.

[0165] The image matching correction method provided in this embodiment is based on a deep-sea robot that can be applied to underwater operations, and provides an image matching correction technology. By correcting image matching, this method can efficiently and robustly correct image matching association information, thereby further improving the positioning accuracy of the robot and enhancing the applicability of the deep-sea robot in marine oil and gas exploration, development and operations.

[0166] The memory may be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as a static random access memory (SRAM), an electrically erasable programmable read-only memory (EEPROM), an erasable programmable read-only memory (EPROM), a programmable read-only memory (PROM), a read-only memory (ROM), a magnetic memory, a flash memory, a disk or an optical disk.

[0167] Embodiment five:

[0168] This embodiment also provides a computer-readable storage medium. Each functional unit in each embodiment of the present invention may be integrated into a processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. If the function is implemented in the form of a software functional unit and sold or used as an independent product, it may be stored in a computer-readable storage medium.

[0169] Based on such understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention.

[0170] The aforementioned storage media include: flash memory, hard disk, multimedia card, card-type memory (for example, SD or DX memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, server, APP application mall, etc. Various media that can store program check codes, on which a computer program is stored, and when the computer program is executed by a processor, the following method steps can be implemented:

[0171] Step 01: Perform preliminary matching on the image pairs to obtain an initial set of matching point pairs.

[0172] Step 02: Grid the initial matching point pair set to obtain an image grid.

[0173] Furthermore, the gridding of the initial matching point pair set to obtain the image grid includes: gridding the initial matching point pair set using a loose quadtree structure. Use a loose quadtree structure to match the initial point pair set Divide the grid to obtain an image grid.

[0174] Step 03: Perform preliminary correction on the erroneous matches in the initial matching, remove the outliers and update the corresponding initial matching point pair set.

[0175] In addition, the image matching correction method also includes: determining a domain set of the initial matching point pair set by constructing a hash function.

[0176] Specifically, construct a hash function to determine the initial matching set Neighborhood information Among them, N i is the neighborhood set of the i-th initial matching point in image I, For images The neighborhood set of the i-th initial matching point in . Since the preliminary matching has many outliers, the divided neighborhood may include both dense and sparse cases.

[0177] The preliminary correction of the mismatch in the initial matching includes: preliminary correction of the mismatch in the initial matching based on multiple local constraint loss functions and a domain set of the initial matching point pair set.

[0178] Multiple local constraints are used to preliminarily correct the mismatches in the initial matching and update the matching set S in , and its overall loss function with multiple local constraints is as follows:

[0179]

[0180] in, is the number of reliable correspondences in image matching, and θ is a parameter that balances multiple local constraints.

[0181] The multiple local constraints include: local neighborhood motion constraints and local neighborhood distance constraints.

[0182] 1. Local neighborhood motion constraints:

[0183] If the matching results within the neighborhood are accurate, then in most cases, the difference between the motion vectors of the feature point pairs will be small. Therefore, the difference in the motion vectors within the neighborhood is used to verify the preservation of the local structure, and then filter out the outlier matching correspondences. The loss function of the local neighborhood motion constraint is as follows:

[0184]

[0185] in, Represents the neighborhood set N of the i-th initial matching point in image I i The number of feature points in v i and v j They represent the motion vectors of the initial matching point pair and its neighborhood matching point pair in the image pair, and the difference between the two is expressed as Dist(v i , v j ) is used to measure the motion of the local neighborhood. The larger the value calculated, the more consistent the motion of the local neighborhood. i , v j ) can be expressed as:

[0186]

[0187] 2. Local neighborhood distance constraint

[0188] The camera's motion process does not affect the distribution of neighboring point pairs for reliable feature matching, that is, the neighborhood corresponding to the correct match can better maintain a smaller distance relationship than the neighborhood corresponding to the incorrect match. Therefore, a local neighborhood distance constraint is set to map the feature points in the first image to the matched second image, and compare the distance with all feature points in the neighborhood in the second image to help filter out obviously wrong feature matches. The loss function of the local neighborhood distance constraint is as follows:

[0189]

[0190] in, is the i-th initial matching point p in image I i The matching image The matching in To match the image The i-th matching point in .

[0191] Step 04: Merge the preliminarily rectified image grids and select image grids that are evenly distributed in the scene based on the maximum inter-grid variance.

[0192] Furthermore, the merging of the preliminarily corrected image grids includes: merging image grids containing less than a preset number of feature points in the preliminarily corrected image grids with adjacent grids, sorting all image grids based on confidence, and retaining image grids with a confidence threshold value.

[0193] In order to reduce the impact of the ocean environment with obvious motion scenes on the positioning accuracy, starting from the scene assumption of rigid structure, the initially corrected image grids are merged with high confidence, and then the maximum inter-grid variance is performed to select image grids that are evenly distributed in the scene.

[0194] Step 05: Construct a transformation matrix based on the updated initial matching point pair set, and determine the transformation matrix using an overall loss function that combines global constraints with local constraints.

[0195] Only focusing on the local information in the image for image matching correction, while ignoring the global features hidden in the visual task, will mislead the final correction result; therefore, in step 5, a combination of global constraints and local constraints is used to perform the final matching correction on the image after merging the grids to obtain the final matching correspondence set S in * :

[0196]

[0197] Among them, θ and θ′ are parameters that balance local constraints and global constraints, Cost dis Cost mot They are the local neighborhood distance constraint and the local neighborhood motion constraint in the multiple local constraints in step 4, respectively. gl is the loss function of the global constraint:

[0198]

[0199] Where T(p) is the image I to its matching corresponding image The global transformation mapping function, T(p) corresponds to the final matching set S in * The solution is a complementary process.

[0200] Step 06: Correct the merged image grid based on the transformation matrix to obtain a final set of matching point pairs to complete image matching correction.

[0201] The image matching correction method provided in this embodiment is based on a deep-sea robot that can be applied to underwater operations, and provides an image matching correction technology. By correcting image matching, this method can efficiently and robustly correct image matching association information, thereby further improving the positioning accuracy of the robot and enhancing the applicability of the deep-sea robot in marine oil and gas exploration, development and operations.

[0202] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working process of the above-described system, device and unit can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0203] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms. The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the scheme of this embodiment.

[0204] The basic principles of the present application are described above in conjunction with specific embodiments. However, it should be noted that the advantages, strengths, effects, etc. mentioned in the present application are only examples and not limitations, and it cannot be considered that these advantages, strengths, effects, etc. are required by each embodiment of the present application. In addition, the specific details disclosed above are only for the purpose of illustration and ease of understanding, not for limitation, and the above details do not limit the present application to being implemented by adopting the above specific details.

[0205] The block diagrams of devices, apparatuses, equipment, and systems involved in this application are only illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As will be appreciated by those skilled in the art, these devices, apparatuses, equipment, and systems may be connected, arranged, or configured in any manner.

[0206] It should also be noted that in the apparatus, device and method of the present application, each component or each step can be decomposed and / or recombined. Such decomposition and / or recombination should be regarded as equivalent solutions of the present application.

[0207] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present application. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of the present application. Therefore, the present application is not intended to be limited to the aspects shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

[0208] In the description of the application, the meaning of "multiple" is at least two, for example, two, three, etc., unless otherwise clearly and specifically limited. In the embodiment of the present application, all directional indications (such as up, down, left, right, front, back, top, bottom ...) are only used to explain the relative position relationship, motion conditions, etc. between the components under a certain specific posture (as shown in the drawings). If the specific posture changes, the directional indication also changes accordingly. In addition, the terms "including" and "having" and any of their deformations are intended to cover non-exclusive inclusions. For example, the process, method, system, product or equipment comprising a series of steps or units are not limited to the steps or units listed, but optionally also include the steps or units not listed, or optionally also include other steps or units inherent to these processes, methods, products or equipment.

[0209] In addition, the reference to "embodiment" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiment may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0210] The above is only a specific implementation of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims. The above is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

[0211] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. An image matching correction method, characterized in that: include: Perform preliminary matching on the image pairs to obtain an initial set of matching point pairs; Gridding the initial matching point pair set to obtain an image grid; Perform preliminary correction on the erroneous matches in the initial matching, remove the outliers and update the corresponding set of initial matching point pairs; The image grids after preliminary correction are merged, and the image grids uniformly distributed in the scene are selected based on the maximum inter-grid variance; A transformation matrix is ​​constructed based on the updated initial matching point pair set, and the transformation matrix is ​​determined by using an overall loss function combining global constraints and local constraints; The merged image grid is corrected based on the transformation matrix to obtain a final set of matching point pairs to complete image matching correction.

2. The image matching correction method according to claim 1, characterized in that: The gridding the initial matching point pair set to obtain an image grid includes: gridding the initial matching point pair set using a loose quadtree structure.

3. The image matching correction method according to claim 1, characterized in that: Also includes: The domain set of the initial matching point pair set is determined by constructing a hash function.

4. The image matching correction method according to claim 3, characterized in that: The preliminary correction of the mismatch in the initial matching includes: preliminary correction of the mismatch in the initial matching based on multiple local constraint loss functions and a domain set of the initial matching point pair set.

5. The image matching correction method according to claim 4, characterized in that: The multiple local constraint loss function includes: a local neighborhood motion constraint and a local neighborhood distance constraint.

6. The image matching correction method according to claim 1, characterized in that: The merging of the preliminarily corrected image grids includes: merging image grids containing less than a preset number of feature points in the preliminarily corrected image grids with adjacent grids, sorting all image grids based on confidence, and retaining image grids with a confidence threshold value.

7. An image matching correction device, characterized in that: include: An initial matching module, used to perform preliminary matching on the image pairs to obtain an initial set of matching point pairs; A grid division module, used for performing grid division on the initial matching point pair set to obtain an image grid; A first correction module, used to perform preliminary correction on the erroneous matches in the initial matching, remove the outliers and update the corresponding initial matching point pair set; An image grid processing module is used to merge the image grids after preliminary correction and select image grids that are evenly distributed in the scene based on the maximum inter-grid variance; The transformation matrix module is used to construct the transformation matrix based on the updated initial matching point pair set, and determine the transformation matrix using the overall loss function combining global constraints and local constraints; The second correction module is used to correct the merged image grid based on the transformation matrix to obtain a final set of matching point pairs to complete image matching correction.

8. A positioning method for submerged sea exploration operations, characterized in that: Image matching correction is performed using the image matching correction method described in any one of claims 1 to 6, and underwater environment positioning is performed based on the image after image matching correction.

9. An electronic device, characterized in that: It comprises a memory and a processor, wherein the memory is used to store one or more computer instructions, wherein when the one or more computer instructions are executed by the processor, the image matching correction method as described in any one of claims 1 to 6 is implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, it is used to implement the image matching correction method as described in any one of claims 1 to 6.

Images