Image feature point matching method and device based on oil field scene and electronic equipment

Abstract

The invention provides an image feature point matching method and device based on an oil field scene and electronic equipment, and belongs to the technical field of visual positioning. The method comprises the following steps: inputting two images to be processed into an image feature point extraction network to obtain feature points of the two images to be processed; feature point matching is carried out on the feature points of the two images to be processed, and matching point pairs between the two images to be processed are determined; the image feature point extraction network is constructed based on two large kernel integral branches, each large kernel integral branch is constructed based on depth separable convolution, depth separable cavity convolution and point convolution, and the image feature point extraction network is obtained based on target triple training; the target triad is used for indicating a non-matching point pair between the two training images and a non-matching point pair in a single training image. The image feature point matching method based on the oil field scene provided by the invention has very strong robustness for the repeated texture condition of the oil field scene and the violent change condition of the illumination visual angle.

Description

Technical Field

[0001] The present invention relates to the field of visual positioning technology, and in particular to an image feature point matching method, device and electronic equipment based on oil field scenes. Background Art

[0002] In the field of petrochemicals, mobile robots are widely used in tasks such as pipeline inspection, oil field inspection, and hazardous environment operations. Visual positioning technology is the key to mobile robots' autonomous positioning and navigation. Image feature point matching is the basic step for accurate visual positioning.

[0003] Existing deep learning-based methods mainly use convolutional neural networks to extract image feature points, ignoring the long-range contextual information in the image and performing poorly in repetitive texture and weak texture areas that are common in oilfield scenes. Summary of the invention

[0004] The present invention provides an oilfield scene-based image feature point matching method, device and electronic equipment, which are used to solve the problem that the existing oilfield scene-based image feature point matching method in the prior art performs poorly in situations such as repeated textures and weak textures.

[0005] The present invention provides an image feature point matching method based on an oilfield scene, comprising: Inputting two images to be processed into an image feature point extraction network respectively to obtain feature points of the two images to be processed; Performing feature point matching on the feature points of the two images to be processed to determine matching point pairs between the two images to be processed; Among them, the image feature point extraction network is constructed based on two large kernel convolution branches, each of the large kernel convolution branches is constructed based on depthwise separable convolution, depthwise separable hole convolution and point convolution, and the image feature point extraction network is trained based on target triples, and the target triples are used to indicate non-matching point pairs between two training images and non-matching point pairs in a single training image.

[0006] In some embodiments, the image feature point extraction network is trained in the following manner: Extracting multi-scale features of the training image; Based on the multi-scale features, respectively determining a target heat map of the training image and a descriptor of the training image, wherein the target heat map is used to indicate key point information of the training image, and the descriptor of the training image is used to characterize local information and long-range context information of the training image, and the long-range context information is extracted by the two large-kernel convolution branches; Determining the target triplet based on the target heat map and the descriptor of the training image; Based on the target triplet, the image feature point extraction network is trained.

[0007] In some embodiments, the determining, based on the multi-scale features, the target heat map of the training image and the descriptor of the training image, respectively, includes: Performing key point detection on the multi-scale features respectively to obtain a multi-scale heat map of the training image, wherein the key point detection includes a 3×3 convolution operation, a Sigmoid function processing, and a bilinear interpolation operation; A weighted sum is performed on the multi-scale thermal maps to obtain the target thermal map.

[0008] In some embodiments, the determining, based on the multi-scale features, the target heat map of the training image and the descriptor of the training image, respectively, includes: Based on the multi-scale features, obtaining the local information and the long-distance context information; A descriptor of the training image is obtained based on the local information and the long-range context information.

[0009] In some embodiments, obtaining the local information and the long-distance context information based on the multi-scale features includes: Downsampling the multi-scale features to obtain downsampled multi-scale features; Concatenating the downsampled multi-scale features along the channel dimension to obtain concatenated multi-scale features; Performing point convolution on the concatenated multi-scale features to obtain the local information; The multi-scale features are respectively input into the two large-kernel convolution branches and the feature information obtained by the 3×3 convolution branch, and processed by the residual connection branch to obtain the long-distance context information.

[0010] In some embodiments, the step of training the image feature point extraction network based on the target triplet includes: Determining a loss function of the descriptor based on the loss function of the target triplet; Determining a loss function for key point detection based on the common view key points in the target heat map; Based on the loss function of the descriptor and the loss function of the key point detection, the loss function of the image feature point extraction network is determined.

[0011] In some embodiments, the target triplet is obtained by: Extract the key point coordinates in the target thermal map of the two training images respectively; Determining descriptors of matching point pairs of the two training images based on the key point coordinates; Determining descriptors of non-matching point pairs of the two training images based on the descriptors of the matching point pairs; The target triplet is determined based on the descriptors of the matching point pairs and the descriptors of the non-matching point pairs.

[0012] The present invention also provides an image feature point matching device based on an oilfield scene, comprising: A first determination module is used to input the two images to be processed into the image feature point extraction network respectively to obtain the feature points of the two images to be processed; A second determination module is used to perform feature point matching on the feature points of the two images to be processed to determine matching point pairs between the two images to be processed; Among them, the image feature point extraction network is constructed based on two large kernel convolution branches, each of the large kernel convolution branches is constructed based on depthwise separable convolution, depthwise separable hole convolution and point convolution, and the image feature point extraction network is trained based on target triples, and the target triples are used to indicate non-matching point pairs between two training images and non-matching point pairs in a single training image.

[0013] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the image feature point matching method based on oilfield scenes as described above is implemented.

[0014] The present invention also provides a non-transitory computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the image feature point matching method based on the oil field scene as described above is implemented.

[0015] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the image feature point matching method based on any one of the above oilfield scenes is implemented.

[0016] The image feature point matching method, device and electronic device based on oilfield scenes provided by the present invention construct an image feature point extraction network through a large-kernel convolution branch, which can perceive a larger area and capture long-distance dependency information, thereby extracting features that are more robust to difficult scenes such as repeated textures; at the same time, non-matching point pairs between training image pairs and non-matching point pairs within a single training image are considered, and more difficult non-matching point pairs can be adaptively selected for training, which can improve the discriminability of descriptors, so that in the case of drastic changes in viewing angle illumination and repeated texture areas in oilfield scene images, feature point extraction is also highly robust, and a large number of correct matching point pairs can be established, which has broad application prospects. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0018] Figure 1 It is a flow chart of the image feature point matching method based on oil field scene provided by the present invention; Figure 2 It is a structural schematic diagram of an image feature point extraction network of an image feature point matching method based on oilfield scenes provided by the present invention; Figure 3 This is an experimental effect diagram of the oilfield scene-based image feature point matching method provided by the present invention in an oil production scene; Figure 4 It is a structural schematic diagram of an image feature point matching device based on an oil field scene provided by the present invention; Figure 5 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0019] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0020] Combine the following Figure 1-Figure 5 The present invention describes an image feature point matching method, device and electronic device based on oil field scenes.

[0021] The execution subject of the image feature point matching method based on oilfield scenes provided by the present invention can be an electronic device, a component in an electronic device, an integrated circuit, or a chip. The electronic device can be a mobile electronic device or a non-mobile electronic device. Exemplarily, the mobile electronic device can be a mobile phone, a tablet computer, a laptop computer, a PDA, an in-vehicle electronic device, a wearable device, an ultra-mobile personal computer (UMPC), a netbook or a personal digital assistant (PDA), etc., and the non-mobile electronic device can be a server, a network attached storage (NAS), a personal computer (PC), a television (TV), a teller machine or a self-service machine, etc., which is not specifically limited by the present invention.

[0022] The following takes the method for matching image feature points based on oil field scenes provided by the present invention executed by a computer as an example to explain the technical solution of the present invention in detail.

[0023] Figure 1 Schematic diagram of the flow of the image feature point matching method based on oil field scene provided by the present invention. Figure 1 The image feature point matching method based on oil field scene provided by the present invention includes: Step 110, inputting the two images to be processed into the image feature point extraction network respectively to obtain the feature points of the two images to be processed; Step 120, performing feature point matching on the feature points of the two images to be processed to determine matching point pairs between the two images to be processed; Among them, the image feature point extraction network is constructed based on two large kernel convolution branches, each of which is constructed based on depthwise separable convolution, depthwise separable atrous convolution and point convolution. The image feature point extraction network is trained based on target triplets, and the target triplets are used to indicate non-matching point pairs between two training images and non-matching point pairs in a single training image.

[0024] In the related art, under the conditions of repetitive and weak textures common in oilfield scenes, the existing image feature point extraction methods cannot provide enough correct matches for accurate visual positioning tasks.

[0025] In order to improve the matching performance of the learning-based image feature point extraction algorithm, the image feature point extraction network in the present invention includes a feature extraction module LKCRFM based on large kernel convolution enhancement. The feature extraction module LKCRFM includes two efficient large kernel convolution branches.

[0026] Each large kernel convolution branch consists of two depth-wise separable convolutions and one point convolution. The large kernel convolution branch can perceive a larger area and capture long-distance dependency information, thereby extracting features that are more robust to difficult scenes such as repeated textures.

[0027] It should be noted that directly increasing the convolution kernel of the ordinary convolution operation will greatly increase the amount of calculation and the number of parameters. Therefore, the present invention adopts an efficient large-kernel convolution layer to construct a feature extraction module LKCRFM. An ordinary K×K convolution is approximately decomposed into the following three parts: a K / d×K / d depth-separable dilated convolution, a (2d-1)×(2d-1) depth-separable convolution and an ordinary 1×1 convolution (point convolution is an ordinary convolution with a convolution kernel size of 1. Where d is the dilation coefficient.

[0028] Compared with an ordinary K×K convolution, the above approximate decomposition process greatly reduces the computational complexity and the number of parameters.

[0029] Different from the existing method of selecting non-matching point pairs only between two training images, the present invention simultaneously considers the non-matching point pairs between the training image pairs and the non-matching point pairs within a single training image, and can adaptively select more difficult non-matching point pairs for training, thereby improving the discriminability of the descriptor.

[0030] In actual implementation, the present invention first trains the image feature point extraction network by constructing target triplets. After the training is completed, the trained image feature point extraction network is used to extract feature points from two images to be processed. Then, the feature points of the two images to be processed are matched using the bidirectional nearest neighbor matching method to obtain point matching between the image pairs. Finally, these point matchings are used for camera motion estimation and visual positioning.

[0031] The two images to be processed are image pairs that require feature point extraction and matching.

[0032] The image feature point matching method based on oilfield scenes provided by the present invention constructs an image feature point extraction network through a large-kernel convolution branch, can perceive a wider range of areas, capture long-distance dependency information, and thus extract features that are more robust to difficult scenes such as repeated textures; at the same time, non-matching point pairs between training image pairs and non-matching point pairs within a single training image are considered, and more difficult non-matching point pairs can be adaptively selected for training, which can improve the discriminability of descriptors, so that in the case of drastic changes in viewing angle illumination and repeated texture areas in oilfield scene images, feature point extraction is also highly robust, a large number of correct matching point pairs can be established, and the method has broad application prospects.

[0033] In some embodiments, the image feature point extraction network is trained in the following manner: Extract multi-scale features of training images; Based on multi-scale features, the target heat map and the descriptor of the training image are determined respectively. The target heat map is used to indicate the key point information of the training image, and the descriptor of the training image is used to characterize the local information and long-distance context information of the training image. The long-distance context information is extracted by two large-kernel convolution branches. Determine the target triplet based on the target heat map and the descriptor of the training image; Based on the target triples, the image feature point extraction network is trained.

[0034] In actual implementation, the image feature point extraction network LKCRF-Net constructed by the present invention is as follows Figure 2 shown.

[0035] Given a width of W and height H The training images I , the training image I Input to the image feature point extraction network.

[0036] The present invention uses multiple consecutive convolutional layers, pooling layers and ReLU (rectified linear unit) activation functions to extract multi-scale features of training images. F 1. F 2 and F 3.

[0037] Specifically, the training images I After 3×3 convolution and ReLU activation function processing, the output channel number is 32 and the width is W and height H Features F 1.

[0038] feature F 1 After the maximum pooling layer, 3×3 convolution and ReLU activation function processing, and then after 3×3 convolution and ReLU activation function processing, the output channel number is 64 and the width is W / 2 and the height is H Features of / 2 F 2.

[0039] feature F 2 After the maximum pooling layer, 3×3 convolution and ReLU activation function processing, and then after 3×3 convolution and ReLU activation function processing, the output channel number is 128 and the width is W / 4 and the height is H Features of / 4 F 3.

[0040] feature F 1. Features F 2 and Features F 3 constitute multi-scale features.

[0041] Based on multi-scale features F 1. F 2 and F 3. Perform multi-scale key point detection and extract descriptors on the training images.

[0042] Key point detection can obtain the target heat map of the training image, and the descriptor can represent the local information and long-distance contextual information of the training image.

[0043] In actual implementation, given two training images, the target heat map and the descriptor of the training image can be obtained through the image feature point extraction network LKCRF-Net.

[0044] In practical applications, the present invention mainly uses a non-maximum suppression algorithm to detect key points in the target heat map, and based on the dense matching relationship between the two training images, the correct corresponding points corresponding to the key points can be determined. Among them, the key point and the correct corresponding point corresponding to the key point constitute a correct matching point pair. Based on the descriptor of the training image, the descriptor of the key point and the descriptor of its correct corresponding point can be determined. Based on the Euclidean distance between the descriptors of the correct matching point pair and the descriptor distance of the most difficult non-matching point pair, the target triplet can be constructed and then the target triplet can be used to train the image feature point extraction network.

[0045] The image feature point matching method based on oilfield scenes provided by the present invention can perform multi-scale key point detection by training an image feature point extraction network, and can also fuse local information and long-distance context information of the training image, thereby extracting a descriptor with stronger discriminability and a more accurate target heat map, thereby improving the image feature point matching performance.

[0046] In some embodiments, based on the multi-scale features, determining the target heat map of the training image and the descriptor of the training image respectively includes: The key points of multi-scale features are detected respectively to obtain the multi-scale heat map of the training image. The key point detection includes 3×3 convolution operation, Sigmoid function processing and bilinear interpolation operation. The multi-scale heat maps are weighted summed to obtain the target heat map.

[0047] In actual implementation, in order to perform accurate multi-scale key point detection, the present invention constructs three key point detection heads using 3×3 convolution, sigmoid function and bilinear interpolation operations.

[0048] The three key point detection heads are based on multi-scale features F 1. F 2 and F 3. Estimating the heatmap of key points at three scales S 1. S 2 and S 3.

[0049] In order to fuse the key point information of multiple scales, the heat map S 1. S 2 and S 3 Perform weighted summation to obtain the final heat map of the key points, that is, the target heat map .

[0050] in, ω 1、 ω 2 and ω 3 is the weight, .

[0051] In some embodiments, based on the multi-scale features, determining the target heat map of the training image and the descriptor of the training image respectively includes: Based on multi-scale features, local information and long-distance context information are obtained; Based on local information and long-range contextual information, the descriptor of the training image is obtained.

[0052] It should be noted that context information refers to the information about the pixels around the target pixel, that is, neighborhood information, which can be used to enhance the details and edges of the image and improve the performance of image processing and recognition; long distance refers to the relationship between the target pixel and the pixels that are far away from it.

[0053] Local information refers to the local structure or texture information in the image that is unique, stable and distinguishable. This local information is usually composed of local feature points such as edges, corners, spots, etc. These feature points can remain unchanged or relatively stable under operations such as image transformation, rotation, and scaling.

[0054] In this step, local information and long-distance contextual information can be fused to extract descriptors with stronger discriminability.

[0055] In some embodiments, local information and long-range context information are obtained based on multi-scale features, including: Down-sample the multi-scale features to obtain down-sampled multi-scale features; The downsampled multi-scale features are concatenated along the channel dimension to obtain concatenated multi-scale features; The concatenated multi-scale features are point-convolved to obtain local information.

[0056] In actual implementation, the present invention combines multi-scale features F 1. F 2 and F 3 are downsampled to H / 4×W / 4 respectively. The downsampling can be bilinear interpolation downsampling

[0057] Specifically, the number of channels is 32 and the width is W and height H Features F After downsampling, the number of channels is 32 and the width is H / 4 and the height is H / 4 features.

[0058] The number of channels is 64 and the width is W / 2 and the height is H Features of / 2 F After downsampling, the number of channels is 64 and the width is H / 4 and the height is H / 4 features.

[0059] The number of channels is 128 and the width is W / 4 and the height is H Features of / 4 F After downsampling, the number of channels is 128 and the width is H / 4 and the height is H / 4 features.

[0060] Then, the downsampled multi-scale features are concatenated along the channel dimension to obtain the concatenated multi-scale features. Finally, point convolution (1×1 convolution) is used to fully fuse and output multi-scale local information. D s .

[0061] In some embodiments, local information and long-range context information are obtained based on multi-scale features, including: The multi-scale features are input into two large-kernel convolution branches and one 3×3 convolution branch respectively, and the feature information is processed by the residual connection branch to obtain long-distance context information.

[0062] In actual implementation, the two large kernel convolution branches are ELKC21 and ELKC9.

[0063] Long-range contextual information D l The calculation process is as follows: ; Among them, ⊕ means adding corresponding elements.

[0064] ELKC21 is an approximation of a normal 21×21 convolution. ELKC21 consists of a 5×5 depthwise separable convolution, a 7×7 depthwise separable atrous convolution (with a dilation factor of 3), and a normal 1×1 convolution.

[0065] ELKC9 consists of a 5×5 depthwise separable convolution, a 3×3 depthwise separable atrous convolution (with a dilation factor of 3), and a normal 1×1 convolution.

[0066] The calculation process of ELKC21 and ELKC9 is as follows: ; ; Among them, Conv1(•) represents a normal 1×1 convolution, DConv7d=3(•) represents a 7×7 depth-wise separable atrous convolution (with a dilation factor of 3), DConv3d=3(•) represents a 3×3 depth-wise separable atrous convolution (with a dilation factor of 3), and DConv5(•) represents a 5×5 depth-wise separable convolution.

[0067] In some embodiments, the target triplet is obtained by: Extract the key point coordinates in the target heat map of the two training images respectively; Based on the key point coordinates, determine the descriptors of the matching point pairs of the two training images; Based on the descriptors of the matching point pairs, the descriptors of the non-matching point pairs of the two training images are determined; Based on the descriptors of the matching point pairs and the descriptors of the non-matching point pairs, the target triplet is determined.

[0068] In actual implementation, given a pair of training images, including training images I 1 and training images I 2. By inputting a pair of training images into the image feature point extraction network LKCRF-Net, the target heat map of the pair of training images is output S (1) and target heatmap S (2) , and the descriptor D 1 and descriptor D 2. Training images I 1 and target heat map S (1) and descriptors D 1 corresponds to the training image I 2 and target heat map S (2) and descriptors D 2 corresponding.

[0069] In practical applications, learning-based image feature point extraction algorithms mainly use non-maximum suppression algorithms to detect key points. In order to simulate the non-maximum suppression algorithm during the training process,

[0070] First, the target heat map S (1) and target heatmap S (2) Divide into non-overlapping grid cells, each of which is a local window. The grid cell size can be N × N , N =8.

[0071] Target heat map S (1) For example, in each grid, a key point is extracted using the Soft-argmax operation. Given a grid of size N × N The calculation process of extracting key points in the grid unit is as follows: ; ; in, is the pixel coordinate of the current key point detection, is the coordinate of any pixel point in the entire grid unit traversed, Represents pixel coordinates is the probability of the key point, The coordinates of the key points extracted from this grid cell; T det is the temperature coefficient, which is used to control the smoothness of Soft-argmax.

[0072] Based on known training images I 1 and training images I 2, can be used in the training image I 2. Determine the target heat map S (1) The correct corresponding points of the key points in .

[0073] Target heat map S (1) A key point in P 1i is the reference point, which is in the training image I The correct corresponding point in 2 is Key Points P 1i And the correct corresponding point is That is the matching point pair.

[0074] Represents the training image I 2. It is worth noting that Distribution on training images I 2, not just in the training image I 2 in the common viewing area.

[0075] Indicates key points P 1i The descriptor of Indicates key points P 1i The correct corresponding point The descriptor of Indicates key points P 2j The descriptor of .

[0076] Euclidean distance of the descriptor of the correctly matched point pair It is expressed by the following formula: ; Euclidean distance of the descriptor of the most difficult non-matching point pair It is expressed by the following formula: ; in, Descriptor representing the most difficult non-matching point.

[0077] It should be noted that the most difficult non-matching point pair refers to the non-matching point pair with the smallest Euclidean distance of the descriptor.

[0078] The number of the most difficult non-matching point E It is expressed by the following formula: ; The above method determines the triple Different from the existing method of selecting non-matching point pairs only between two training images, the method proposed in the present invention considers both the non-matching point pairs between the training image pairs and the non-matching point pairs within a single image, and can adaptively select more difficult non-matching point pairs for training, thereby improving the discriminability of the descriptor.

[0079] The loss function of the target triplet It is expressed by the following formula: ; in, , is the descriptor Euclidean distance margin of the correctly matched point pair, is the Euclidean distance margin of the descriptor of non-matching point pairs. Target heat mapS (1) All common view key points in can be selected as reference points.

[0080] The present invention is a target heat map S (1) The function for calculating all reference points in is the loss term shown in the above formula.

[0081] L des1 Represents the target heat map S (1) The average of the loss terms of all reference points in , L des1 It is expressed by the following formula:

[0082] in, C Represents the target heat map S (1) The number of reference points in the.

[0083] Similarly, the present invention also selects a target thermal map S (2) All the common view key points in are used as reference points to calculate the target heat map S (2) The average of the loss terms of all reference points in L des2 .

[0084] The loss function of the final descriptor L des It is expressed by the following formula: .

[0085] The image feature point matching method based on oil field scenes provided by the present invention innovatively expands the construction range of the target triplet from the common view area to the entire training image, thereby improving the similarity between the training process and the actual application scene.

[0086] In some embodiments, based on the target triplet, training an image feature point extraction network includes: Based on the loss function of the target triplet, determine the loss function of the descriptor; Based on the common view key points in the target heat map, determine the loss function of key point detection; Based on the loss function of the descriptor and the loss function of key point detection, the loss function of the image feature point extraction network is determined.

[0087] In actual implementation, the present invention converts the target heat map S (1) and target heatmap S(2) Divided into grids, the grid unit size is N × N , N =8.

[0088] A key point is extracted from each grid using the Soft-argmax function. S (1) A common view key point in P 1i As the reference point, based on the known training images I 1 and training images I The dense matching relationship between 2 can obtain the common view key points P 1i In the training images I The corresponding points in 2 .

[0089] Assumptions P 2i is the training image I 2 and corresponding points The closest key point, and ,in, t kp represents the distance threshold, t kp =8, corresponding point With key points P 2i The Euclidean distance is based on the common view key point P 1i is the loss function term of the reference point, as shown below: ; Target heat map S (1) All common view key points in can be used as reference points, and their loss function terms are averaged. It is expressed by the following formula: ; in, K Represents the target heat map S (1) Similarly, the present invention also uses the target heat map S (2) All the common view key points in are used as reference points to obtain the average value of their loss function terms L kp2。

[0090] The final loss function used to learn key point detection As shown below: ; According to the loss function of key point detection L And the loss function of the target triplet, it can be determined that the total loss function of the image feature point extraction network is: .

[0091] The image feature point matching method of the present invention is highly robust to the repeated texture conditions and drastic changes in illumination and viewing angle of oilfield scenes. In image matching and visual positioning tasks, the present invention achieves results superior to those of advanced existing algorithms. Even in the face of challenging repeated texture images of oilfield scenes, the present invention can establish many correct matches and has broad application prospects.

[0092] The evaluation results of various image feature point extraction methods on the HPatches dataset are shown in Table 1. Table 1 shows the quantitative results on the HPatches dataset. The higher the Mean Matching Accuracy (MMA), Matching Score (MS) and Homography Accuracy (HA), the better the algorithm effect. In general, the present invention achieves the best image matching effect.

[0093] Table 1

[0094] The outdoor large scene visual positioning accuracy of various image feature point extraction methods on the Aachen Day-Night v1.1 dataset is shown in Table 2. Table 2 shows the outdoor large scene visual positioning evaluation results on the Aachen Day-Night v1.1 dataset. The higher the value, the better the visual positioning effect of the algorithm. It can be seen that the present invention achieves the best visual positioning effect.

[0095] Table 2

[0096] The experimental results of the image feature point matching method provided by the present invention in the oil production scene are as follows: Figure 3 shown.

[0097] Figure 3 Figures (a) and (b) in the figure are the experimental results of the image feature point matching method provided by the present invention in an outdoor oil field scene.

[0098] Figure 3 Figures (c) and (d) in the figure are the experimental results of the image feature point matching method provided by the present invention in indoor production scenes.

[0099] Depend on Figure 3It can be seen that in outdoor oilfield scenes containing a large number of weak texture areas and indoor oil production scenes containing a large number of repeated texture areas, the image feature point extraction network of the present invention can estimate a large number of correct matching point pairs.

[0100] The image feature point matching device based on oilfield scenes provided by the present invention is described below. The image feature point matching device based on oilfield scenes described below and the image feature point matching method based on oilfield scenes described above can be referred to each other.

[0101] Figure 4 Schematic diagram of the structure of the image feature point matching device based on the oil field scene provided by the present invention. Figure 4 The image feature point matching device based on the oil field scene provided by the present invention comprises: A first determination module 410 is used to input two images to be processed into an image feature point extraction network respectively to obtain feature points of the two images to be processed; A second determination module 420 is used to perform feature point matching on the feature points of the two images to be processed to determine matching point pairs between the two images to be processed; Among them, the image feature point extraction network is constructed based on two large kernel convolution branches, each of the large kernel convolution branches is constructed based on depthwise separable convolution, depthwise separable hole convolution and point convolution, and the image feature point extraction network is trained based on target triples, and the target triples are used to indicate non-matching point pairs between two training images and non-matching point pairs in a single training image.

[0102] The image feature point matching device based on oilfield scenes provided by the present invention constructs an image feature point extraction network through a large-kernel convolution branch, can perceive a wider range of areas, capture long-distance dependency information, and thus extract features that are more robust to difficult scenes such as repeated textures; at the same time, non-matching point pairs between training image pairs and non-matching point pairs within a single training image are considered, and more difficult non-matching point pairs can be adaptively selected for training, which can improve the discriminability of descriptors, so that in the case of drastic changes in viewing angle illumination and repeated texture areas in oilfield scene images, feature point extraction is also highly robust, and a large number of correct matching point pairs can be established, which has broad application prospects.

[0103] In some embodiments, the image feature point extraction network is trained in the following manner: Extracting multi-scale features of the training image; Based on the multi-scale features, respectively determining a target heat map of the training image and a descriptor of the training image, wherein the target heat map is used to indicate key point information of the training image, and the descriptor of the training image is used to characterize local information and long-range context information of the training image, and the long-range context information is extracted by the two large-kernel convolution branches; Determining the target triplet based on the target heat map and the descriptor of the training image; Based on the target triplet, the image feature point extraction network is trained.

[0104] In some embodiments, the determining, based on the multi-scale features, the target heat map of the training image and the descriptor of the training image, respectively, includes: Performing key point detection on the multi-scale features respectively to obtain a multi-scale heat map of the training image, wherein the key point detection includes a 3×3 convolution operation, a Sigmoid function processing, and a bilinear interpolation operation; A weighted sum is performed on the multi-scale thermal maps to obtain the target thermal map.

[0105] In some embodiments, the determining, based on the multi-scale features, the target heat map of the training image and the descriptor of the training image, respectively, includes: Based on the multi-scale features, obtaining the local information and the long-distance context information; A descriptor of the training image is obtained based on the local information and the long-range context information.

[0106] In some embodiments, obtaining the local information and the long-distance context information based on the multi-scale features includes: Downsampling the multi-scale features to obtain downsampled multi-scale features; Concatenating the downsampled multi-scale features along the channel dimension to obtain concatenated multi-scale features; Performing point convolution on the concatenated multi-scale features to obtain the local information; The multi-scale features are respectively input into the two large-kernel convolution branches and the feature information obtained by the 3×3 convolution branch, and processed by the residual connection branch to obtain the long-distance context information.

[0107] In some embodiments, the step of training the image feature point extraction network based on the target triplet includes: Determining a loss function of the descriptor based on the loss function of the target triplet; Determining a loss function for key point detection based on the common view key points in the target heat map; Based on the loss function of the descriptor and the loss function of the key point detection, the loss function of the image feature point extraction network is determined.

[0108] In some embodiments, the target triplet is obtained by: Extract the key point coordinates in the target thermal map of the two training images respectively; Determining descriptors of matching point pairs of the two training images based on the key point coordinates; Determining descriptors of non-matching point pairs of the two training images based on the descriptors of the matching point pairs; The target triplet is determined based on the descriptors of the matching point pairs and the descriptors of the non-matching point pairs.

[0109] Figure 5 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 5 As shown, the electronic device may include: a processor 510, a communication interface 520, a memory 530 and a communication bus 540, wherein the processor 510, the communication interface 520 and the memory 530 communicate with each other through the communication bus 540. The processor 510 may call the logic instructions in the memory 530 to execute the image feature point matching method based on the oilfield scene, and the method includes: Inputting two images to be processed into an image feature point extraction network respectively to obtain feature points of the two images to be processed; Performing feature point matching on the feature points of the two images to be processed to determine matching point pairs between the two images to be processed; Among them, the image feature point extraction network is constructed based on two large kernel convolution branches, each of the large kernel convolution branches is constructed based on depthwise separable convolution, depthwise separable hole convolution and point convolution, and the image feature point extraction network is trained based on target triples, and the target triples are used to indicate non-matching point pairs between two training images and non-matching point pairs in a single training image.

[0110] In addition, the logic instructions in the above-mentioned memory 530 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0111] On the other hand, the present invention further provides a computer program product, the computer program product includes a computer program, the computer program can be stored in a non-transitory computer-readable storage medium, when the computer program is executed by a processor, the computer can execute the image feature point matching method based on the oilfield scene provided by the above methods, the method includes: inputting two images to be processed into an image feature point extraction network respectively, to obtain feature points of the two images to be processed; Performing feature point matching on the feature points of the two images to be processed to determine matching point pairs between the two images to be processed; Among them, the image feature point extraction network is constructed based on two large kernel convolution branches, each of the large kernel convolution branches is constructed based on depthwise separable convolution, depthwise separable hole convolution and point convolution, and the image feature point extraction network is trained based on target triples, and the target triples are used to indicate non-matching point pairs between two training images and non-matching point pairs in a single training image.

[0112] In another aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, which is implemented when the computer program is executed by a processor to perform the image feature point matching method based on the oilfield scene provided by the above methods, the method comprising: Inputting two images to be processed into an image feature point extraction network respectively to obtain feature points of the two images to be processed; Performing feature point matching on the feature points of the two images to be processed to determine matching point pairs between the two images to be processed; Among them, the image feature point extraction network is constructed based on two large kernel convolution branches, each of the large kernel convolution branches is constructed based on depthwise separable convolution, depthwise separable hole convolution and point convolution, and the image feature point extraction network is trained based on target triples, and the target triples are used to indicate non-matching point pairs between two training images and non-matching point pairs in a single training image.

[0113] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0114] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0115] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An image feature point matching method based on oil field scenes, characterized in that: include: Inputting two images to be processed into an image feature point extraction network respectively to obtain feature points of the two images to be processed; Performing feature point matching on the feature points of the two images to be processed to determine matching point pairs between the two images to be processed; Among them, the image feature point extraction network is constructed based on two large kernel convolution branches, each of the large kernel convolution branches is constructed based on depthwise separable convolution, depthwise separable hole convolution and point convolution, and the image feature point extraction network is trained based on target triples, and the target triples are used to indicate non-matching point pairs between two training images and non-matching point pairs in a single training image.

2. The image feature point matching method based on oil field scene according to claim 1 is characterized in that: The image feature point extraction network is trained in the following way: Extracting multi-scale features of the training image; Based on the multi-scale features, respectively determining a target heat map of the training image and a descriptor of the training image, wherein the target heat map is used to indicate key point information of the training image, and the descriptor of the training image is used to characterize local information and long-range context information of the training image, and the long-range context information is extracted by the two large-kernel convolution branches; Determining the target triplet based on the target heat map and the descriptor of the training image; Based on the target triplet, the image feature point extraction network is trained.

3. The image feature point matching method based on oil field scene according to claim 2 is characterized in that: The step of determining the target heat map of the training image and the descriptor of the training image based on the multi-scale features includes: Performing key point detection on the multi-scale features respectively to obtain a multi-scale heat map of the training image, wherein the key point detection includes a 3×3 convolution operation, a Sigmoid function processing, and a bilinear interpolation operation; A weighted sum is performed on the multi-scale thermal maps to obtain the target thermal map.

4. The image feature point matching method based on oil field scene according to claim 2 is characterized in that: The step of determining the target heat map of the training image and the descriptor of the training image based on the multi-scale features includes: Based on the multi-scale features, obtaining the local information and the long-distance context information; A descriptor of the training image is obtained based on the local information and the long-range context information.

5. The image feature point matching method based on oil field scene according to claim 4 is characterized in that: The obtaining the local information and the long-distance context information based on the multi-scale features includes: Downsampling the multi-scale features to obtain downsampled multi-scale features; Concatenating the downsampled multi-scale features along the channel dimension to obtain concatenated multi-scale features; Performing point convolution on the concatenated multi-scale features to obtain the local information; The multi-scale features are respectively input into the two large-kernel convolution branches and the feature information obtained by the 3×3 convolution branch, and processed by the residual connection branch to obtain the long-distance context information.

6. The image feature point matching method based on oil field scene according to claim 2 is characterized in that: The step of training the image feature point extraction network based on the target triplet includes: Determining a loss function of the descriptor based on the loss function of the target triplet; Determining a loss function for key point detection based on the common view key points in the target heat map; Based on the loss function of the descriptor and the loss function of the key point detection, the loss function of the image feature point extraction network is determined.

7. The image feature point matching method based on oil field scene according to any one of claims 1 to 6, characterized in that: The target triplet is obtained in the following way: Extract the key point coordinates in the target thermal map of the two training images respectively; Determining descriptors of matching point pairs of the two training images based on the key point coordinates; Determining descriptors of non-matching point pairs of the two training images based on the descriptors of the matching point pairs; The target triplet is determined based on the descriptors of the matching point pairs and the descriptors of the non-matching point pairs.

8. An image feature point matching device based on oil field scenes, characterized in that: include: A first determination module is used to input the two images to be processed into the image feature point extraction network respectively to obtain the feature points of the two images to be processed; A second determination module is used to perform feature point matching on the feature points of the two images to be processed to determine matching point pairs between the two images to be processed; Among them, the image feature point extraction network is constructed based on two large kernel convolution branches, each of the large kernel convolution branches is constructed based on depthwise separable convolution, depthwise separable hole convolution and point convolution, and the image feature point extraction network is trained based on target triples, and the target triples are used to indicate non-matching point pairs between two training images and non-matching point pairs in a single training image.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the image feature point matching method based on the oil field scene as claimed in claim 7 is implemented.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the image feature point matching method based on the oil field scene as claimed in claim 7 is implemented.

Images