Image matching method and device, electronic equipment, storage medium and related product

Abstract

The application discloses an image matching method and device, electronic equipment, storage medium and related products, comprising: receiving two images to be matched; for each image to be matched, extracting the feature structure and the corresponding feature descriptor in the image, wherein the feature structure comprises a feature line and / or a feature shape; determining whether the feature structures of the two images to be matched match according to the feature descriptors of the two images to be matched. By extracting the feature structure including the line and shape type in the image, the feature points manually labeled in the prior art are covered by the line and / or shape, the image is divided into the combination of the line and / or shape feature structure, the corresponding relationship of the image feature structure is obtained, the complex step of manual labeling is avoided, the speed and accuracy of image matching are improved, and the application can be effectively applied to the scene of large data volume image matching.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to an image matching method, apparatus, electronic device, storage medium and related products. Background Technology

[0002] With the development of computer technology, image processing technology has also developed rapidly. Image matching technology is a fundamental task of image processing technology and plays an important role in fields such as image stitching and robot spatial positioning.

[0003] In existing technologies, point matching is mainly used. This type of method is slow and has low accuracy. Moreover, it often requires manual selection of initial matching points, which cannot adapt to scenarios with large amounts of image data. As a result, it has problems with low image matching efficiency and accuracy. Summary of the Invention

[0004] This invention provides an image matching method, apparatus, electronic device, storage medium, and related products to solve the technical problems of low image matching efficiency and accuracy in the prior art.

[0005] According to a first aspect of the present invention, an image matching method is disclosed, the method comprising:

[0006] Receive two images to be matched;

[0007] For each image to be matched, extract the feature structure and corresponding feature descriptor in the image, wherein the feature structure includes: feature lines and / or feature shapes;

[0008] Based on the feature descriptors of the two images to be matched, determine whether the feature structures of the two images to be matched match.

[0009] Optionally, as an embodiment, the extraction of feature structures and corresponding feature descriptors from the image includes:

[0010] The image is input into a pre-trained feature extraction model for processing to obtain the feature structure and corresponding feature descriptors in the image.

[0011] The feature extraction model is trained based on multiple sample images and the feature structures in the sample images.

[0012] Optionally, as an embodiment, the training process of the feature extraction model includes:

[0013] Obtain a training dataset, wherein the training dataset includes: a synthetic image set and a real image set, the synthetic image set contains multiple synthetic sample images containing basic shapes, and the real image set contains multiple real sample images;

[0014] The sample images in the synthetic image set are used as input, and the corresponding feature structures are used as output. The backbone network constructed based on the Magicpoint algorithm and Hough transform function is trained to obtain an intermediate model, wherein the intermediate model is used to extract the feature structures in the image.

[0015] The intermediate model and homography adaptation technique are used to process the sample images in the real image set to obtain the corresponding feature structure.

[0016] For each sample image Pi in the real image set, a homography transformation is performed on Pi to obtain Qi. Pi and Qi are used as inputs, and the feature structures in Pi and Qi are used as outputs. The backbone network constructed based on the SuperPoint algorithm and the Hough transform function is trained to obtain the feature extraction model. The Hough transform function is used to perform the transformation between feature lines, shapes and points during the model training process.

[0017] Alternatively, as an example, when using sample images from the training dataset as input, the sample images are converted to grayscale before being used as input.

[0018] Optionally, as an embodiment, determining whether the feature structures of the two images to be matched match based on their feature descriptors includes:

[0019] The elements in the feature descriptors of the two images to be matched are subtracted bit by bit to obtain N difference values, where N is the number of elements in the feature descriptors;

[0020] The differences are summed to obtain the sum value;

[0021] If the summation value is less than a preset threshold, then the feature structures of the two images to be matched are determined to be matched; otherwise, they are determined not to match.

[0022] Optionally, as an embodiment, the feature structure further includes: corner points and boundary points.

[0023] According to a second aspect of the present invention, an image matching apparatus is disclosed, the apparatus comprising:

[0024] The receiving module is used to receive two images to be matched;

[0025] The extraction module is used to extract the feature structure and corresponding feature descriptor in each image to be matched, wherein the feature structure includes: feature lines and / or feature shapes;

[0026] The determination module is used to determine whether the feature structures of the two images to be matched match based on the feature descriptors of the two images to be matched.

[0027] Optionally, as an embodiment, the extraction module includes:

[0028] The processing submodule is used to input the image into a pre-trained feature extraction model for processing, so as to obtain the feature structure and corresponding feature descriptors in the image;

[0029] The feature extraction model is trained based on multiple sample images and the feature structures in the sample images.

[0030] Optionally, as an embodiment, the apparatus further includes:

[0031] The training module is used to acquire a training dataset, wherein the training dataset includes: a synthetic image set and a real image set, the synthetic image set contains multiple synthetic sample images containing basic shapes, and the real image set contains multiple real sample images;

[0032] The sample images in the synthetic image set are used as input, and the corresponding feature structures are used as output. The backbone network constructed based on the Magicpoint algorithm and Hough transform function is trained to obtain an intermediate model, wherein the intermediate model is used to extract the feature structures in the image.

[0033] The intermediate model and homography adaptation technique are used to process the sample images in the real image set to obtain the corresponding feature structure.

[0034] For each sample image Pi in the real image set, a homography transformation is performed on Pi to obtain Qi. Pi and Qi are used as inputs, and the feature structures in Pi and Qi are used as outputs. The backbone network constructed based on the SuperPoint algorithm and the Hough transform function is trained to obtain the feature extraction model. The Hough transform function is used to perform the transformation between feature lines, shapes and points during the model training process.

[0035] Alternatively, as an example, when using sample images from the training dataset as input, the sample images are converted to grayscale before being used as input.

[0036] Optionally, as an embodiment, the determining module includes:

[0037] The first operation submodule is used to perform bit-by-bit subtraction on the elements in the feature descriptors of the two images to be matched to obtain N difference values, where N is the number of elements in the feature descriptors;

[0038] The second calculation submodule is used to perform a summation operation on the difference to obtain a sum value;

[0039] The determination submodule is used to determine that the feature structures of the two images to be matched match if the summation value is less than a preset threshold, otherwise determine that they do not match.

[0040] Optionally, as an embodiment, the feature structure further includes: corner points and boundary points.

[0041] According to a third aspect of the present invention, an electronic device is disclosed, comprising: a memory, a processor, and a program stored in the memory and executable on the processor, wherein the program, when executed by the processor, implements the steps of the image matching method as described in the first aspect.

[0042] According to a fourth aspect of the present invention, a computer-readable storage medium is disclosed, on which a program is stored, which, when executed by the processor, implements the steps of the image matching method as described in the first aspect.

[0043] According to a fifth aspect of the present invention, a computer program product is disclosed that, when instructions in the computer program product are executed by a processor of an electronic device, enables the electronic device to perform the steps of the image matching method as described in the first aspect.

[0044] In this embodiment of the invention, by extracting feature structures including lines and shapes from the image, feature points that require manual annotation in the prior art are covered by lines and / or shapes. The image is split into a combination of feature structures of lines and / or shapes to obtain the correspondence of image feature structures. This avoids the complicated steps of manual annotation, improves the speed and accuracy of image matching, and can be effectively applied to scenarios with large amounts of image matching. Attached Figure Description

[0045] Figure 1 This is a flowchart of an image matching method according to an embodiment of the present invention;

[0046] Figure 2 This is a flowchart of a model training method according to an embodiment of the present invention;

[0047] Figure 3 This is a schematic diagram of the structure of an image matching device according to an embodiment of the present invention. Detailed Implementation

[0048] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0049] It should be noted that, for the sake of simplicity, the method embodiments are all described as a series of actions. However, those skilled in the art should understand that the embodiments of the present invention are not limited to the described order of actions, because according to the embodiments of the present invention, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions involved are not necessarily essential to the embodiments of the present invention.

[0050] With the development of computer technology, image processing technology has also advanced rapidly. Image matching technology is a fundamental task in image processing, playing a crucial role in areas such as image stitching and robot spatial localization. Taking image stitching as an example, image stitching technology combines several images with overlapping parts (possibly obtained at different times, from different perspectives, or by different sensors) into a seamless panoramic or high-resolution image. There are many methods for image stitching, and different algorithms may have slightly different steps, but the general process is the same. Generally speaking, image stitching mainly includes the following steps:

[0051] Image preprocessing includes basic digital image processing operations (such as denoising, edge extraction, histogram processing, etc.), establishing matching templates for images, and performing certain transformations on images (such as Fourier transform, wavelet transform, etc.).

[0052] Image matching involves using a specific matching strategy to find the corresponding positions of templates or feature points in the image to be stitched in the reference image, thereby determining the transformation relationship between the two images.

[0053] Establish a transformation model: Based on the correspondence between templates or image features, calculate the parameter values ​​in the mathematical model to establish a mathematical transformation model for the two images;

[0054] Unified coordinate transformation: Based on the established mathematical transformation model, the image to be stitched is transformed into the coordinate system of the reference image, thus completing the unified coordinate transformation;

[0055] Image fusion: The overlapping areas of the images to be stitched together are merged to obtain a smooth and seamless panoramic image.

[0056] Image matching and image fusion are two key technologies in image stitching, with image matching being the foundation of image fusion. Image matching algorithms generally involve a very large amount of computation, so the development of image stitching technology largely depends on the innovation of image matching technology.

[0057] Existing image matching techniques first utilize traditional algorithms such as SIFT and ORB to extract points (e.g., corner points, boundary points) and feature descriptors from the image, then employ point matching methods for matching. These methods are slow, have low accuracy, and often require manual selection of initial matching points, making them unsuitable for fusion of large datasets. Furthermore, traditional feature extraction algorithms cannot adjust the number and types of points extracted, and are susceptible to noise, affecting the accuracy of feature descriptors and the location of extracted features. Therefore, with the development of stitching technology, more feature structures are needed, especially those distributed across various regions, even in areas lacking texture features.

[0058] To address the aforementioned technical problems, embodiments of the present invention provide an image matching method, apparatus, electronic device, storage medium, and related products.

[0059] The following section will first introduce an image matching method provided by an embodiment of the present invention.

[0060] It should be noted that the method provided in this embodiment of the invention is applicable to electronic devices. In practical applications, the electronic device may be a server, and this embodiment of the invention does not limit it.

[0061] Figure 1 This is a flowchart of an image matching method according to an embodiment of the present invention, as follows: Figure 1 As shown, the method may include the following steps: step 101, step 102, and step 103, wherein,

[0062] In step 101, two images to be matched are received.

[0063] In step 102, for each image to be matched, the feature structure and corresponding feature descriptor in the image are extracted, wherein the feature structure includes: feature lines and / or feature shapes.

[0064] In this embodiment of the invention, the feature descriptor is a feature vector, for example, a 512-bit vector.

[0065] Considering that in some scenarios, it is impossible to extract feature structures such as corner points and boundary points, such as when there are a large number of walls or a simple background, existing extraction algorithms have difficulty extracting them and thus cannot achieve image matching. In this case, the embodiments of the present invention can extract line structures and shape structures (such as geometric shapes). Based on the line structure and shape structure, the inability of traditional algorithms to extract matching points is avoided. By splitting the image into a combination of feature structures, the positional correspondence of the image can be obtained for subsequent image matching.

[0066] In this embodiment of the invention, the feature structure may further include: corner points and boundary points.

[0067] As can be seen, in this embodiment of the invention, various types of feature structures can be extracted, including not only "points" but also "lines" and "shapes", in order to increase the variety of feature structures extracted and improve efficiency and accuracy in subsequent image matching.

[0068] In this embodiment of the invention, a network model (i.e., a feature extraction model) for extracting points, lines, and shapes can be pre-trained. In actual use, two images are input into the network model respectively, and the feature structures and feature descriptors in the two images are output. Image matching is then performed based on the feature descriptors.

[0069] Accordingly, in this embodiment of the invention, when extracting the feature structure and corresponding feature descriptor in an image, the image can be input into a pre-trained feature extraction model for processing to obtain the feature structure and corresponding feature descriptor in the image; wherein, the feature extraction model is trained based on multiple sample images and the feature structure in the sample images.

[0070] In this embodiment of the invention, the feature extraction model can be a model trained based on a neural network. Compared with traditional algorithms, the neural network method has higher extraction accuracy, stronger noise resistance, and can reduce the redundancy of a large number of feature points in close proximity (for example, for moving objects in an image, using only point structures will result in motion blur, while using shape structures, such as quadrilaterals surrounding the moving object, will be more stable); it can be computed in parallel, improving the computation speed; since the feature structures in the image can be defined manually, different types of feature structures can be extracted by modifying the position of the feature structures in the sample image; the number of feature structures, the type of feature structures, and the dimension of the feature descriptors can all be adjusted according to user needs.

[0071] In step 103, the feature structures of the two images to be matched are determined based on their feature descriptors.

[0072] In this embodiment of the invention, the elements in the feature descriptors of two images to be matched can be subtracted bit by bit to obtain N difference values, where N is the number of elements in the feature descriptors; the difference values ​​are summed to obtain a sum value; if the sum value is less than a preset threshold, the feature structures of the two images to be matched are determined to match, otherwise they are determined not to match.

[0073] As can be seen from the above embodiments, in this embodiment, by extracting feature structures including lines and shapes from the image, the feature points that require manual annotation in the prior art are covered by lines and / or shapes. The image is split into a combination of feature structures of lines and / or shapes to obtain the correspondence of image feature structures. This avoids the complicated steps of manual annotation, improves the speed and accuracy of image matching, and can be effectively applied to scenarios with large amounts of image matching.

[0074] Figure 2 This is a flowchart of a model training method according to an embodiment of the present invention, used to train a feature extraction model, such as... Figure 2 As shown, the method may include the following steps: step 201, step 202, step 203, and step 204, wherein,

[0075] In step 201, a training dataset is obtained, which includes a synthetic image set and a real image set. The synthetic image set contains multiple synthetic sample images containing basic shapes, and the real image set contains multiple real sample images.

[0076] In this embodiment of the invention, for sample images in the synthetic image set, when generating sample images, various basic shapes and some background images can be artificially generated, with the basic shapes used as the foreground of the image and the background images used as the background of the image.

[0077] In this embodiment of the invention, the real image set can be the MS-COCO dataset.

[0078] In step 202, the sample images in the synthetic image set are used as input and the corresponding feature structures are used as output to train the backbone network constructed based on the Magicpoint algorithm and the Hough transform function to obtain an intermediate model, which is used to extract the feature structures in the image.

[0079] In this embodiment of the invention, when training the intermediate model, the Magicpoint algorithm and the Hough transform function can be used to construct the backbone network. A synthetic image set is used as the training set to train the feature structure extraction network. The Hough transform function is used to perform the transformation between feature lines, shapes and points during the model training process. The network structure of Magicpoint includes an encoder and a decoder.

[0080] In this embodiment of the invention, the reason for using a synthetic image set is that the true values ​​of the feature structures can be obtained from the synthetic sample images, because training Magicpoint requires the true values ​​of the feature structures. It is difficult to determine the location of feature structures for any image, but if the image only contains basic shape elements such as line segments, triangles, rectangles, and cubes, then the location of the feature structures is generally at the endpoints and fixed points, and the true values ​​are easy to determine.

[0081] In step 203, the sample images in the real image set are processed using an intermediate model and homography adaptation technique to obtain the corresponding feature structure.

[0082] In this embodiment of the invention, for real scene images in the real image set, the intermediate model trained in step 202 + Homographic Adaptation is used to extract the ground truth value of the feature structure of the real scene image.

[0083] Specifically, since the intermediate model is only trained on images of basic shape elements, its performance for general image extraction may not be very good. Therefore, it is necessary to use a large number of images from the real image set to further train a general model. However, since the real image set does not have ground truth values ​​for feature structures, it is necessary to obtain the ground truth values ​​of feature structures from the real image set through the previously trained intermediate model.

[0084] In this embodiment of the invention, when obtaining the ground truth of the feature structure of the real image set through the previously trained intermediate model, n homography transformations (e.g., 100) are performed on each image to obtain n transformed images. The intermediate model is used to extract the feature structure on these images respectively to obtain n feature structure heatmaps. These n heatmaps are accumulated together to obtain the final heatmap. Then, a threshold is used to truncate and obtain the feature structure at each position. This is the ground truth of the feature structure of the original image.

[0085] In step 204, for each sample image Pi in the real image set, a homography transformation is performed on Pi to obtain Qi. Pi and Qi are used as inputs, and the feature structures in Pi and Qi are used as outputs. The backbone network built based on the SuperPoint algorithm and the Hough transform function is trained to obtain the feature extraction model. The Hough transform function is used to perform the transformation between feature lines, shapes and points during the model training process.

[0086] In this embodiment of the invention, when training the feature extraction model, the SuperPoint algorithm and the Hough transform function can be used to construct the initial network model. A real image set is used as the training set for model training. The main structure of SuperPoint includes an encoding network and two decoding networks. The two decoding networks share an encoding network. One decoding network is used to generate the feature structure, and the other decoding network is used to generate the corresponding feature descriptor.

[0087] In this embodiment of the invention, for a real image set, the ground truth of the feature structure in the image has already been obtained through step 203. However, like the feature structure, the feature descriptor also has the problem of ground truth. For any two images, the ground truth cannot be determined or labeled. The input of Superpoint is two images, one of which is obtained by homography transformation of the other image. The reason for using two images as input is related to the output of Superpoint. Similar to the traditional SIFT algorithm, the output of Superpoint is not only the feature structure but also the descriptor of the feature structure.

[0088] As shown above, feature structures have ground truth values, while descriptors do not. Therefore, we use two homography-transformed images as input (the point matching is known), and the matching relationship between the feature structures of these two images is used to constrain the descriptor of the feature structure. That is, we transform the real scene image to obtain another image, and the correspondence between the feature structures of the two images is also known, determined by the transformation function. For any two pairs of points in the two images, we calculate the loss to optimize so that the distance between matching points is small and the distance between non-matching points is large. In this way, the final descriptor meets the requirements.

[0089] In this embodiment of the invention, considering that grayscale images have an advantage in processing speed, when using sample images in the training dataset as input, the sample images can be converted into grayscale images before being used as input.

[0090] As can be seen, in this embodiment of the invention, as long as a reasonable feature structure is specified during training, the network can learn the rules of such feature structures and achieve the extraction of more types of feature structures during testing. For example, training the network to stably extract body boundaries as feature structures can help the subsequent stitching process bypass moving objects such as pedestrians and construct more reasonable stitches.

[0091] Figure 3 This is a schematic diagram of the structure of an image matching device according to an embodiment of the present invention, as shown below. Figure 3 As shown, the image matching device 300 may include: a receiving module 301, an extraction module 302, and a determining module 303, wherein,

[0092] Receiver module 301 is used to receive two images to be matched;

[0093] The extraction module 302 is used to extract the feature structure and corresponding feature descriptor in each image to be matched, wherein the feature structure includes: feature lines and / or feature shapes;

[0094] The determining module 303 is used to determine whether the feature structures of the two images to be matched match based on the feature descriptors of the two images to be matched.

[0095] As can be seen from the above embodiments, in this embodiment, by extracting feature structures including lines and shapes from the image, the feature points that require manual annotation in the prior art are covered by lines and / or shapes. The image is split into a combination of feature structures of lines and / or shapes to obtain the correspondence of image feature structures. This avoids the complicated steps of manual annotation, improves the speed and accuracy of image matching, and can be effectively applied to scenarios with large amounts of image matching.

[0096] Optionally, as an embodiment, the extraction module 302 may include:

[0097] The processing submodule is used to input the image into a pre-trained feature extraction model for processing to obtain the feature structure and corresponding feature descriptor in the image; wherein, the feature extraction model is trained based on multiple sample images and the feature structure in the sample images.

[0098] Optionally, as an embodiment, the image matching device 300 may further include:

[0099] The training module is used to acquire a training dataset, wherein the training dataset includes: a synthetic image set and a real image set, the synthetic image set contains multiple synthetic sample images containing basic shapes, and the real image set contains multiple real sample images;

[0100] The sample images in the synthetic image set are used as input, and the corresponding feature structures are used as output. The backbone network constructed based on the Magicpoint algorithm and Hough transform function is trained to obtain an intermediate model, wherein the intermediate model is used to extract the feature structures in the image.

[0101] The intermediate model and homography adaptation technique are used to process the sample images in the real image set to obtain the corresponding feature structure.

[0102] For each sample image Pi in the real image set, a homography transformation is performed on Pi to obtain Qi. Pi and Qi are used as inputs, and the feature structures in Pi and Qi are used as outputs. The backbone network constructed based on the SuperPoint algorithm and the Hough transform function is trained to obtain the feature extraction model. The Hough transform function is used to perform the transformation between feature lines, shapes and points during the model training process.

[0103] Alternatively, as an example, when using sample images from the training dataset as input, the sample images can be converted into grayscale images before being used as input.

[0104] Optionally, as an embodiment, the determining module 303 may include:

[0105] The first operation submodule is used to perform bit-by-bit subtraction on the elements in the feature descriptors of the two images to be matched to obtain N difference values, where N is the number of elements in the feature descriptors;

[0106] The second calculation submodule is used to perform a summation operation on the difference to obtain a sum value;

[0107] The determination submodule is used to determine that the feature structures of the two images to be matched match if the summation value is less than a preset threshold, otherwise determine that they do not match.

[0108] Optionally, as an embodiment, the feature structure further includes: corner points and boundary points.

[0109] As the device embodiment is basically similar to the method embodiment, the description is relatively simple, and relevant parts can be found in the description of the method embodiment.

[0110] According to another embodiment of the present invention, the present invention also provides an electronic device, including: a memory, a processor, and a program stored in the memory and executable on the processor, wherein the program, when executed by the processor, implements the steps in the image matching method as described in any of the above embodiments.

[0111] According to another embodiment of the present invention, the present invention also provides a computer-readable storage medium storing a program that, when executed by a processor, implements the steps of the image matching method as described in any of the above embodiments.

[0112] According to another embodiment of the present invention, the present invention also provides a computer program product, wherein when the instructions in the computer program product are executed by a processor of an electronic device, the electronic device is enabled to perform the steps in the image matching method as described in any of the above embodiments.

[0113] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0114] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, apparatus, or computer program products. Therefore, embodiments of the present invention can take the form of entirely hardware embodiments, entirely software embodiments, or embodiments combining software and hardware aspects. Furthermore, embodiments of the present invention can take the form of computer program products implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0115] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, terminal devices (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing terminal device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal device, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0116] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing terminal device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0117] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal equipment, causing a series of operational steps to be performed on the computer or other programmable terminal equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable terminal equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0118] Although preferred embodiments of the present invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of the present invention.

[0119] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal device. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or terminal device that includes said element.

[0120] The above provides a detailed description of the image matching method, apparatus, electronic device, storage medium, and related products provided by the present invention. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. An image matching method, characterized in that, The method includes: Receive two images to be matched; For each image to be matched, the feature structure and corresponding feature descriptor in the image are extracted, wherein the feature structure includes: feature lines and feature shapes; Based on the feature descriptors of the two images to be matched, determine whether the feature structures of the two images to be matched match. The extraction of feature structures and corresponding feature descriptors from the image includes: The image is input into a pre-trained feature extraction model for processing to obtain the feature structure and corresponding feature descriptors in the image. The feature extraction model is trained based on multiple sample images and the feature structures in the sample images; The training process of the feature extraction model includes: Obtain a training dataset, wherein the training dataset includes: a synthetic image set and a real image set, the synthetic image set contains multiple synthetic sample images containing basic shapes, and the real image set contains multiple real sample images; The sample images in the synthetic image set are used as input, and the corresponding feature structures are used as output. The backbone network constructed based on the Magicpoint algorithm and Hough transform function is trained to obtain an intermediate model, wherein the intermediate model is used to extract the feature structures in the image. The intermediate model and homography adaptation technique are used to process the sample images in the real image set to obtain the corresponding feature structure. For each sample image Pi in the real image set, a homography transformation is performed on Pi to obtain Qi. Pi and Qi are used as inputs, and the feature structures in Pi and Qi are used as outputs. The backbone network constructed based on the SuperPoint algorithm and the Hough transform function is trained to obtain the feature extraction model. The Hough transform function is used to perform the transformation between feature lines, shapes and points during the model training process.

2. The method according to claim 1, characterized in that, When using sample images from the training dataset as input, the sample images are converted to grayscale before being used as input.

3. The method according to claim 1, characterized in that, The step of determining whether the feature structures of the two images to be matched match based on their feature descriptors includes: The elements in the feature descriptors of the two images to be matched are subtracted bit by bit to obtain N difference values, where N is the number of elements in the feature descriptors; The differences are summed to obtain the sum value; If the summation value is less than a preset threshold, then the feature structures of the two images to be matched are determined to be matched; otherwise, they are determined not to match.

4. The method according to any one of claims 1 to 3, characterized in that, The feature structure also includes: corner points and boundary points.

5. An image matching device, characterized in that, The device includes: The receiving module is used to receive two images to be matched; The extraction module is used to extract the feature structure and corresponding feature descriptor in each image to be matched, wherein the feature structure includes: feature lines and feature shapes; The determining module is used to determine whether the feature structures of the two images to be matched match based on the feature descriptors of the two images to be matched. The extraction module includes: The processing submodule is used to input the image into a pre-trained feature extraction model for processing, to obtain the feature structure and corresponding feature descriptors in the image; wherein, the feature extraction model is trained based on multiple sample images and the feature structure in the sample images; The device further includes: The training module is used to acquire a training dataset, wherein the training dataset includes: a synthetic image set and a real image set, the synthetic image set contains multiple synthetic sample images containing basic shapes, and the real image set contains multiple real sample images; The sample images in the synthetic image set are used as input, and the corresponding feature structures are used as output. The backbone network constructed based on the Magicpoint algorithm and Hough transform function is trained to obtain an intermediate model, wherein the intermediate model is used to extract the feature structures in the image. The intermediate model and homography adaptation technique are used to process the sample images in the real image set to obtain the corresponding feature structure. For each sample image Pi in the real image set, a homography transformation is performed on Pi to obtain Qi. Pi and Qi are used as inputs, and the feature structures in Pi and Qi are used as outputs. The backbone network constructed based on the SuperPoint algorithm and the Hough transform function is trained to obtain the feature extraction model. The Hough transform function is used to perform the transformation between feature lines, shapes and points during the model training process.

6. An electronic device, characterized in that, include: A memory, a processor, and a program stored in the memory and executable on the processor, wherein the program, when executed by the processor, implements the steps of the image matching method as described in any one of claims 1 to 4.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a program that, when executed by a processor, implements the steps of the image matching method as described in any one of claims 1 to 4.

8. A computer program product, characterized in that, When the instructions in the computer program product are executed by the processor of the electronic device, the electronic device is enabled to perform the steps of the image matching method as described in any one of claims 1 to 4.

Images