Feature point matching method and device, terminal equipment and computer program product

Abstract

The invention is suitable for the technical field of image processing, and provides a feature point matching method and device, terminal equipment and a computer program product, and the method comprises the steps: carrying out the feature point matching of each to-be-matched image in an image set through a conventional algorithm, and obtaining a first matching result; processing the feature points in each to-be-matched image according to the first matching result to obtain a target image corresponding to each to-be-matched image; the number of the feature points in the target image is smaller than that of the feature points in the corresponding to-be-matched image; and performing feature point matching on each target image through a deep learning algorithm to obtain a target matching result. According to the method, the image set can be rapidly and preliminarily matched by using a traditional algorithm, the time is shortened, and the matching efficiency is improved. And subsequently, the target image with the reduced number of feature points is matched by using the deep learning algorithm, so that compared with the mode of directly using the deep learning algorithm on the image set, the calculation amount is further reduced, and the matching success rate is also improved.

Description

Technical Field

[0001] This application belongs to the technical field of image processing, and particularly relates to a feature point matching method, apparatus, terminal device, and computer program product. Background Art

[0002] In the process of image processing, feature matching of images is an important task in computer vision, mainly used to establish the corresponding relationship between images by means of feature points in different images. Existing feature point matching methods include using traditional algorithms for feature point matching and using deep learning algorithms for feature point matching. Among them, traditional algorithms specifically refer to solutions designed based on explicit rules and mathematical formulas, and algorithms that solve problems through step-by-step deterministic logic (such as brute-force matching algorithms).

[0003] However, in the above methods, traditional algorithms are prone to failing to match a large number of feature points in the image, thus unable to meet actual needs and reducing the matching success rate; deep learning algorithms take a long time, reducing the matching efficiency. Summary of the Invention

[0004] Embodiments of this application provide a feature point matching method, apparatus, terminal device, and computer program product to solve the problems in the prior art of being unable to meet actual needs, reducing the matching success rate, taking a long time, and reducing the matching efficiency.

[0005] In a first aspect, embodiments of this application provide a feature point matching method, including:

[0006] Performing feature point matching on each to-be-matched image in the image set through a traditional algorithm to obtain a first matching result;

[0007] Processing the feature points in each to-be-matched image according to the first matching result to obtain a target image corresponding to each to-be-matched image; the number of feature points in the target image is less than the number of feature points in the corresponding to-be-matched image;

[0008] Performing feature point matching on each target image through a deep learning algorithm to obtain a target matching result.

[0009] Optionally, before performing feature point matching on each to-be-matched image in the image set through a traditional algorithm to obtain a first matching result, it further includes:

[0010] Determining feature point matching information; the feature point matching information is used to describe the matching accuracy requirement;

[0011] If the feature point matching information is the first information, perform feature point screening on each original image in the image set to obtain each image to be matched; the number of feature points in each image to be matched is less than the number of feature points in the corresponding original image; the first information is used to describe a low matching accuracy requirement.

[0012] Optionally, determining the feature point matching information includes:

[0013] Obtain the task information corresponding to the image set; the task information is used to describe the purpose of performing image processing on the image set;

[0014] Analyze the task information to obtain the feature point matching information.

[0015] Optionally, analyzing the task information to obtain the feature point matching information includes:

[0016] Obtain the historical feature point information corresponding to the task information;

[0017] Input the task information into a trained analysis model for processing to obtain initial feature point information;

[0018] Combine the historical feature point information and the initial feature point information to obtain the feature point matching information.

[0019] Optionally, performing feature point matching on each image to be matched in the image set through a traditional algorithm to obtain a first matching result includes:

[0020] Perform feature point matching on each image to be matched in the image set through the traditional algorithm to obtain an initial matching result; the initial matching result includes the initial sub-results of each feature point in each image to be matched;

[0021] Determine the similarity of the alternative feature point groups; the alternative feature point groups refer to the feature point groups in each image to be matched where the initial sub-result is a successful match;

[0022] If the similarity is less than a set threshold, adjust the initial sub-result of the alternative feature point group to a failed match;

[0023] Obtain the first matching result according to the adjusted initial sub-result and the initial sub-results of the remaining feature points except the alternative feature point group.

[0024] Optionally, the first matching result includes the matching sub-results of each feature point in each image to be matched; processing the feature points in each image to be matched according to the first matching result to obtain the target image corresponding to each image to be matched includes:

[0025] For any image to be matched, delete the target feature points in the any image to be matched to obtain the target image corresponding to the any image to be matched; the target feature points refer to the feature points with a successful matching sub-result.

[0026] Optionally, before performing feature point matching on each of the target images through a deep learning algorithm to obtain a target matching result, it further includes:

[0027] Determine feature point matching information; the feature point matching information is used to describe the matching accuracy requirement;

[0028] If the feature point matching information is the first information, perform feature point screening on each of the target images to obtain each processed target image; the number of feature points in each processed target image is less than the number of feature points in the corresponding target image before screening; the first information is used to describe a low matching accuracy requirement.

[0029] Correspondingly, performing feature point matching on each of the target images through a deep learning algorithm to obtain a target matching result includes:

[0030] Perform feature point matching on each of the processed target images through the deep learning algorithm to obtain the target matching result.

[0031] In a second aspect, an embodiment of the present application provides a feature point matching device, including:

[0032] A first matching unit, configured to perform feature point matching on each image to be matched in an image set through a traditional algorithm to obtain a first matching result;

[0033] A first processing unit, configured to process the feature points in each image to be matched according to the first matching result to obtain the target image corresponding to each image to be matched; the number of feature points in the target image is less than the number of feature points in the corresponding image to be matched;

[0034] A second matching unit, configured to perform feature point matching on each of the target images through a deep learning algorithm to obtain a target matching result.

[0035] In a third aspect, an embodiment of the present application provides a terminal device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, where when the processor executes the computer program, the feature point matching method described in any one of the first aspects above is implemented.

[0036] Fourthly, an embodiment of the present application provides a computer-readable storage medium storing a computer program, which when executed by a processor, implements the feature point matching method as described in any one of the above first aspects.

[0037] Fifthly, an embodiment of the present application provides a computer program product, which when running on a terminal device, enables the terminal device to execute the feature point matching method as described in any one of the above first aspects.

[0038] The beneficial effects of the embodiments of the present application compared with the prior art are as follows:

[0039] A feature point matching method provided by an embodiment of the present application performs feature point matching on each image to be matched in an image set through a traditional algorithm to obtain a first matching result. Since the traditional algorithm usually has a relatively fixed calculation process and a low calculation complexity, the present application can quickly perform preliminary matching on a large number of feature points through the traditional algorithm, that is, process each image to be matched in the image set in a short time, greatly reducing the overall calculation amount. Then, the feature points in each image to be matched are processed according to the first matching result to obtain a target image corresponding to each image to be matched; the number of feature points in the target image is less than the number of feature points in the corresponding image to be matched; the feature point matching is performed on each target image through a deep learning algorithm to obtain a target matching result. Since the deep learning algorithm usually requires a large amount of computing resources to process complex models and large-scale data, the present application first uses a traditional algorithm to perform preliminary processing on the image, reducing the number of feature points entering the deep learning algorithm, that is, reducing the input scale of the deep learning model, so that when the deep learning algorithm is subsequently run, the required resources such as memory and computing power are correspondingly reduced, so that a relatively efficient feature point matching task can be realized even in the case of limited resources (such as mobile devices or embedded systems), broadening the application scenarios and also improving the matching success rate. At the same time, the subsequent use of the deep learning algorithm to match the target image with the reduced number of feature points further reduces the calculation amount compared with directly using the deep learning algorithm for the image set, thus significantly improving the efficiency of the entire feature point matching process. Description of the Drawings

[0040] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings required for use in the embodiments or the description of the prior art. Obviously, the following drawings are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained according to these drawings without creative efforts.

[0041] Figure 1It is a flowchart of the implementation of the feature point matching method provided by an embodiment of the present application;

[0042] Figure 2 It is a flowchart of the implementation of the feature point matching method provided by another embodiment of the present application;

[0043] Figure 3 It is a flowchart of the implementation of the feature point matching method provided by still another embodiment of the present application;

[0044] Figure 4 It is a flowchart of the implementation of the feature point matching method provided by yet another embodiment of the present application;

[0045] Figure 5 It is a schematic structural diagram of the feature point matching device provided by an embodiment of the present application;

[0046] Figure 6 It is a schematic structural diagram of the terminal device provided by an embodiment of the present application. Detailed implementation manners

[0047] In the following description, specific details such as specific system structures and technologies are presented for the purpose of illustration rather than limitation, so as to thoroughly understand the embodiments of the present application. However, those skilled in the art should clearly understand that the present application can also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid unnecessary details from interfering with the description of the present application.

[0048] It should be understood that when used in the specification and appended claims of the present application, the term "comprising" indicates the presence of the described features, wholes, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components, and / or their combinations.

[0049] It should also be understood that the term "and / or" as used in the specification and appended claims of the present application refers to any combination and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0050] As used in the specification and appended claims of the present application, the term "if" can be interpreted as "when...", "once", "in response to a determination", or "in response to a detection" according to the context. Similarly, the phrase "if a determination is made" or "if [the described condition or event] is detected" can be interpreted as meaning "once a determination is made", "in response to a determination", "once [the described condition or event] is detected", or "in response to a detection of [the described condition or event]" according to the context.

[0051] In addition, in the description of the specification and the appended claims of the present application, the terms "first", "second", "third", etc. are only used for distinguishing descriptions and should not be construed as indicating or implying relative importance.

[0052] Reference to "one embodiment" or "some embodiments" etc. described in the specification of the present application means that a specific feature, structure, or characteristic described in connection with that embodiment is included in one or more embodiments of the present application. Thus, statements such as "in one embodiment", "in some embodiments", "in other some embodiments", "in still other embodiments", etc. that appear in different places in this specification do not necessarily all refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "comprising", "including", "having" and their variants all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0053] Please refer to Figure 1 , Figure 1 which is a flowchart of the implementation of the feature point matching method provided by an embodiment of the present application. In the embodiment of the present application, the execution subject of the feature point matching method is a terminal device. Among them, the terminal device includes but is not limited to devices such as notebooks, desktop computers, and computers.

[0054] As Figure 1 shown, the feature point matching method provided by an embodiment of the present application may include S101 to S103, which are described in detail as follows:

[0055] In S101, feature point matching is performed on each image to be matched in the image set through a traditional algorithm to obtain a first matching result.

[0056] In practical applications, when a user needs to perform related operations (such as image registration, target recognition and tracking, image stitching, etc.) using certain images, it is usually necessary to perform feature point matching on these images to implement subsequent related operations on these images.

[0057] In the embodiment of the present application, the terminal device can obtain in real time an image set that needs to perform feature point matching through a server wirelessly communicatively connected thereto. Among them, the server includes but is not limited to devices such as desktop computers and computers.

[0058] It should be noted that the above image set includes at least two images to be matched that need to perform feature point matching.

[0059] In the embodiment of the present application, after obtaining the above image set, the terminal device can use a feature point extraction algorithm to perform feature extraction on each image to be matched in the image set to obtain multiple feature points corresponding to each image to be matched.

[0060] In practical applications, feature point extraction algorithms include, but are not limited to, algorithms such as Scale Invariant Feature Transform (SIFT) and Speeded-Up Robust Features (SURF).

[0061] It should be noted that each feature point extracted by the above feature point extraction algorithm carries the coordinate position of the feature point in the corresponding image.

[0062] After that, for each of the above feature points, the terminal device can generate a corresponding feature descriptor according to the image information in its neighborhood. Among them, the feature descriptor is a vector that contains the local feature information of the feature point and is used for subsequent matching operations.

[0063] It should be noted that different feature point extraction algorithms usually have corresponding feature descriptor generation methods. Exemplarily, the SIFT feature descriptor is obtained by calculating the gradient information in the neighborhood of the feature point and organizing it into a 128-dimensional vector.

[0064] In the embodiments of the present application, after the terminal device obtains multiple feature points corresponding to each to-be-matched image in the image set, since there are usually a large number of feature points in each to-be-matched image, in order to improve the matching efficiency, the terminal device can use a traditional algorithm to perform preliminary feature point matching on the multiple feature points corresponding to each to-be-matched image in the image set to obtain a first matching result. Among them, the first matching result includes the matching results of the multiple feature points corresponding to each to-be-matched image in the image set.

[0065] The matching results of the above respective feature points include, but are not limited to, matching success and matching failure.

[0066] It should be noted that traditional algorithms are solutions designed based on explicit rules and mathematical formulas, and solve problems through step-by-step deterministic logic. Traditional algorithms do not rely on data training, but directly process the input through manually designed rules.

[0067] In practical applications, traditional algorithms include, but are not limited to: Brute-Force Matcher (BFM) algorithm, Fast Library for Approximate Nearest Neighbors (FLANN) algorithm, Ratio Test algorithm, Random Sample Consensus (RANSAC) algorithm, and Epipolar Constraint algorithm, etc.

[0068] In one embodiment of the present application, in order to improve the matching accuracy of feature point matching by traditional algorithms, the terminal device may specifically implement step S101 through steps S201 to S204 as shown below, which are described in detail as follows: Figure 2 In S201, perform feature point matching on each image to be matched in the image set through the traditional algorithm to obtain an initial matching result; the initial matching result includes initial sub-results of each feature point in each image to be matched.

[0069] In S202, determine the similarity of the alternative feature point groups; the alternative feature point groups refer to the groups of feature points in each image to be matched where the initial sub-results are successful matches.

[0070] In S203, if the similarity is less than the set threshold, adjust the initial sub-results of the alternative feature point groups to failed matches.

[0071] In S204, obtain the first matching result according to the adjusted initial sub-results and the initial sub-results of the remaining feature points except the alternative feature point groups.

[0072] It should be noted that the initial sub-results of each feature point in each image to be matched include, but are not limited to, failed matches and successful matches.

[0073] In this embodiment, after the terminal device obtains the initial matching results of each image to be matched, it can extract alternative feature points with successful initial sub-results from multiple feature points of each image to be matched according to each initial matching result, and form alternative feature point groups with the mutually matching alternative feature points.

[0074] Since feature point matching specifically refers to calculating the similarity between the feature descriptors of two feature points, and when the similarity is greater than the preset value, it can be determined that the two feature points are successfully matched. Therefore, in this embodiment, the initial sub-results of the above-mentioned feature points also include the similarity calculated when performing feature point matching for each feature point.

[0075] Since in the process of image feature point matching, traditional algorithms may produce incorrect matches due to factors such as image noise, illumination changes, and perspective differences, or for images of some similar scenes, there may be a large number of feature point pairs that seemingly match but are actually inaccurate. Therefore, the terminal device can compare the similarity in the initial sub-results of the feature points in the above-mentioned alternative feature point groups with the set threshold. The set threshold can be set according to actual needs and is not limited here.

[0076]

[0077] ​It should be noted that the above set threshold is greater than the preset value set during feature point matching.

[0078] In this embodiment, when the terminal device detects that the similarity of the above alternative feature point group is less than the set threshold, it indicates that the initial sub-results of the feature points in the alternative feature point group are incorrect, that is, the feature points in the alternative feature point group are mis-matched. Therefore, the terminal device can adjust the initial sub-result of the alternative feature point group to a matching failure.

[0079] After that, the terminal device can combine the above adjusted initial sub-result and the initial sub-results of the remaining feature points other than the alternative feature point group to obtain the final first matching result.

[0080] Exemplarily, assuming that the adjusted initial sub-result is X1 and the initial sub-results of the remaining feature points are X2, the first matching result obtained by combination can be [X1, X2].

[0081] In an embodiment of the present application, when the terminal device detects that the similarity of the above alternative feature point group is greater than the set threshold, it indicates that the initial sub-results of the feature points in the alternative feature point group are accurate. Therefore, the terminal device does not need to make any adjustment to the initial sub-result of the alternative feature point group.

[0082] Through the above implementation manner, since in the process of image feature point matching, traditional algorithms may produce mis-matches due to factors such as image noise, illumination changes, and perspective differences, the terminal device can determine the similarity of the alternative feature point group and adjust the initial sub-results of the feature point groups with similarity less than the set threshold to matching failures, which can effectively filter out these mis-matched points, thereby improving the accuracy of the final matching result. At the same time, for images of some similar scenes, there may be a large number of feature point pairs that seem to match but are actually inaccurate. Therefore, by adjusting the initial sub-results through the above similarity, the terminal device can make the matching result more stable, reduce the incorrect matching caused by accidental factors, and make the matching result maintain high reliability under different image conditions.

[0083] In S102, the feature points in each image to be matched are processed according to the first matching result to obtain the target image corresponding to each image to be matched; the number of feature points in the target image is less than the number of feature points in the corresponding image to be matched.

[0084] In practical applications, in order to implement subsequent operations on the image set, it is necessary to successfully match as many feature points of each image to be matched in the image set as possible, so as to reduce the number of feature points that need to be re-matched later. Therefore, in the embodiments of the present application, after the terminal device obtains the first matching result, the terminal device can process the feature points in each image to be matched according to the first matching result, so as to reduce the number of feature points that need to be re-matched in each image to be matched, thereby obtaining the target image corresponding to each image to be matched. Among them, the number of feature points in the target image is less than the number of feature points in the corresponding image to be matched.

[0085] In an embodiment of the present application, the first matching result may include the matching sub-results of each feature point in each image to be matched. Among them, each matching sub-result includes, but is not limited to, matching success and matching failure.

[0086] It should be noted that, in order to reduce the workload of subsequent feature point matching and improve the matching efficiency, after feature point matching is performed through a traditional algorithm, the feature points that are successfully matched do not need to be re-matched. Therefore, in this embodiment, for any image to be matched, the terminal device can delete the target feature points in any image to be matched to obtain the target image corresponding to the image to be matched. Among them, the target feature points refer to the feature points whose matching sub-results are matching success.

[0087] In some possible embodiments, in order to improve the success rate of subsequent operations on the image set, before the terminal device deletes the above-mentioned target feature points that are successfully matched, it can store the target feature points in its own memory for subsequent use.

[0088] In S103, feature point matching is performed on each of the target images through a deep learning algorithm to obtain a target matching result.

[0089] In the embodiments of the present application, after the terminal device obtains the target image, since after the preliminary feature point matching by the traditional algorithm, the number of feature points that need to be feature point matched in the target image is small, that is to say, at this time, using the deep learning algorithm for feature point matching requires much less computation than performing feature point matching on each image to be matched. Therefore, in order to improve the matching accuracy, the terminal device can use the deep learning algorithm to perform final feature point matching on each feature point in the target image to obtain a target matching result. Among them, the target matching result includes the matching results of each of the multiple feature points corresponding to the target image.

[0090] The matching results of each of the above-mentioned feature points include, but are not limited to, matching success and matching failure.

[0091] It should be noted that deep learning algorithms are machine learning methods based on artificial neural networks (especially deep networks), which complete tasks by automatically learning features and patterns from data and belong to data-driven solutions.

[0092] In practical applications, deep learning algorithms include but are not limited to: feature matching (SuperGlue) algorithm based on graph neural network, feature matching (LightGlue) algorithm based on deep neural network, and end-to-end image feature matching (Detector-Free Local Feature Matching with Transformers, LoFTR) algorithm based on Transformer, etc.

[0093] As can be seen from the above, the feature point matching method provided by the embodiment of the present application obtains an image set to be matched; performs feature point matching on each image to be matched in the image set through a traditional algorithm to obtain a first matching result. Since the traditional algorithm usually has a relatively fixed calculation process and low computational complexity, the present application can quickly perform preliminary matching on a large number of feature points through the traditional algorithm, that is, process each image to be matched in the image set in a short time, greatly reducing the overall computational amount. Then, the feature points in each image to be matched are processed according to the first matching result to obtain a target image corresponding to each image to be matched; the number of feature points in the target image is less than the number of feature points in the corresponding image to be matched; feature point matching is performed on each target image through a deep learning algorithm to obtain a target matching result. Since deep learning algorithms usually require a large amount of computing resources to process complex models and large-scale data, the present application first uses a traditional algorithm to perform preliminary processing on the image, reducing the number of feature points entering the deep learning algorithm, that is, reducing the input scale of the deep learning model, so that when the deep learning algorithm is subsequently run, the required resources such as memory and computing power are correspondingly reduced, so that a more efficient feature point matching task can be achieved even in the case of limited resources (such as mobile devices or embedded systems), broadening the application scenario and also improving the matching success rate. At the same time, subsequent use of a deep learning algorithm to match the target image with a reduced number of feature points further reduces the computational amount compared to directly using a deep learning algorithm for the image set, thus significantly improving the efficiency of the entire feature point matching process.

[0094] Please refer to Figure 3 , Figure 3 which is a flowchart of the implementation of the feature point matching method provided by another embodiment of the present application. Compared with Figure 1 the corresponding embodiment, before S101, this embodiment may further include S301 to S302, which are described in detail as follows:

[0095] In S301, determine the feature point matching information; the feature point matching information is used to describe the matching accuracy requirement.

[0096] In this embodiment, since the original images in the image set usually contain a large number of feature points, processing all these feature points will consume a large amount of computing time. Therefore, in order to improve the matching efficiency, the terminal device can determine the feature point matching information corresponding to the image set. Among them, the feature point matching information is used to describe the matching accuracy requirement.

[0097] In an implementation manner of this embodiment, the user can input the matching accuracy requirements for each original image in the image set on the display screen of the terminal device. After that, the terminal device can receive the feature point matching information of each original image in the image set from the user.

[0098] It should be noted that the feature point matching information includes but is not limited to: the first information and the second information. Among them, the first information is used to describe a low matching accuracy requirement, and the second information is used to describe a high matching accuracy requirement.

[0099] In an embodiment of the present application, the terminal device can also determine the feature point matching information through steps S401 to S402 as shown below, which are described in detail as follows: Figure 4 as shown below

[0100] In S401, obtain the task information corresponding to the image set; the task information is used to describe the purpose of image processing on the image set.

[0101] In an implementation manner of this embodiment, after the user determines that the terminal device has obtained the image set, the user can manually input a task description text on the display screen of the terminal device. Based on this, the terminal device can determine the task information corresponding to the image set according to the received task description text.

[0102] In another implementation manner of this embodiment, the terminal device can also automatically generate the task information of the image set through the user's related image operations on the terminal device.

[0103] In S402, analyze the task information to obtain the feature point matching information.

[0104] In this embodiment, after the terminal device obtains the task information corresponding to the image set, the terminal device can extract keywords from the task information based on natural language processing technology to obtain the keywords corresponding to the task information. After that, the terminal device can determine the task type according to the keywords.

[0105] Exemplarily, assuming that the keyword in the task information is "stitching", the task type can be determined as image stitching; if the keyword is "identifying" a certain type of object, the task type is object recognition.

[0106] It should be noted that different task types have different requirements for feature point matching.

[0107] In this embodiment, the terminal device pre-stores the corresponding relationship between different task types and the matching requirement accuracy. Therefore, the terminal device can determine the matching accuracy requirement corresponding to the image set according to the task type corresponding to the image set and the above pre-stored corresponding relationship, that is, obtain the feature point matching information.

[0108] In an embodiment of the present application, the terminal device can determine the feature point matching information according to the following steps, which are described in detail as follows:

[0109] Obtain the historical feature point information corresponding to the task information;

[0110] Input the task information into the trained analysis model for processing to obtain the initial feature point information;

[0111] Combine the historical feature point information and the initial feature point information to obtain the feature point matching information.

[0112] In this embodiment, the terminal device can obtain the historical feature point information corresponding to the historical task that is the same as the task information. Among them, the historical feature point information is used to describe the matching accuracy requirement corresponding to the historical task.

[0113] At the same time, the terminal device can input the above task information into the trained analysis model for processing to obtain the initial feature point information. Among them, the initial feature point information is used to describe the initial matching accuracy requirement corresponding to the task information.

[0114] It should be noted that the analysis model can be obtained by training a pre-constructed deep learning model based on a preset sample set. Each sample data in the preset sample set includes sample task information and the feature point information corresponding to the sample task information. When training the pre-constructed deep learning model, the sample task information in each sample is used as the input of the deep learning model, and the feature point information corresponding to the sample task information in each sample is used as the output of the deep learning model. Through training, the deep learning model can learn the corresponding relationship between all possible sample task information and feature point information, and the trained deep learning model is used as the analysis model.

[0115] In this embodiment, after obtaining the historical feature point information and the initial feature point information, the terminal device can quantify the historical feature point information and the initial feature point information to obtain the specific value corresponding to the historical feature point information and the specific value corresponding to the initial feature point information.

[0116] It should be noted that the matching accuracy requirements for different values are different.

[0117] After that, the terminal device can perform weighted summation on the quantified historical feature point information and initial feature point information above to obtain a sum value. After that, the terminal device can generate feature point matching information according to the matching accuracy requirement corresponding to the sum value.

[0118] In S302, if the feature point matching information is the first information, feature point screening is performed on each original image in the image set to obtain each image to be matched; the number of feature points in each image to be matched is less than the number of feature points in the corresponding original image; the first information is used to describe a low matching accuracy requirement.

[0119] In this embodiment, when the terminal device detects that the feature point matching information is the first information, it indicates that the matching accuracy requirement for feature points is low at this time. That is to say, at this time, the terminal device does not need to match all the feature points in each original image in the image set one by one. Therefore, the terminal device can perform feature point screening on each original image in the image set to obtain each image to be matched after screening. Among them, the number of feature points in each image to be matched is less than the number of feature points in the corresponding original image.

[0120] In some possible embodiments, the terminal device can perform screening based on the response intensity of the feature points, that is, delete the feature points whose response intensity is less than the set intensity. Among them, the response intensity reflects the significance degree of the feature points. In the SIFT algorithm, the scale space extreme response value of the feature points can be used as a measure of the response intensity.

[0121] In some other possible embodiments, the terminal device can perform screening according to the spatial distribution of the feature points. Specifically, the terminal device can divide each original image into several uniform grid regions, and in each grid region, only retain a set number of feature points. This can ensure that the screened feature points are relatively evenly distributed on the image, avoid too many feature points concentrating in some local regions of the image, and at the same time reduce the total number of feature points.

[0122] As can be seen above, for the feature point matching method provided in this embodiment, by determining the feature point matching information, which is used to describe the matching accuracy requirement, and when it is detected that the feature point matching information is the first information, feature points of each original image in the image set are screened to obtain each image to be matched; the number of feature points in each image to be matched is less than the number of feature points in the corresponding original image; the first information is used to describe a low matching accuracy requirement. Since the original image usually contains a large number of feature points, processing all these feature points will consume a large amount of computing time. Therefore, when the matching accuracy requirement is detected to be low, this method can reduce the number of feature points through screening, so that when the matching requirement is met, the computational complexity of the subsequent feature point matching algorithm can also be significantly reduced, and the subsequent matching efficiency can be further improved.

[0123] In an embodiment of the present application, before step S103, the terminal device may specifically perform the following steps, which are described in detail as follows:

[0124] Determine the feature point matching information; the feature point matching information is used to describe the matching accuracy requirement;

[0125] If the feature point matching information is the first information, then feature points of each of the target images are screened to obtain each of the processed target images; the number of feature points in each of the processed target images is less than the number of feature points in the corresponding target image before screening; the first information is used to describe a low matching accuracy requirement.

[0126] In this embodiment, since the computational complexity of the deep learning algorithm in processing feature points is usually related to the number of feature points, that is to say, the reduction of the number of feature points can directly reduce the computational complexity of the subsequent deep learning algorithm for feature point matching. Therefore, in order to improve the matching efficiency of using the deep learning algorithm for feature point matching, the terminal device can determine the feature point matching information corresponding to each target image. Among them, the feature point matching information is used to describe the matching accuracy requirement.

[0127] In an implementation manner of this embodiment, the user can input the matching accuracy requirement for each target image in the image set on the display screen of the terminal device. After that, the terminal device can receive the feature point matching information of each target image in the image set from the user.

[0128] Among them, the feature point matching information includes but is not limited to: the first information and the second information. Among them, the first information is used to describe a low matching accuracy requirement, and the second information is used to describe a high matching accuracy requirement.

[0129] It should be noted that the terminal device can also determine the feature point matching information through steps S401 to S402 as shown in Figure 4 which will not be elaborated here.

[0130] In this embodiment, when the terminal device detects that the feature point matching information is the first information, it indicates that the matching accuracy requirement for feature points is low at this time. That is to say, at this time, the terminal device does not need to match each feature point in each target image one by one. Therefore, the terminal device can perform feature point screening on each target image to obtain each target image after screening. Among them, the number of feature points in each processed target image is less than the number of feature points in the corresponding original image.

[0131] In some possible embodiments, the terminal device can perform screening based on the response intensity of the feature points, that is, delete the feature points with a response intensity less than the set intensity. Among them, the response intensity reflects the significance degree of the feature points. In the SIFT algorithm, the scale-space extreme response value of the feature points can be used as a measure of the response intensity.

[0132] In some other possible embodiments, the terminal device can perform screening according to the spatial distribution of the feature points. Specifically, the terminal device can divide each original image into several uniform grid regions, and in each grid region, only retain a set number of feature points. This can ensure that the screened feature points are relatively evenly distributed on the image, avoid too many feature points concentrating in some local regions of the image, and at the same time reduce the overall number of feature points.

[0133] Based on this, after the terminal device obtains the processed target images, the terminal device can perform feature point matching on each processed target image through a deep learning algorithm to obtain a target matching result.

[0134] As can be seen from the above, the feature point matching method provided in this embodiment determines the feature point matching information; the feature point matching information is used to describe the matching accuracy requirement; if the feature point matching information is the first information, then perform feature point screening on each target image to obtain each processed target image; the number of feature points in each processed target image is less than the number of feature points in the corresponding target image before screening; the first information is used to describe a low matching accuracy requirement. After that, perform feature point matching on each processed target image through a deep learning algorithm to obtain a target matching result. Since the computational complexity of the deep learning algorithm in processing feature points is usually related to the number of feature points, the reduction of the number of feature points can directly reduce the computational complexity of the subsequent deep learning algorithm for feature point matching, thereby improving the matching efficiency of using the deep learning algorithm for feature point matching.

[0135] It should be understood that the magnitudes of the sequence numbers of the steps in the above embodiments do not mean the order of execution. The order of execution of each process should be determined according to its function and internal logic, and should not constitute any limitation to the implementation process of the embodiments of the present application.

[0136] Corresponding to a feature point matching method described in the above embodiments, Figure 5 FIG. shows a schematic structural diagram of a feature point matching device provided by an embodiment of the present application. For ease of description, only parts related to the embodiments of the present application are shown. Refer to Figure 5 , the feature point matching device 500 includes: a first matching unit 51, a first processing unit 52, and a second matching unit 53. Among them:

[0137] The first matching unit 51 is configured to perform feature point matching on each to-be-matched image in the image set through a traditional algorithm to obtain a first matching result.

[0138] The first processing unit 52 is configured to process the feature points in each to-be-matched image according to the first matching result to obtain a target image corresponding to each to-be-matched image; the number of feature points in the target image is less than the number of feature points in the corresponding to-be-matched image.

[0139] The second matching unit 53 is configured to perform feature point matching on each of the target images through a deep learning algorithm to obtain a target matching result.

[0140] In an embodiment of the present application, the feature point matching device 500 further includes: a first determination unit and a first screening unit. Among them:

[0141] The first determination unit is configured to determine feature point matching information; the feature point matching information is used to describe the matching accuracy requirement.

[0142] The first screening unit is configured to, if the feature point matching information is the first information, perform feature point screening on each original image in the image set to obtain each to-be-matched image; the number of feature points in each to-be-matched image is less than the number of feature points in the corresponding original image; the first information is used to describe a low matching accuracy requirement.

[0143] In an embodiment of the present application, the first determination unit specifically includes: a first acquisition unit and an analysis unit. Among them:

[0144] The first acquisition unit is configured to acquire task information corresponding to the image set; the task information is used to describe the purpose of performing image processing on the image set.

[0145] The analysis unit is configured to analyze the task information to obtain the feature point matching information.

[0146] In an embodiment of the present application, the analysis unit specifically includes: a second acquisition unit, a second processing unit, and a combination unit. Among them:

[0147] The second acquisition unit is used to acquire the historical feature point information corresponding to the task information.

[0148] The second processing unit is used to input the task information into a trained analysis model for processing to obtain initial feature point information.

[0149] The combination unit is used to combine the historical feature point information and the initial feature point information to obtain the feature point matching information.

[0150] In an embodiment of the present application, the first matching unit 51 specifically includes: a third matching unit, a second determination unit, an adjustment unit, and a third determination unit. Among them:

[0151] The third matching unit is used to perform feature point matching on each image to be matched in the image set through the traditional algorithm to obtain an initial matching result; the initial matching result includes the initial sub-results of each feature point in each image to be matched.

[0152] The second determination unit is used to determine the similarity of the alternative feature point groups; the alternative feature point groups refer to the feature point groups in each image to be matched where the initial sub-result is a successful match.

[0153] The adjustment unit is used to adjust the initial sub-result of the alternative feature point group to a failed match if the similarity is less than a set threshold.

[0154] The third determination unit is used to obtain the first matching result according to the adjusted initial sub-result and the initial sub-results of the remaining feature points except the alternative feature point groups.

[0155] In an embodiment of the present application, the first matching result includes the matching sub-results of each feature point in each image to be matched; the first processing unit 52 specifically includes: a deletion unit.

[0156] The deletion unit is used to delete the target feature points in any one image to be matched to obtain the target image corresponding to any one image to be matched; the target feature points refer to the feature points where the matching sub-result is a successful match.

[0157] In an embodiment of the present application, the feature point matching device 500 further includes: a fourth determination unit and a second screening unit; correspondingly, the second matching unit 53 specifically includes: a fourth matching unit. Among them:

[0158] The fourth determination unit is used to determine the feature point matching information; the feature point matching information is used to describe the matching accuracy requirement.

[0159] The second screening unit is configured to screen the feature points of each of the target images if the feature point matching information is the first information, so as to obtain each of the processed target images; the number of feature points in each of the processed target images is less than the number of feature points in the corresponding target image before screening; the first information is used to describe a low matching accuracy requirement.

[0160] The fourth matching unit is configured to perform feature point matching on each of the processed target images through the deep learning algorithm to obtain the target matching result.

[0161] It should be noted that for the information interaction, execution process, etc. between the above-mentioned device / units, since they are based on the same concept as the method embodiment of the present application, for their specific functions and the technical effects brought, reference can be specifically made to the method embodiment part, and details are not described herein again.

[0162] Those skilled in the art can clearly understand that for the convenience and conciseness of description, only the above division of each functional unit and module is used as an example for illustration. In practical applications, the above functions can be allocated to different functional units and modules according to needs, that is, the internal structure of the device is divided into different functional units or modules to complete all or part of the functions described above. Each functional unit and module in the embodiment can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above integrated unit can be implemented in the form of hardware or in the form of a software functional unit. In addition, the specific names of each functional unit and module are only for the convenience of mutual distinction and do not limit the protection scope of the present application. The specific working process of the units and modules in the above system can refer to the corresponding process in the foregoing method embodiment, and details are not described herein again.

[0163] Figure 6 This is a schematic structural diagram of a terminal device provided by an embodiment of the present application. As Figure 6 shown, the terminal device 6 of this embodiment includes: at least one processor 60 ( Figure 6 only one is shown in the figure), a memory 61, and a computer program 62 stored in the memory 61 and executable on the at least one processor 60. When the processor 60 executes the computer program 62, the steps in any of the above method embodiments of the feature point matching method are implemented.

[0164] The terminal device may include, but is not limited to, a processor 60 and a memory 61. Those skilled in the art can understand, Figure 6The above are merely examples of the terminal device 6, which do not constitute a limitation on the terminal device 6. It may include more or fewer components than those shown in the figure, or combine certain components, or different components. For example, it may also include input / output devices, network access devices, etc.

[0165] The so-called processor 60 may be a central processing unit (CPU), and this processor 60 may also be other general-purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or this processor may also be any conventional processor, etc.

[0166] In some embodiments, the memory 61 may be an internal storage unit of the terminal device 6, such as the memory of the terminal device 6. In some other embodiments, the memory 61 may also be an external storage device of the terminal device 6, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc., equipped on the terminal device 6. Further, the memory 61 may also include both the internal storage unit and the external storage device of the terminal device 6. The memory 61 is used to store an operating system, application programs, a boot loader, data, and other programs, such as the program code of the computer program, etc. The memory 61 may also be used to temporarily store data that has been output or is to be output.

[0167] An embodiment of the present application also provides a computer-readable storage medium storing a computer program, and when the computer program is executed by a processor, the steps in the above various method embodiments can be implemented.

[0168] An embodiment of the present application provides a computer program product, and when the computer program product runs on a terminal device, the terminal device is enabled to implement the steps in the above various method embodiments when executed.

[0169] When the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, to implement all or part of the processes in the above-described embodiment methods of this application, a computer program can be used to instruct the relevant hardware to complete. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by a processor, the steps of the above-described method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in the form of source code, object code, executable file, or some intermediate form, etc. The computer-readable medium can at least include: any entity or device capable of carrying the computer program code to the terminal device, recording medium, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal, and software distribution medium. For example, a USB flash drive, a portable hard drive, a magnetic disk, or an optical disc, etc. In some jurisdictions, according to legislation and patent practice, the computer-readable medium cannot be an electrical carrier signal and a telecommunication signal.

[0170] In the above embodiments, the descriptions of the various embodiments have their own emphases. For the parts not detailed or recorded in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0171] The above-described embodiments are only used to illustrate the technical solutions of this application, rather than to limit it; although this application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements on some of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the various embodiments of this application, and should all be included in the protection scope of this application.

Claims

1. A feature point matching method, characterized in that, Including: Performing feature point matching on each image to be matched in the image set through a traditional algorithm to obtain a first matching result; Processing the feature points in each image to be matched according to the first matching result to obtain a target image corresponding to each image to be matched; the number of feature points in the target image is less than the number of feature points in the corresponding image to be matched; Performing feature point matching on each of the target images through a deep learning algorithm to obtain a target matching result.

2. The feature point matching method according to claim 1, wherein Before performing feature point matching on each image to be matched in the image set through the traditional algorithm to obtain a first matching result, it further includes: Determining feature point matching information; the feature point matching information is used to describe the matching accuracy requirement; If the feature point matching information is the first information, performing feature point screening on each original image in the image set to obtain each image to be matched; the number of feature points in each image to be matched is less than the number of feature points in the corresponding original image; the first information is used to describe a low matching accuracy requirement.

3. The feature point matching method according to claim 2, wherein The determining the feature point matching information includes: Obtaining task information corresponding to the image set; the task information is used to describe the purpose of performing image processing on the image set; Analyzing the task information to obtain the feature point matching information.

4. The feature point matching method according to claim 3, wherein The analyzing the task information to obtain the feature point matching information includes: Obtaining historical feature point information corresponding to the task information; Inputting the task information into a trained analysis model for processing to obtain initial feature point information; Combining the historical feature point information and the initial feature point information to obtain the feature point matching information.

5. The feature point matching method according to claim 1, wherein The performing feature point matching on each image to be matched in the image set through the traditional algorithm to obtain a first matching result includes: Performing feature point matching on each image to be matched in the image set through the traditional algorithm to obtain an initial matching result; the initial matching result includes initial sub-results of each feature point in each image to be matched; Determining the similarity of alternative feature point groups; the alternative feature point groups refer to the feature point groups in each image to be matched where the initial sub-result is a successful match; If the similarity is less than a set threshold, adjusting the initial sub-result of the alternative feature point group to a failed match; Obtaining the first matching result according to the adjusted initial sub-result and the initial sub-results of the remaining feature points other than the alternative feature point group.

6. The feature point matching method according to claim 1, wherein The first matching result includes matching sub-results of each feature point in each image to be matched; the processing the feature points in each image to be matched according to the first matching result to obtain a target image corresponding to each image to be matched includes: For any image to be matched, deleting the target feature points in the any image to be matched to obtain a target image corresponding to the any image to be matched; the target feature points refer to the feature points where the matching sub-result is a successful match.

7. The feature point matching method according to any one of claims 1-6, characterized in that, Before performing feature point matching on each of the target images through the deep learning algorithm to obtain a target matching result, it further includes: Determine the feature point matching information; the feature point matching information is used to describe the matching accuracy requirement; If the feature point matching information is the first information, perform feature point screening on each of the target images to obtain each of the processed target images; the number of feature points in each of the processed target images is less than the number of feature points in the corresponding target image before screening; the first information is used to describe a low matching accuracy requirement; Correspondingly, the feature point matching of each of the target images by the deep learning algorithm to obtain the target matching result includes: Performing feature point matching on each of the processed target images by the deep learning algorithm to obtain the target matching result.

8. A feature point matching device, characterized in that Including: A first matching unit, configured to perform feature point matching on each of the images to be matched in the image set by a traditional algorithm to obtain a first matching result; A first processing unit, configured to process the feature points in each of the images to be matched according to the first matching result to obtain the target image corresponding to each of the images to be matched; the number of feature points in the target image is less than the number of feature points in the corresponding image to be matched; A second matching unit, configured to perform feature point matching on each of the target images by a deep learning algorithm to obtain a target matching result.

9. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the feature point matching method according to any one of claims 1 to 7.

10. A computer program product, characterized in that, Including a computer program, which implements the feature point matching method according to any one of claims 1 to 7 when running.

Images