Image feature point pair screening, image processing method and device, and electronic equipment

Abstract

The application provides an image feature point pair screening method, an image processing method and device, and an electronic device. The image feature point pair screening method comprises: determining a plurality of feature point pairs corresponding to a target image and a first error value of the feature point pairs; screening at least one to-be-detected feature point pair from the plurality of feature point pairs according to the first error value; determining a second error value of the to-be-detected feature point pair; determining a third error value according to the first error value and the second error value; and screening at least one target feature point pair from the plurality of to-be-detected feature point pairs according to the third error value. In this way, the to-be-detected feature point pairs screened after the initial screening are subjected to secondary screening, the probability of removing correct feature point pairs as mis-matched feature point pairs is reduced, and the screening accuracy of the feature point pairs is improved. Furthermore, the target feature point pair is determined based on the third error value, which can make the matching accuracy of the feature point pairs higher compared with determining the correct feature point pair only according to the first error value.

Description

[0001] TECHNICAL FIELD

[0002] The present application relates to the image processing technical field, and in particular to a screening of image feature point pairs, an image processing method and device, and an electronic device. BACKGROUND

[0003] The screening of image feature point pairs is a basic part in the computer vision field, which can provide data support for object recognition, target tracking, three-dimensional reconstruction and other specific applications in the computer vision field in image processing. The screening of image feature point pairs refers to finding out the correctly matched feature points between two images to be registered. The matched feature points between the two images form a feature point pair.

[0004] At present, for the screening of feature point pairs in, for example, simultaneous localization and mapping (SLAM), a feature point descriptor method is mostly used for matching of feature point pairs, that is, feature point extraction and descriptor calculation are performed on each frame of image, and then Hamming distance and other methods are used to match the feature point pairs based on the descriptors. However, in a low-texture scene or a scene with high similarity, this method will produce many mis-matched feature point pairs, and for the mis-matched feature point pairs, the prior art directly eliminates the mis-matched feature point pairs. In this way, the correct feature point pairs may be eliminated as mis-matched feature point pairs, resulting in inaccurate screening of feature point pairs and low matching accuracy between feature point pairs. SUMMARY

[0005] The present application provides a screening of image feature point pairs, an image processing method and device, and an electronic device, to solve the problem of inaccurate screening of feature point pairs and low matching accuracy between feature point pairs caused by the elimination of correct feature point pairs as mis-matched feature point pairs in the prior art.

[0006] In a first aspect, the present application provides a screening method of image feature point pairs, applied to an electronic device, which comprises: determining at least one feature point pair corresponding to a target image, and determining a first error value of the feature point pair; screening at least one feature point pair from the at least one feature point pair as a to-be-detected feature point pair according to the first error value of the feature point pair; determining a second error value of the to-be-detected feature point pair; determining a third error value of the to-be-detected feature point pair according to the first error value and the second error value of the to-be-detected feature point pair; and screening at least one to-be-detected feature point pair from the at least one to-be-detected feature point pair as a target feature point pair according to the third error value.

[0007] In the implementation of the present application, the first error value is, for example, an optical flow error (i.e., LK error) determined based on the LK optical flow method, and the second error value is specifically a spatial similarity error of the feature point pair.

[0008] The screening method for image feature point pairs provided in the implementation of the present application determines at least one feature point pair and the first error value of each feature point pair, preliminarily determines the mismatched feature point pair as the to-be-detected feature point pair according to the first error value, determines the second error value, and then determines the third error value according to the first error value and the second error value, so as to screen the target feature point pair from the at least one to-be-detected feature point pair according to the third error value. In this way, the to-be-detected feature point pair screened out is subjected to secondary screening according to the third error value of the to-be-detected feature point pair after the initial screening, which reduces the probability of removing the correct feature point pair as the mismatched feature point pair, and improves the screening accuracy of the feature point pair. Moreover, the target feature point pair is determined based on the third error value, which can make the matching accuracy between the feature point pairs higher compared with determining the correct feature point pair only based on the first error value.

[0009] In one implementation of the screening method for image feature point pairs provided in the implementation of the present application, determining the second error value of the to-be-detected feature point pair includes: determining at least one vector direction similarity and at least one vector modulus similarity of the to-be-detected feature point pair; and determining the second error value of the to-be-detected feature point pair according to the at least one vector direction similarity and the at least one vector modulus similarity.

[0010] In the implementation of the present application, the spatial similarity error of the to-be-detected feature point pair is obtained according to the vector direction similarity and the vector modulus similarity of the to-be-detected feature point pair, so that the to-be-detected feature point pair can be subjected to secondary screening based on the spatial similarity error of the to-be-detected feature point pair, and the feature point pair with higher matching accuracy can be screened out.

[0011] In an implementation of the method for screening image feature point pairs provided in the present application, the feature point pairs include correct feature point pairs, the correct feature point pairs are other feature point pairs than the to-be-detected feature point pairs in the at least one feature point pair, the correct feature point pairs include first correct feature points and second correct feature points, the to-be-detected feature point pairs include first to-be-detected feature points and second to-be-detected feature points, and determining the at least one vector direction similarity and the at least one vector modulus similarity of the to-be-detected feature point pairs includes: determining at least one first correct feature point corresponding to the first to-be-detected feature point and at least one second correct feature point corresponding to the second to-be-detected feature point; determining coordinates and modulus values of at least one first vector and coordinates and modulus values of at least one second vector, wherein the first vector is a vector with the first to-be-detected feature point as a starting point and the first correct feature point as an ending point, and the second vector is a vector with the second to-be-detected feature point as a starting point and the second correct feature point as an ending point; determining the at least one vector direction similarity of the to-be-detected feature point pairs according to the coordinates of the at least one first vector and the coordinates of the at least one second vector; and determining the at least one vector modulus similarity of the to-be-detected feature point pairs according to the modulus values of the at least one first vector and the modulus values of the at least one second vector.

[0012] In the present application, the vector direction similarity of the to-be-detected feature point pairs is determined based on the coordinates of the first vectors formed by the first to-be-detected feature point and the plurality of first correct feature points and the coordinates of the second vectors formed by the second to-be-detected feature point and the plurality of second correct feature points, and the vector modulus similarity of the to-be-detected feature point pairs is determined based on the modulus values of the first vectors and the modulus values of the second vectors. In this way, the second error value of the to-be-detected feature point pairs is obtained through the vector relationship between the to-be-detected feature point pairs and the plurality of correct feature point pairs, which can make the obtained vector direction similarity and vector modulus similarity of the to-be-detected feature point pairs more accurate, and further obtain a more accurate spatial similarity error, so that the screening of the to-be-detected feature point pairs is more accurate and the purpose of accurately screening feature point pairs can be achieved.

[0013] In an implementation of the method for screening image feature point pairs provided in the present application, the first vector and the second vector are obtained in the following manner: determining a first neighborhood of the first to-be-detected feature point and a second neighborhood of the second to-be-detected feature point; determining a first gray centroid point of the at least one first correct feature point in the first neighborhood and a second gray centroid point of the at least one second correct feature point in the second neighborhood according to the gray centroid method; determining a first directional vector of the first neighborhood and a second directional vector of the second neighborhood, wherein the first directional vector is a vector with the first to-be-detected feature point as the starting point and the first gray centroid point as the ending point, and the second directional vector is a vector with the second to-be-detected feature point as the starting point and the second gray centroid point as the ending point; determining a first included angle between the first vector direction and the first directional axis, and determining a second included angle between the second vector direction and the second directional axis; rotating a vector with the first to-be-detected feature point as the starting point and the first correct feature point as the ending point by the first included angle along the first direction to obtain the first vector, and rotating a vector with the second to-be-detected feature point as the starting point and the second correct feature point as the ending point by the second included angle along the first direction to obtain the second vector.

[0014] In the present application, the gray centroid points of the to-be-detected feature point pairs are determined based on the gray centroid method, and the directional vectors of the neighborhoods composed of the to-be-detected feature points and the gray centroid points are determined, so as to determine the rotation angles of the correct feature points according to the included angles between the directional vectors and the first directional axis, and rotate the correct feature points, and then obtain the first vector and the second vector based on the rotated correct feature points, thereby ensuring the rotation invariance of the image.

[0015] In an implementation of the method for screening image feature point pairs provided in the present application, the vector direction similarity of the to-be-detected feature point pairs is obtained in the following manner:

[0016]

[0017] wherein direct_sim is the vector direction similarity of the to-be-detected feature point pairs, (x1, y1) is the coordinate of the first vector, and (x2, y2) is the coordinate of the second vector.

[0018] In an implementation of the method for screening image feature point pairs provided in the present application, the vector modulus similarity of the to-be-detected feature point pairs is obtained in the following manner:

[0019]

[0020] wherein dis_sim is the vector modulus similarity of the to-be-detected feature point pairs, d1 is the modulus of the first vector, and d'1 is the modulus of the second vector.

[0021] In the implementation of the present application, the calculation formula of the vector direction similarity and the vector modulus similarity is used to obtain more accurate calculation results of the vector direction similarity and the vector modulus similarity.

[0022] In an implementation of the image feature point pair screening method provided by the present application, the second error value is obtained by the following method:

[0023]

[0024] wherein, error sp is the second error value, direct_sim i is the i-th vector direction similarity of the to-be-detected feature point pair, dis_sim i is the i-th vector modulus similarity of the to-be-detected feature point pair, and i≥1.

[0025] In the implementation of the present application, the spatial similarity of the to-be-detected feature point pair can be obtained according to the plurality of vector direction similarities and the plurality of vector modulus similarities. In this way, the feature point pair is screened based on the spatial similarity, so that the matching accuracy of the feature point pair is higher.

[0026] In an implementation of the image feature point pair screening method provided by the present application, the third error value of the to-be-detected feature point pair is determined according to the first error value and the second error value of the to-be-detected feature point pair, including: determining a first weight of the first error value and a second weight of the second error value; and performing weighted average processing on the first error value and the second error value according to the first weight of the first error value and the second weight of the second error value to obtain the third error value.

[0027] In the implementation of the present application, the to-be-detected feature point pair is screened based on the LK error, and then the spatial similarity error of the to-be-detected feature point pair is calculated. The LK error and the spatial similarity error are weighted and averaged to obtain the final third error value, so that the matching accuracy of the feature point pair is determined based on the third error value, thereby avoiding the problem that only the LK error is used to screen the feature point pair and the correct feature point pair is removed as an error feature point pair.

[0028] In an implementation of the image feature point pair screening method provided by the present application, the at least one feature point pair is determined, including: determining a first target image and a second target image; determining a first feature point in the first target image and a second feature point corresponding to the first feature point in the second target image according to an optical flow method; and obtaining a feature point pair according to the first feature point and the second feature point.

[0029] In the implementation provided in the present application, the feature point matching method using the optical flow method can reduce the time for calculating the feature points of a single image, compared with the method of determining the feature point pairs using the feature point descriptor.

[0030] In an implementation of the method for screening the image feature point pairs provided in the present application, the first error value of the feature point pair is determined by using the error function.

[0031] In the implementation provided in the present application, the first error value of the feature point pair is determined by using the error function of the optical flow method. In this way, the feature point pairs to be detected can be determined based on the first error value, which facilitates the subsequent screening of the feature point pairs to be detected.

[0032] In an implementation of the method for screening the image feature point pairs provided in the present application, the at least one feature point pair is screened from the at least one feature point pair as the feature point pair to be detected according to the first error value of the feature point pair, including: screening the at least one feature point pair corresponding to the proportion threshold from the at least one feature point pair in descending order according to the first error value and the preset proportion threshold, as the feature point pair to be detected; or screening the at least one feature point pair with the first error value greater than the first error threshold from the at least one feature point pair according to the first error value and the preset first error threshold, as the feature point pair to be detected.

[0033] In the implementation provided in the present application, after the first error values of the feature point pairs are determined, the first error values of the feature point pairs are sorted, and the feature point pairs corresponding to the proportion threshold are screened from the feature point pairs in descending order according to the proportion threshold and the sorted first error values, as the feature point pairs to be detected. Or the feature point pairs with the first error value greater than the preset first error value are screened from the feature point pairs according to the preset first error threshold and the first error value, as the feature point pairs to be detected. In this way, the feature point pairs with larger first error values can be screened out, which facilitates the subsequent secondary screening of the feature point pairs to be detected.

[0034] In an implementation of the method for screening the image feature point pairs provided in the present application, the method further includes: taking the other feature point pairs in the at least one feature point pair except the feature point pairs to be detected as the target feature point pairs.

[0035] In the implementation provided in the present application, the feature point pairs with smaller first error values are taken as the correct feature point pairs (i.e., the target feature point pairs), and the feature point pairs to be detected with smaller third error values are also taken as the correct feature point pairs. In this way, the feature point pairs with more accurate matching can be obtained, which facilitates the subsequent image processing based on the feature point pairs.

[0036] In an implementation of the method for screening image feature point pairs provided in the present application, the at least one to-be-detected feature point pair is screened as the target feature point pair according to the third error value, comprising: screening the at least one to-be-detected feature point pair with the third error value less than or equal to the preset second error threshold as the target feature point pair according to the third error value and the preset second error threshold.

[0037] In the implementation of the present application, the plurality of to-be-detected feature point pairs with the third error value less than or equal to the preset second error threshold are screened as the target feature point pairs. In this way, more feature point pairs with smaller error values can be retained.

[0038] In an implementation of the method for screening image feature point pairs provided in the present application, the first error value is an optical flow error determined based on an optical flow method, and the second error value is a spatial similarity error of the feature point pair.

[0039] In the implementation of the present application, whether the feature point pair is a correct feature point pair or an incorrect feature point pair is determined according to the optical flow error and the spatial similarity error. In this way, the screening accuracy of the feature point pair can be improved, and the matching accuracy of the retained feature point pair is thus higher.

[0040] In a second aspect, the present application provides a method for image processing, comprising: determining a target feature point pair corresponding to a target image, the target feature point pair being obtained by the method for screening image feature point pairs provided in the first aspect; and performing image processing on the target image according to the target feature point pair.

[0041] In a third aspect, the present application provides a device for screening feature point pairs, comprising: a first processing module configured to determine at least one feature point pair corresponding to a target image and determine a first error value of the feature point pair; a second processing module configured to screen at least one feature point pair from the at least one feature point pair as a to-be-detected feature point pair according to the first error value of the feature point pair; a third processing module configured to determine a second error value of the to-be-detected feature point pair; a fourth processing module configured to determine a third error value of the to-be-detected feature point pair according to the first error value and the second error value of the to-be-detected feature point pair; and a fifth processing module configured to screen at least one to-be-detected feature point pair from the at least one to-be-detected feature point pair as a target feature point pair according to the third error value.

[0042] The device for screening feature point pairs provided in the present application comprises a module for executing the method for screening image feature point pairs provided in the first aspect, and thus can also achieve the beneficial effects (or advantages) of the method for screening image feature point pairs provided in the first aspect.

[0043] In a fourth aspect, the present application provides an image processing apparatus, comprising: a first processing module configured to determine a target feature point pair corresponding to a target image, the target feature point pair being obtained by the screening method of the image feature point pair provided in the first aspect; and a second processing module configured to perform image processing on the target image according to the target feature point pair.

[0044] In a fifth aspect, an embodiment of the present application provides an electronic device, comprising: a processor, and a memory connected to the processor in communication; the memory stores computer execution instructions; and the processor executes the computer execution instructions stored in the memory to implement the screening method of the image feature point pair provided in the implementation manners of the first aspect or the image processing method provided in the second aspect.

[0045] In a sixth aspect, an embodiment of the present application provides a computer readable storage medium, the computer readable storage medium storing computer execution instructions, the computer execution instructions being executed by a processor to implement the screening method of the image feature point pair provided in the implementation manners of the first aspect or the image processing method provided in the second aspect.

[0046] In a seventh aspect, an embodiment of the present application provides a computer program product, comprising a computer program, the computer program being executed by a processor to implement the screening method of the image feature point pair provided in the implementation manners of the first aspect or the image processing method provided in the second aspect.

[0047] It can be understood that the beneficial effects of the second aspect to the seventh aspect can also be referred to the related description in the first aspect, which will not be repeated here. BRIEF DESCRIPTION OF DRAWINGS

[0048] The accompanying drawings, which are incorporated herein and constitute part of the specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0049] Figure 1 A flowchart of the screening method of the image feature point pair provided in an embodiment of the present application;

[0050] Figure 2 A corresponding relationship diagram of the feature point pair provided in an embodiment of the present application;

[0051] Figure 3 A diagram of the first error value of the feature point pair provided in an embodiment of the present application;

[0052] Figure 4 A corresponding relationship diagram of the to-be-detected feature point pair and the correct feature point pair provided in an embodiment of the present application;

[0053] Figure 5 A corresponding relationship diagram of a feature point pair to be detected and a gray centroid point provided by an embodiment of the present application is shown in the following figure:

[0054] Figure 6 A flowchart of another screening method of an image feature point pair provided by an embodiment of the present application is shown in the following figure:

[0055] Figure 7 A flowchart of an image processing method provided by an embodiment of the present application is shown in the following figure:

[0056] Figure 8 A structural diagram of a screening device of a feature point pair provided by an embodiment of the present application is shown in the following figure:

[0057] Figure 9 A structural diagram of an image processing device provided by an embodiment of the present application is shown in the following figure:

[0058] Figure 10 A structural diagram of an electronic device provided by an embodiment of the present application is shown in the following figure.

[0059] The above figures have shown the explicit embodiments of the present application, which will be described in more detail hereinafter. These figures and the written description are not intended to limit the scope of the present application concept in any way, but to illustrate the present application concept to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0060] The exemplary embodiments will be described in detail herein with reference to the attached drawings. The following description is only one of the exemplary embodiments in accordance with the present application. Therefore, it is not intended to limit the scope of the present application, but to explain the present application concept to those skilled in the art by referring to the specific embodiments.

[0061] As described above, the screening of the feature point pair belongs to the computer vision positioning and mapping (simultaneous localization and mapping, SLAM for short) field. In the prior art, there is a case where the correct feature point pair is removed as a mis-matched feature point pair, thereby resulting in inaccurate screening of the feature point pair and low matching accuracy between the feature point pairs.

[0062] The image feature point pair screening method provided by the implementation manner of the present application, after obtaining the matched feature point pairs, determines the first error value of the matched feature point pairs, and screens out the to-be-detected feature point pairs based on the first error value, and determines the second error value of the to-be-detected feature point pairs, determines the third error value according to the first error value and the second error value, and screens the to-be-detected feature point pairs according to the third error value to obtain the target feature point pairs. In this way, compared with the manner of directly eliminating the mismatched feature point pairs in the prior art, the probability of eliminating the correct feature point pairs as the mismatched feature point pairs can be reduced, and the screening accuracy of the feature point pairs is improved. Moreover, the matching accuracy between the feature point pairs can be improved.

[0063] Referring to Figure 1 The image feature point pair screening method provided by the implementation manner of the present application is applied to an electronic device, and the electronic device specifically performs the following steps to realize the screening of the feature point pairs.

[0064] S110, at least one feature point pair corresponding to a target image is determined, and a first error value of the feature point pair is determined.

[0065] S120, at least one feature point pair is screened out from the at least one feature point pair as the to-be-detected feature point pair according to the first error value of the feature point pair.

[0066] S130, a second error value of the to-be-detected feature point pair is determined.

[0067] S140, a third error value of the to-be-detected feature point pair is determined according to the first error value and the second error value of the to-be-detected feature point pair.

[0068] S150, at least one to-be-detected feature point pair is screened out from the at least one to-be-detected feature point pair as the target feature point pair according to the third error value.

[0069] The image feature point pair filtering method provided in this application, after determining at least one feature point pair and a first error value for each feature point pair, determines at least one feature point pair to be detected based on the first error value, determines a second error value for each feature point pair to be detected, and determines a third error value based on the first and second error values. The feature point pairs to be detected are then filtered based on the third error value. Thus, mismatched feature point pairs are initially identified as feature point pairs to be detected based on the first error value. By determining the second error value, and then the third error value, target feature point pairs are determined from at least one pair to be detected based on the third error value. After the initial filtering, a second filtering is performed on the selected feature point pairs based on the third error value, reducing the probability of rejecting correct feature point pairs as mismatched feature point pairs and improving the filtering accuracy of feature point pairs. Furthermore, determining target feature point pairs based on the third error value, compared to determining correct feature point pairs only based on the first error value, results in higher matching accuracy between feature point pairs.

[0070] First, the technical content of determining at least one pair of feature points corresponding to the target image and determining the first error value of the feature point pair in step S110 is executed.

[0071] As mentioned earlier, feature point pairs can be matched using feature point descriptors. However, this method requires feature point extraction and descriptor calculation for each frame of the image. Since there are many feature points, the calculation of the descriptors is very time-consuming, thus increasing the processing time per frame and resulting in high latency in the selection of image feature point pairs.

[0072] Based on this, the implementation method of this application adopts the Lucas-Kanade optical flow method (i.e., LK optical flow method) for feature point pair matching.

[0073] For example, such as Figure 2 As shown, a first target image 100 and a second target image 200 are determined. A first feature point in the first target image 100 and a second feature point in the second target image 200 corresponding to the first feature point are determined. Feature point pairs are obtained based on the first feature point and the second feature point (e.g., ...). Figure 2 The endpoints of any connecting line in the diagram are the first feature point and its corresponding second feature point.

[0074] The first target image 100 and the second target image 200 can be two consecutive frames during vehicle movement or two frames with intervals. Of course, in this implementation, to make feature point matching more accurate, two consecutive frames are used as the first target image 100 and the second target image 200.

[0075] For example, based on the optical flow method, the pixel gray scale of the same spatial point in the two consecutive image frames is constant and the pixel motion of the spatial point neighborhood window is constant, the motion of the feature points in the X direction and the Y direction in the two image frames is obtained to obtain the initial feature point pair.

[0076] In the implementation of the present application, in order to solve the problem of fast object motion, the pyramid LK optical flow method is introduced to determine and match the feature points.

[0077] Next, the specific process of matching the feature points based on the pyramid LK optical flow method to obtain the initial feature point pair in the implementation of the present application will be described in detail.

[0078] In the LK optical flow method, it is assumed that the pixels in a certain image window have the same motion. And assuming a window with a size of w x w, the window will contain w 2 number of pixels.

[0079] Based on the above assumptions, for each pixel, the following equation is obtained:

[0080]

[0081] Where I x is the gradient change of the pixel in the x-axis direction, I y is the gradient change of the pixel in the y-axis direction, I tk is the change of the pixel over time, and u, v are the motion speeds of the pixel in the x, y-axis directions, respectively. And u, v can be obtained by the least square method.

[0082] According to the above equation, based on the motion process of the pixel points from the first image to the second image, the pixel points in the first image (as an example of the first feature point) corresponding to the pixel points in the second image (as an example of the second feature point) can be determined.

[0083] The way of matching feature points by using the LK optical flow method in the implementation of the present application can reduce the time of calculating the feature points of a single image, compared with the way of matching feature points by using the feature point descriptor. Only the key points (i.e. feature points) are calculated, and there is no need to calculate the descriptor. Even the calculation of the key points can be omitted for the adjacent image frames, and the LK optical flow method can be directly used for feature point matching.

[0084] Further, after obtaining the matched feature point pair by the LK optical flow method, if in the scene where there is occlusion or the image texture is relatively similar, there will still be many false matching points, which will affect the subsequent camera pose transformation, object tracking or spatial point three-dimensional reconstruction, etc. Figure 2 As shown in Figure 2For each pair of feature points between two images obtained based on the optical flow method, two end points of each black line represent a pair of feature points.

[0085] Based on this, the implementation of the present application needs to determine the mis-matched feature point pairs (as an example of the feature point pairs to be detected).

[0086] The implementation of the present application determines the first error value of the feature point pair by using the error function of the optical flow method.

[0087] For example, the first error value of each feature point pair can be obtained by using the LK optical flow function of OpenCV, such as cv: calcOpticalFlowPyrLK.

[0088] Wherein, cv: calcOpticalFlowPyrLK is an error function for determining the error value of the feature point pair in the pyramid LK optical flow method.

[0089] As shown in Figure 3 , the LK error (as an example of the first error value) between the feature point pairs of the first target image 100 and the second target image 200 can be obtained based on the error function. For example, the error value is 31, 6, 32,..., 78, etc. It can be seen that the greater the matching deviation of the feature point pair, the greater the error value will be.

[0090] That is, the smaller the LK error, the higher the matching degree of the feature point, and the greater the LK error value, the lower the matching degree of the feature point.

[0091] Next, the technical content of step S120 is executed, that is, at least one feature point pair is selected from the at least one feature point pair according to the first error value of the feature point pair, as the feature point pair to be detected.

[0092] For example, after determining the first error value of several feature point pairs, at least one feature point pair corresponding to the preset proportion threshold is selected from the several feature point pairs in descending order according to the first error value and the preset proportion threshold, as the feature point pair to be detected.

[0093] For example, the preset proportion threshold can be 10%, 20%, 30%, 40%, 50%, etc.

[0094] In the implementation of the present application, the first error value is sorted in descending order or in ascending order, and the feature point pairs with larger first error values, for example, the top 20% of feature point pairs, are determined as the feature point pairs to be detected. That is, 80% of the feature point pairs with smaller LK error are correct feature point pairs.

[0095] Further, in the implementation of the present application, after the first error values of the plurality of feature point pairs are determined, the plurality of feature point pairs having first error values greater than the preset first error threshold value can be selected from the plurality of feature point pairs as the to-be-detected feature point pairs according to the preset first error threshold value and the first error values.

[0096] For example, the preset first error threshold value can be 4, 6, 10, 15, 20, etc. The smaller the value of the preset first error threshold value, the more to-be-detected feature point pairs will be selected, and the matching accuracy of the retained feature point pairs (i.e. correct feature point pairs) will be higher.

[0097] In the implementation of the present application, the feature point pairs other than the to-be-detected feature point pairs in the at least one feature point pair are retained as correct feature point pairs (as part of the target feature point pairs).

[0098] Next, the technical content of determining the second error value of the to-be-detected feature point pair in step S130 is executed.

[0099] For example, for any one to-be-detected feature point pair in the two adjacent image frames, the matching feature points (i.e. the first to-be-detected feature point and the second to-be-detected feature point) each have a similar geometric relationship with other feature points (i.e. the first correct feature point and the second correct feature point) in the respective neighborhood window (i.e. the first neighborhood and the second neighborhood). As shown in Figure 4 Figure 4 The left side is a distribution diagram of feature points of the previous frame image (i.e. the first target image 100), with a black solid dot as a to-be-detected feature point pair (as an example of the first to-be-detected feature point), in the previous frame image, the to-be-detected feature points in the to-be-detected feature point pair have five correct feature points A1, B1, C1, D1, E1 (as examples of the first correct feature point) in their neighborhood (as an example of the first neighborhood), and the five correct feature points can obtain the matching feature points (i.e. the second correct feature points corresponding to the first correct feature point pairs) in the next frame image through the LK optical flow method. As shown in Figure 4 Figure 4 ​​The right side is a distribution diagram of feature points of the next frame image (i.e., the second target image 200). The next frame image also has black solid dots (as an example of the second to-be-detected feature points), and in the neighborhood (as an example of the second neighborhood) of the black solid dots, there are five correct feature points A2, B2, C2, D2, and E2 (as examples of the second correct feature points). Among them, A1 and A2 form a correct feature point pair, B1 and B2 form a correct feature point pair, C1 and C2 form a correct feature point pair, D1 and D2 form a correct feature point pair, and E1 and E2 form a correct feature point pair. Further, if the feature point pair is a correct matching point pair, the vector similarity of the first to-be-detected feature point in the previous frame and any feature point (i.e., the first correct feature point) in the neighborhood (i.e., the first neighborhood) of the first to-be-detected feature point and the vector similarity of the corresponding second to-be-detected feature point in the next frame and any feature point (i.e., the second correct feature point) in the neighborhood (i.e., the second neighborhood) of the second to-be-detected feature point will be high. For example, the vector d1 of the previous frame image has high similarity with the vector d'1 of the next frame image, the vector d2 of the previous frame image has high similarity with the vector d'2 of the next frame image, the vector d3 of the previous frame image has high similarity with the vector d'3 of the next frame image, the vector d4 of the previous frame image has high similarity with the vector d'4 of the next frame image, and the vector d5 of the previous frame image has high similarity with the vector d'5 of the next frame image. The vector similarity is also the spatial similarity error (as an example of the second error value).

[0100] In the implementation of the present application, the vector direction similarity and the vector modulus similarity of the to-be-detected feature point pair are determined, and the second error value of the to-be-detected feature point is determined according to the vector direction similarity and the vector modulus similarity.

[0101] For example, first, the neighborhood space of the to-be-detected feature point is established. The to-be-detected feature point is taken as the center point, a circle with a radius r is drawn, and the center point and the feature points in the circle are connected to obtain the vectors of the center point and the feature points. The feature points connected to the center point are correct matching points determined based on the LK optical flow method.

[0102] Further, in order to ensure the rotation invariance, as shown in Figure 5 the gray centroid points (as examples of the first gray centroid points) of the correct feature points in the first neighborhood of the first target image 100 are determined according to the gray centroid method, and the gray centroid points (as examples of the second gray centroid points) of the correct feature points in the second neighborhood of the second target image 200 are determined.

[0103] In the implementation of the present application, the first gray centroid points in the first target image 100 and the second gray centroid points in the second target image 200 are determined in the same way. As shown in Figure 5 Figure 5 ​This is a mapping diagram showing the correspondence between the feature points to be detected and the gray-level centroids in the first target image 100 or the second target image 200. As shown in the figure, the feature point to be detected is point O (as an example of the first or second feature point to be detected), and the gray-level centroid is point A (as an example of the first or second gray-level centroid). The resulting vector is a vector originating from the feature point O and ending at the gray-level centroid A. (As an example of a first-direction vector or a second-direction vector), and obtain the vector. The angle between the vector and the X-axis (as an example of the first direction axis) is θ (as an example of the first or second angle). Connect the target feature point and the correct feature point (as an example of the first or second correct feature point) within the neighborhood to obtain the corresponding vector. Rotate the vector formed by the target feature point and the correct feature point clockwise (as an example of the first direction) by an angle θ. After rotation, the direction coordinates and magnitude of the rotated vector (as an example of the first or second vector) formed with the target feature point as the starting point and the correct feature point as the ending point are obtained. Figure 4 As shown, the matching vectors are d1 (as an example of the first vector) and d′1 (as an example of the second vector), where d1 = (x1, y1) (as an example of the coordinates of the first vector) and d′1 = (x2, y2) (as an example of the coordinates of the second vector).

[0104] In the implementation of the present application, the first neighborhood of the first to-be-detected feature point and the second neighborhood of the second to-be-detected feature point are determined, and then the first gray centroid of the first correct feature points in the first neighborhood and the second gray centroid of the second correct feature points in the second neighborhood are determined according to the gray centroid method, to obtain the first direction vector in the first neighborhood with the first to-be-detected feature point as the starting point and the first gray centroid as the ending point, and the second direction vector in the second neighborhood with the second to-be-detected feature point as the starting point and the second gray centroid as the ending point, and then the first angle between the first direction vector and the X axis and the second angle between the second direction vector and the X axis are obtained, the vector with the to-be-detected feature point as the starting point and each first correct feature point as the ending point is rotated by the first angle in the clockwise direction to obtain the first vector with the first to-be-detected feature point as the starting point and the first correct feature point as the ending point, and the vector with the to-be-detected feature point as the starting point and each second correct feature point as the ending point is rotated by the second angle in the clockwise direction to obtain the second vector with the second to-be-detected feature point as the starting point and the second correct feature point as the ending point. In this way, the first direction vector of the first target image 100 and the second direction vector of the second target image 200 are both direction vectors of the X axis, which can ensure the rotation invariance of the first target image 100 and the second target image, and then ensure the accuracy of the vector direction similarity and the vector modulus similarity obtained based on the first vector and the second vector.

[0105] It should be noted that the angles of the first angle and the second angle can be the same or different, and the specific angle value is determined based on the gray centroid points in each image.

[0106] Further, the vector direction similarity of the to-be-detected feature point pair is determined according to the coordinates of the first vector and the coordinates of the second vector.

[0107] For example, the vector direction similarity is expressed by the cosine theorem, and the vector direction similarity is obtained by the following formula:

[0108]

[0109] Wherein, direct_sim is the vector direction similarity of the to-be-detected feature point pair, (x1, y1) is the coordinate of the first vector, and (x2, y2) is the coordinate of the second vector.

[0110] Further, the vector modulus similarity of the to-be-detected feature point pair is determined according to the modulus of the first vector and the modulus of the second vector.

[0111] For example, the vector modulus similarity is obtained by the following formula:

[0112]

[0113] wherein dis_sim is the vector modulus value similarity of the to-be-detected feature point pair, d1 is the modulus value of the first vector, and d'1 is the modulus value of the second vector.

[0114] wherein,

[0115] Thus, in the neighborhood centered on the to-be-detected feature point, the vector direction similarity direct_sim and the vector modulus value similarity dis_sim of the to-be-detected feature point and each correct feature point are calculated. For example, as shown in FIG. 2, the vector direction similarity direct_sim1 and the vector modulus value similarity dis_sim1 based on d1 and d'1, the vector direction similarity direct_sim2 and the vector modulus value similarity dis_sim2 based on d2 and d'2, the vector direction similarity direct_sim3 and the vector modulus value similarity dis_sim3 based on d3 and d'3, the vector direction similarity direct_sim4 and the vector modulus value similarity dis_sim4 based on d4 and d'4, and the vector direction similarity direct_sim5 and the vector modulus value similarity dis_sim5 based on d5 and d'5 are obtained, respectively. Figure 4

[0116] Further, the second error value of the to-be-detected feature point pair is determined according to at least one vector direction similarity and at least one vector modulus value similarity.

[0117] For example, the plurality of vector direction similarities and vector modulus value similarities are processed by weighted average to eliminate the influence of the number of correct feature points in the neighborhood on the spatial similarity.

[0118] For example, the spatial similarity (as an example of the second error value) is obtained by the following formula:

[0119]

[0120] wherein error sp is the second error value, direct_sim i is the i-th vector direction similarity of the to-be-detected feature point pair, dis_sim i is the i-th vector modulus value similarity of the to-be-detected feature point pair, and i≥1.

[0121] In the implementation of the present application, different proportion of weight values can be taken for the vector direction similarity and the vector modulus value similarity to obtain the spatial similarity.

[0122] Next, the technical content of determining the third error value of the to-be-detected feature point pair according to the first error value and the second error value of the to-be-detected feature point pair in step S140 is executed.

[0123] ​Exemplarily, after obtaining the LK error (as an example of the first error value) and the spatial similarity error (as an example of the second error value), since the spatial similarity error is an error value between the to-be-detected feature point and the plurality of correct feature points, the orders of magnitude of the LK error and the spatial similarity error are inconsistent, and thus the two need to be normalized first and then weighted and averaged.

[0124] The normalization of the LK error and the spatial similarity error can be specifically implemented by using a Min-Max normalization method or a standardization method.

[0125] For example, after obtaining the normalized result, the first weight of the LK error and the second weight of the spatial similarity error are determined, the LK error and the spatial similarity error are weighted and averaged according to the first weight of the LK error and the second weight of the spatial similarity error, and a third error value is obtained.

[0126] Exemplarily, the first weight of the LK error and the second weight of the spatial similarity error can be a weight ratio of 1:1, and of course can also be other weight ratios, and the specific weight value can be adjusted according to actual needs.

[0127] Next, the technical content of step S150 of screening at least one to-be-detected feature point pair from the at least one to-be-detected feature point pair according to the third error value as a target feature point pair is executed.

[0128] Exemplarily, the third error value of the to-be-detected feature point pair that is less than or equal to the preset second error threshold is determined as the target feature point pair according to the third error value of the at least one to-be-detected feature point pair.

[0129] For example, after determining the third error values of the plurality of to-be-detected feature points, the to-be-detected feature point pair whose third error value is greater than the preset second error threshold is screened as an error feature point pair and is directly eliminated.

[0130] The to-be-detected feature point pair whose third error value is less than or equal to the preset second error threshold is screened as a correct feature point pair (as another part of an example of the target feature point pair) and is retained, and a final feature point pair used for image processing is obtained.

[0131] In the implementation manner of the present application, the preset second error threshold can be 0.5, 0.6, 0.7, etc., and can be set according to actual conditions.

[0132] In the implementation of the present application, after the initial feature point pairs are obtained by using the LK optical flow method, the to-be-detected feature point pairs are determined, the final error value (i.e., the third error value) of the to-be-detected feature point pairs is obtained based on the spatial similarity error of the to-be-detected feature point pairs, and the false matching point pairs are removed based on the third error value, thereby solving the problem that in a low-texture scene or a scene with high similarity, there are many false matching feature point pairs, which interfere with image recognition.

[0133] In another implementation of the image feature point pair screening method provided by the present application, referring to Figure 6 , the image feature point pair screening method further includes the following steps.

[0134] In S210, the matching points (i.e., the aforementioned feature point pairs) and the LK error (as an example of the first error value) are obtained based on the OpenCV LK optical flow method.

[0135] For example, the Lucas-Kanade optical flow method (i.e., the LK optical flow method) is used to match the feature point pairs. The LK error of each feature point pair is obtained by using the LK optical flow function of OpenCV, such as cv: calcOpticalFlowPyrLK.

[0136] In S220, it is determined whether the LK error is large (e.g., whether it is in the top 20% (as an example of the preset error proportion threshold)). If not, step S230 is performed, and if yes, step S240 is performed.

[0137] In S230, the matching points are accurate matching points (i.e., the aforementioned correct feature point pairs).

[0138] In S240, the matching points are to-be-calibrated matching points (i.e., the aforementioned to-be-detected feature point pairs), which need to be further verified.

[0139] For example, the first error values are sorted in descending order or ascending order, and the feature point pairs with the larger first error values, for example, the top 20%, are determined as the to-be-calibrated matching points. That is, the 80% of the feature point pairs with smaller LK error are determined as the accurate matching points.

[0140] In S250, a neighborhood window is determined with each to-be-calibrated matching point as the center.

[0141] For example, the to-be-detected feature point is taken as the center point, and a circle with a radius of r is drawn, which is the neighborhood window.

[0142] In S260, the gray centroid (i.e., the aforementioned gray centroid point) of the correct matching points in the neighborhood is calculated, the direction vector of the to-be-calibrated matching point and the gray centroid point is determined, and the correct matching points in the neighborhood are rotated so that the direction vector in the neighborhood becomes the X-axis direction.

[0143] For example, the gray centroid of the correct matching points in the neighborhood is determined according to the gray centroid method, and a direction vector (i.e., the direction of the feature point to be detected) from the feature point to be detected to the gray centroid is obtained with the feature point to be detected as the starting point and the gray centroid as the ending point. Each correct matching point in the neighborhood is rotated clockwise by an angle that is the included angle between the direction vector formed by the feature point to be detected and the gray centroid and the X axis, so that the direction vector of the neighborhood is in the X axis direction, thereby ensuring the rotation invariance of the image.

[0144] S270, the center point in the neighborhood and the correct matching points form several vectors, and the vector direction similarity and the vector modulus similarity of the matching vectors are calculated.

[0145] For example, in the neighborhood centered on the feature point to be detected, the vector direction similarity and the vector modulus similarity of the feature point to be detected and each rotated correct feature point are calculated.

[0146] S280, for each to-be-calibrated matching point, the spatial similarity (as an example of the second error value) with the correct matching points in the neighborhood is calculated: the weighted average of the direction similarity and the vector similarity.

[0147] For example, the vector direction similarity and the vector modulus similarity of a plurality of vectors formed by the feature point to be detected as the starting point and the correct matching point as the ending point are calculated and weighted averaged to obtain the spatial similarity of each feature point pair.

[0148] S290, for each to-be-calibrated matching point, the spatial similarity and the LK error are normalized respectively, and then weighted averaged as the error value (as an example of the third error value).

[0149] After obtaining the LK error and the spatial similarity error, since the spatial similarity error is the error value between the feature point to be detected and a plurality of correct feature points, the orders of magnitude of the LK error and the spatial similarity error are inconsistent, so the two are normalized respectively before defining the error function for weighted averaging to obtain the final error value.

[0150] S310, it is judged whether the error value is less than a threshold value, if not, step S320 is executed, and if yes, step S330 is executed.

[0151] S320, the calibration point is a false matching point (i.e., a false feature point pair), which is rejected.

[0152] S330, the calibration point is a correct matching point (i.e., an accurate matching point or a correct feature point pair).

[0153] For example, after determining the third error values of the to-be-detected feature points, the to-be-detected feature point pairs with third error values greater than the preset second error threshold are screened out as error feature point pairs and are removed.

[0154] The to-be-detected feature point pairs with third error values less than or equal to the preset second error threshold are screened out as correct feature point pairs and are reserved.

[0155] The specific implementation of the steps S210 to S320 can refer to the content of the steps S110 to S150.

[0156] In the implementation of the present application, after the initial matching points are obtained by using the LK optical flow method, the LK error of each matching point can be obtained, the matching points with smaller error are taken as correct matching points, and the matching points with larger error need to be further verified and are called to-be-calibrated matching points. In order to improve the matching accuracy, the similar position relationship between the matching feature points is used to establish an error function, and for each to-be-calibrated matching point, a neighborhood range containing a plurality of correct matching points is determined, the to-be-calibrated matching point and the correct matching point in the neighborhood form a plurality of vectors, the consistency of the direction of the vector and the modulus of the vector is used to establish a spatial error function (i.e. a spatial similarity error formula, also referred to as a spatial similarity error formula) of each matching point. In order to ensure the rotation invariance, before the spatial error function is established, the gray centroid point of the feature points in the neighborhood is calculated, and the to-be-calibrated matching point and the gray centroid point are connected as the direction vector of the to-be-calibrated matching point, the direction vector and the positive direction of the X axis form an angle θ, the vector formed by the to-be-calibrated matching point and the correct matching point in the neighborhood is rotated clockwise by θ, and after the rotation is completed, the spatial similarity error function is calculated again. Finally, after the LK error and the spatial similarity error are normalized, weighted average processing is performed, the final error of each to-be-calibrated matching point is obtained, and the to-be-calibrated matching points exceeding the threshold value (i.e. the second error threshold) are removed through comparison with the set threshold value.

[0157] The screening method of the image feature point pairs provided by the implementation of the present application can also be referred to as a feature point matching algorithm based on the LK optical flow method and spatial similarity. The matching point pairs are determined based on the LK optical flow method, and compared with the feature matching method based on the descriptor, the time delay is small. Moreover, the feature point matching method based on the LK optical flow method does not need to detect the feature points for each frame of image, thereby reducing the time delay. By determining the spatial similarity error of the feature point pairs, the problem of too many error matching point pairs in the scene with low texture or high similarity is solved. Moreover, based on the determination of the LK error of the feature point pairs by the LK optical flow method, a matching point verification method based on spatial similarity is proposed, and the error feature point pairs are effectively removed.

[0158] Referring to Figure 7The implementation of the present application also provides an image processing method, specifically comprising the following steps.

[0159] S410, determining a target feature point pair corresponding to a target image.

[0160] For example, the target feature point pair is obtained based on the screening method of the image feature point pair provided in the above implementation.

[0161] S420, performing image processing according to the target feature point pair.

[0162] As described above, the purpose of determining the feature point pair is to perform subsequent camera pose transformation, object tracking or spatial point three-dimensional reconstruction, etc. That is, by determining the feature point pair in two frames of images, the motion trajectory of the same object in the two frames of images is determined to perform object tracking (as an example of image processing). Or by determining the feature point pair in two frames of images, the angle at which the camera captures the two frames of images, or the current deviation angle of the camera, is determined to perform rotation processing on the images captured by the camera (as another example of image processing), or to transform the camera pose. Or by determining the feature point pair in two frames of images, the current geographical position and the current spatial point information of the vehicle are determined to perform spatial point three-dimensional reconstruction (as another example of image processing). In this way, the feature point pair determined based on the screening method of the image feature point pair described above can obtain more accurate image processing results.

[0163] Further, referring to Figure 8 , Figure 8 a structural schematic diagram of the feature point pair screening device provided by the embodiment of the present application. As Figure 8 shown, the feature point pair screening device comprises a first processing module 601, a second processing module 602, a third processing module 603, a fourth processing module 604 and a fifth processing module 605.

[0164] The first processing module 601 is configured to determine at least one feature point pair corresponding to a target image, and determine a first error value of the feature point pair.

[0165] The second processing module 602 is configured to screen at least one feature point pair from the at least one feature point pair as a to-be-detected feature point pair according to the first error value of the feature point pair.

[0166] The third processing module 603 is configured to determine a second error value of the to-be-detected feature point pair.

[0167] The fourth processing module 604 is configured to determine a third error value of the to-be-detected feature point pair according to the first error value and the second error value of the to-be-detected feature point pair.

[0168] The fifth processing module 605 is used to select at least one pair of feature points to be detected from at least one pair of feature points to be detected, based on the third error value, as the target feature point pair.

[0169] The feature point pair filtering device provided in this application embodiment can be used to execute the image feature point pair filtering method described in the above embodiment. Its implementation principle and technical effect are similar, and will not be repeated here.

[0170] Further, see Figure 9 , Figure 9 This is a schematic diagram of the structure of an image processing apparatus provided in an embodiment of this application. Figure 9 As shown, the image processing device includes: a first processing module 701 and a second processing module 702.

[0171] The first processing module 701 is used to determine the target feature point pairs corresponding to the target image.

[0172] The second processing module 702 is used to perform image processing on the target image based on the target feature point pairs.

[0173] It should be noted that the division of the various modules in the above device is merely a logical functional division. In actual implementation, they can be fully or partially integrated into a single physical entity, or they can be physically separated. Furthermore, these modules can be implemented entirely in software via processing element calls; they can be fully implemented in hardware; or some modules can be implemented by processing element calls to software, while others are implemented in hardware. For example, a processing module can be a separate processing element, or it can be integrated into a chip within the device. Alternatively, it can be stored as program code in the device's memory, and its functions can be called and executed by a processing element. The implementation of other modules is similar. Moreover, these modules can be fully or partially integrated together, or they can be implemented independently. The processing element here can be an integrated circuit with signal processing capabilities. During implementation, each step of the above method or each of the above modules can be completed through integrated logic circuits in the hardware of the processor element or through software instructions.

[0174] In the implementation of this application, the feature point pair filtering method can also be applied to electronic devices such as mobile terminals, for example, mobile phones and computers.

[0175] Figure 8 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 8 As shown, the electronic device may include: transceiver 121, processor 122, and memory 123.

[0176] The processor 122 executes computer-executable instructions stored in memory to cause the processor 122 to perform the techniques described above in the embodiments. The processor 122 can be a general purpose processor, including a central processing unit (CPU), a network processor (NP), or the like; a digital signal processor (DSP); an application-specific integrated circuit (ASIC); a field programmable gate array (FPGA); or other programmable logic component, discrete gate or transistor logic, discrete hardware components.

[0177] The memory 123 is connected with the processor 122 through the system bus and completes mutual communication, and the memory 123 is used for storing computer program instructions.

[0178] By way of example, and without limitation, memory 123 can include a hard disk drive (HDD), a floppy disk drive, a flash drive, an optical disc, a magneto-optical disc, a magnetic tape, or a universal serial bus (USB) drive or a combination of two or more of these. Storage memory 123, where appropriate, can include removable or non-removable (or fixed) media. Storage memory 123, where appropriate, can be internal or external to the integrated gateway device. In particular embodiments, storage memory 123 is non-volatile solid-state memory. In particular embodiments, storage memory 123 includes read-only memory (ROM). Where appropriate, this ROM can be mask-programmed ROM, programmable ROM (PROM), erasable PROM (EPROM), electrically erasable PROM (EEPROM), electrically alterable ROM (EAROM), or flash memory, or a combination of two or more of these.

[0179] The transceiver 121 can be used to obtain a task to be run and configuration information of the task to be run.

[0180] The system bus can be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, or the like. The system bus can be divided into an address bus, a data bus, a control bus, and the like. For ease of representation, only one thick line is shown in the figure, but it does not mean that there is only one bus or only one type of bus. The transceiver is used to realize the communication between the database access device and other computers (such as clients, read-write libraries and read-only libraries). The memory can include random access memory (RAM), and can also include non-volatile memory.

[0181] The embodiment of the present application further provides a chip for executing the image feature point pair screening method or the image processing method.

[0182] The embodiment of the present application further provides a computer readable storage medium, which stores computer instructions, and when the computer instructions are executed on a processor of an electronic device, the processor of the electronic device executes the image feature point pair screening method or the image processing method.

[0183] In some possible implementation manners, various aspects of the method provided by the present application can also be implemented in the form of a program product, which includes program codes for causing a processor of an electronic device to execute the steps in the method according to various exemplary embodiments of the present application described above in the specification, for example, the electronic device can execute the image feature point pair screening method or the image processing method.

[0184] The program product can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium may, for example, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or component, or any combination thereof. More specific examples (non-exhaustive list) of the readable storage medium include an electrical connection having one or more wires, a portable disc, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof.

[0185] The implementation manner of the present application further provides a computer program product, which includes a computer program stored in a computer readable storage medium, at least one processor can read the computer program from the computer readable storage medium, and when the at least one processor executes the computer program, the image feature point pair screening method or the image processing method can be implemented.

[0186] The computer program instructions can also be loaded onto a computer or other programmable information processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart Figure 1 one or more functions specified in the flowchart or multiple flows and / or block Figure 1 one or more blocks or multiple flows and / or blocks.

[0187] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable information processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instructions which implement the functions specified in the flowchart Figure 1 one or more functions specified in the flowchart or multiple flows and / or block Figure 1 one or more blocks or multiple flows and / or blocks.

[0188] The computer program instructions can also be loaded onto a computer or other programmable information processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart Figure 1 one or more functions specified in the flowchart or multiple flows and / or block Figure 1 one or more blocks or multiple flows and / or blocks.

[0189] Other embodiments of the application will be apparent to those skilled in the art from consideration of the specification and practice of the application disclosed herein. It is intended that the specification and examples be considered as exemplary only, with the true scope and spirit of the application being indicated by the following claims.

[0190] It will be understood that the application is not limited to the precise structures herein described and illustrated in the drawings and that various modifications and changes can be made without departing from the scope thereof.

Claims

1. A method for filtering image feature point pairs, characterized in that, Applied to electronic devices, the method includes: Determine at least one pair of feature points corresponding to the target image, and determine a first error value for the pair of feature points; Based on the first error value of the feature point pair, at least one feature point pair is selected from the at least one feature point pair as a feature point pair to be detected; Determine at least one vector direction similarity and at least one vector magnitude similarity of the feature point pair to be detected, and determine a second error value of the feature point pair to be detected based on the at least one vector direction similarity and the at least one vector magnitude similarity; Based on the first error value and the second error value of the feature point pair to be detected, a third error value of the feature point pair to be detected is determined; Based on the third error value, at least one pair of feature points to be detected is selected from the at least one pair of feature points to be detected as the target feature point pair.

2. The method for filtering image feature point pairs according to claim 1, characterized in that, The feature point pair includes a correct feature point pair, which is another feature point pair besides the feature point pair to be detected among the at least one feature point pair. The correct feature point pair includes a first correct feature point and a second correct feature point. The feature point pair to be detected includes a first feature point to be detected and a second feature point to be detected. Determining at least one vector direction similarity and at least one vector magnitude similarity of the feature point pair to be detected includes: Determine at least one first correct feature point corresponding to the first feature point to be detected and at least one second correct feature point corresponding to the second feature point to be detected; Determine the coordinates and magnitude of at least one first vector, and determine the coordinates and magnitude of at least one second vector, wherein the first vector is a vector that starts from the first feature point to be detected and ends at the first correct feature point, and the second vector is a vector that starts from the second feature point to be detected and ends at the second correct feature point. The similarity of at least one vector direction of the detected feature point pair is determined based on the coordinates of the at least one first vector and the coordinates of the at least one second vector; and The similarity of the at least one vector magnitude of the detected feature point pair is determined based on the magnitude of the at least one first vector and the magnitude of the at least one second vector.

3. The method for filtering image feature point pairs according to claim 2, characterized in that, The first vector and the second vector are obtained in the following way: Determine the first neighborhood of the first feature point to be detected and the second neighborhood of the second feature point to be detected; The first gray-scale centroid of the at least one first correct feature point in the first neighborhood and the second gray-scale centroid of the at least one second correct feature point in the second neighborhood are determined according to the gray-scale centroid method. Determine a first direction vector of the first neighborhood and a second direction vector of the second neighborhood, wherein the first direction vector is a vector starting from the first feature point to be detected and ending at the first gray-scale centroid, and the second direction vector is a vector starting from the second feature point to be detected and ending at the second gray-scale centroid. Determine the first angle between the first direction vector and the first direction axis, and determine the second angle between the second direction vector and the first direction axis; The vector starting from the first feature point to be detected and ending at the first correct feature point is rotated along the first direction by the first included angle to obtain the first vector, and the vector starting from the second feature point to be detected and ending at the second correct feature point is rotated along the first direction by the second included angle to obtain the second vector.

4. The method for filtering image feature point pairs according to claim 3, characterized in that, The vector direction similarity of the feature point pairs to be detected is obtained in the following way: in, The vector direction similarity of the pair of feature points to be detected. Let be the coordinates of the first vector. The coordinates of the second vector are given.

5. The method for filtering image feature point pairs according to claim 4, characterized in that, The similarity of the vector magnitudes of the feature point pairs to be detected is obtained in the following way: in, The similarity of the vector magnitudes of the feature point pairs to be detected. Let be the modulus of the first vector. Let be the modulus of the second vector.

6. The method for filtering image feature point pairs according to claim 5, characterized in that, The second error value is obtained in the following way: in, This is the second error value. For the first feature point pair to be detected Vector directional similarity, For the first feature point pair to be detected Similarity of vector moduli, i≥1.

7. The method for filtering image feature point pairs according to any one of claims 1-6, characterized in that, Based on the first error value and the second error value of the feature point pair to be detected, a third error value of the feature point pair to be detected is determined, including: Determine a first weight for the first error value and a second weight for the second error value; The first error value and the second error value are weighted and averaged according to the first weight and the second weight to obtain the third error value.

8. The method for filtering image feature point pairs according to any one of claims 1-6, characterized in that, Determine at least one pair of feature points corresponding to the target image, including: Determine the first target image and the second target image; The first feature point in the first target image and the second feature point in the second target image corresponding to the first feature point are determined according to the optical flow method. The feature point pair is obtained based on the first feature point and the second feature point.

9. The method for filtering image feature point pairs according to any one of claims 1-6, characterized in that, Determining the first error value of the feature point pair includes: The first error value of the feature point pair is determined based on the error function.

10. The method for filtering image feature point pairs according to any one of claims 1-6, characterized in that, Based on the first error value of the feature point pair, at least one feature point pair is selected from the at least one feature point pair as a feature point pair to be detected, including: Based on the first error value and a preset ratio threshold, at least one feature point pair corresponding to the ratio threshold is selected from the at least one feature point pair in descending order, and used as the feature point pair to be detected; or Based on the first error value and the preset first error threshold, at least one feature point pair with a first error value greater than the first error threshold is selected from the at least one feature point pair and used as the feature point pair to be detected.

11. The method for filtering image feature point pairs according to any one of claims 1-6, characterized in that, The method further includes: The other feature point pairs besides the feature point pair to be detected in the at least one feature point pair are taken as the target feature point pair.

12. The method for filtering image feature point pairs according to any one of claims 1-6, characterized in that, Based on the third error value, at least one pair of feature points to be detected is selected from the at least one pair of feature points to be detected as target feature point pairs, including: Based on the third error value and the preset second error threshold, at least one pair of the target feature points is selected from the at least one pair of target feature points whose third error value is less than or equal to the second error threshold, and is selected as the target feature point pair.

13. The method for filtering image feature point pairs according to any one of claims 1-6, characterized in that, The first error value is the optical flow error determined based on the optical flow method, and the second error value is the spatial similarity error of the feature point pairs.

14. An image processing method, characterized in that, Applied to electronic devices, the method includes: Determine the target feature point pairs corresponding to the target image, wherein the target feature point pairs are obtained by the image feature point pair filtering method according to any one of claims 1-13; Image processing is performed on the target image based on the target feature point pairs.

15. A device for filtering image feature point pairs, characterized in that, include: The first processing module is used to determine at least one pair of feature points corresponding to the target image, and to determine a first error value of the pair of feature points; The second processing module is used to select at least one feature point pair from the at least one feature point pair as a feature point pair to be detected based on the first error value of the feature point pair. The third processing module is used to determine at least one vector direction similarity and at least one vector magnitude similarity of the feature point pair to be detected, and to determine a second error value of the feature point pair to be detected based on the at least one vector direction similarity and the at least one vector magnitude similarity. The fourth processing module is used to determine the third error value of the feature point pair to be detected based on the first error value and the second error value of the feature point pair to be detected. The fifth processing module is used to select at least one pair of feature points to be detected from the at least one pair of feature points to be detected, based on the third error value, as the target feature point pair.

16. An image processing apparatus, characterized in that, include: The first processing module is used to determine the target feature point pairs corresponding to the target image, wherein the target feature point pairs are obtained by the image feature point pair filtering method according to any one of claims 1-13; The second processing module is used to perform image processing on the target image based on the target feature point pairs.

17. An electronic device, characterized in that, include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executable instructions; the processor executes the computer-executable instructions stored in the memory to implement the image feature point pair filtering method as described in any one of claims 1-13 or the image processing method as described in claim 14.

Images