Image feature point recognition method, device, electronic device and storage medium

Abstract

The present invention provides an image feature point recognition method, device, electronic device and storage medium. The image feature point recognition method includes: focusing on different areas of the target object and taking a to-be-processed image of a clear area of ​​the target object with a clarity higher than a preset threshold after each focusing; determining corresponding feature points of the to-be-processed image with an overlapping area in the overlapping area, wherein each pair of feature points in the corresponding feature points includes two feature points in the first image and the second image, respectively, in the overlapping area and with corresponding positions; determining first feature points in the target image corresponding to feature points in the clear area of ​​the non-target image according to the position of each feature point in the corresponding feature points in the respective images; determining feature points in the area of ​​the target image corresponding to the clear area of ​​the non-target image according to the first feature points. This solution can obtain more accurate image information in the target image.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to an image feature point recognition method, an image feature point recognition device, an electronic device, a storage medium and a computer program product. Background Art

[0002] The depth of field (DoF) of a camera refers to the range in which the subject can still maintain a clear image before and after the plane where the camera is in focus. In the image captured by the camera, due to the influence of the depth of field, the clear and effective image area may only occupy a part of the image. For an imaging system with a shallow depth of field, such as a shallow depth of field camera, part of the image area in the captured image will be blurred due to the short depth of field range. If the captured object is tilted, the tilted part of the object will be out of focus relative to the focal plane, which will further affect the blurred range in the captured image. The image area in the blurred range of the captured image cannot accurately provide the key image information in this part of the image area, which is not conducive to subsequent processing of the image.

[0003] Usually, the camera aperture of the imaging system can be reduced to obtain a larger depth of field to reduce the blur range in the captured image. Alternatively, the camera can be moved to shoot in sequence and the temporal correlation between images can be used to connect the optical flow fields between the captured images, thereby calibrating and restoring the image information of the blur range.

[0004] The method of reducing the camera aperture of the imaging system is only applicable to imaging systems with adjustable apertures. The method of connecting the optical flow field to restore the blurred range image requires precise focus stack alignment, the quality of which directly affects the accuracy of image restoration and requires a huge amount of calculation. Summary of the invention

[0005] The present invention has been made in view of the above-mentioned problems.

[0006] According to a first aspect of the present invention, a method for identifying image feature points is provided. The method comprises: focusing on different areas of a target object and taking images to be processed of the target object after each focusing to obtain a plurality of images to be processed, wherein each image to be processed has a clear area with a clarity higher than a preset threshold; determining corresponding feature points in an overlapping area of ​​a first image and a second image, wherein the first image is one of the images to be processed, and the second image is an image to be processed in which a clear area and a clear area in the first image have an overlapping area, and the corresponding feature points include a plurality of pairs of feature points, each pair of feature points includes two feature points in the first image and the second image, respectively, in the overlapping area, and at corresponding positions; determining first feature points in the target image corresponding to feature points in clear areas in non-target images according to the positions of each feature point in the corresponding feature points in their respective images, wherein the target image is one of the images to be processed, and the non-target image is an image other than the target image in the images to be processed; determining feature points in an area in the target image corresponding to the clear area in the non-target image according to the first feature points.

[0007] Exemplarily, the determining, in the target image, first feature points respectively corresponding to feature points of clear areas in the non-target image according to the position of each feature point in the corresponding feature points in the respective images includes:

[0008] For each non-target image,

[0009] According to the positions of corresponding feature points in every two different intermediate images, a first difference in image parameters between the intermediate images is determined, wherein the intermediate image is an image to be processed whose clear area is an intermediate clear area, and the intermediate clear area is a clear area located between the clear area of ​​the non-target image and the clear area of ​​the target image; according to the first difference, a second difference in image parameters between the non-target image and the target image is determined; according to the second difference, the feature points of the clear area in the non-target image are mapped to the target image to obtain the first feature points of the target image.

[0010] Exemplarily, the different regions are arranged along a predetermined direction and there are overlapping regions in adjacent regions of the different regions;

[0011] Determining a second difference in image parameters between the non-target image and the target image according to the first difference includes: accumulating all the first differences to determine the second difference in image parameters between the non-target image and the target image.

[0012] Exemplarily, determining the first difference in image parameters between the intermediate images based on the positions of corresponding feature points in each two different intermediate images includes: determining the variance of coordinate information between all affine transformed feature points and corresponding feature points in the second image to which all affine transformed feature points correspond, based on the difference in coordinate information between each affine transformed feature point and a corresponding feature point in the second image to which the affine transformed feature point corresponds, wherein the coordinate information of the affine transformed feature point is represented by the product of the coordinate information of the corresponding feature point in the second image corresponding to the affine transformed feature point and an affine transformation matrix, and the elements in the affine transformation matrix include unknown quantities represented by the first difference; determining the first difference that minimizes the variance; wherein the two different intermediate images include the first image and the second image, and a clear area in the first image is farther from a clear area in the target image than a clear area in the second image.

[0013] Exemplarily, determining the first difference that minimizes the variance includes: establishing an error function based on the variance; performing a line search on the error function to determine all elements in the affine transformation matrix, wherein all elements in the affine transformation matrix minimize the value of the error function; and determining the first difference based on all elements in the determined affine transformation matrix.

[0014] Exemplarily, the method further includes: using a quasi-Newton method to determine a search direction for line searching the error function.

[0015] Exemplarily, the two feature points corresponding to the position are two feature points that are closest in position and whose distance is less than a distance threshold.

[0016] Exemplarily, the method further includes: determining a clear area of ​​the image to be processed according to local contrast and / or gradient information of the image to be processed.

[0017] Exemplarily, determining, based on the first feature points, feature points in an area in the target image corresponding to a clear area in the non-target image includes:

[0018] For each first feature point,

[0019] In the case where the first feature point is located in the overlapping area of ​​different images to be processed, determining the feature point corresponding to the first feature point in the target image as the second feature point; determining the average value of the coordinates of the first feature point and the corresponding second feature point, and using the pixel point at the average value as the feature point in the area of ​​the target image corresponding to the overlapping area of ​​the different images to be processed;

[0020] If the first feature point is outside the overlapping area of ​​the different images to be processed, the first feature point is used as the feature point of the target image.

[0021] According to a second aspect of the present invention, there is also provided an image feature point recognition device, comprising:

[0022] An image acquisition module, for focusing on different areas of a target object and taking an image to be processed of the target object after each focusing, so as to obtain a plurality of images to be processed, wherein each image to be processed has a clear area with a clarity higher than a preset threshold;

[0023] a feature point matching module, configured to determine corresponding feature points in an overlapping area of ​​a first image and a second image, wherein the first image is one of the images to be processed, and the second image is an image to be processed in which a clear area overlaps with a clear area in the first image, and the corresponding feature points include a plurality of pairs of feature points, each pair of feature points includes two feature points in the first image and the second image, respectively, in the overlapping area and at corresponding positions;

[0024] A feature point recognition module, used to determine, in a target image, first feature points respectively corresponding to feature points of a clear area in a non-target image according to the position of each feature point in the corresponding feature points in the respective images, wherein the target image is one of the images to be processed, and the non-target image is another image in the images to be processed except the target image;

[0025] The feature point selection module is used to determine the feature points in the area of ​​the target image corresponding to the clear area in the non-target image based on the first feature points.

[0026] According to a third aspect of the present invention, there is also provided an electronic device, comprising: a processor and a memory, wherein the memory stores computer program instructions, and the computer program instructions are used to execute the above-mentioned image feature point recognition method when the processor is running.

[0027] According to a fourth aspect of the present invention, a storage medium is further provided, on which program instructions are stored, and the program instructions are used to execute the above-mentioned image feature point recognition method when running.

[0028] According to a fifth aspect of the present invention, there is further provided a computer program product, comprising computer program instructions, wherein the computer program instructions are used to execute the above-mentioned image feature point recognition method when running.

[0029] In the above technical solution, different areas of the target object are focused respectively, and after each focusing, an image to be processed with a clear area of ​​the target object having a clarity higher than a preset threshold is captured; corresponding feature points of the image to be processed with an overlapping area in the overlapping area are determined, and each pair of feature points in the corresponding feature points includes two feature points in the first image and the second image, respectively, in the overlapping area, and with corresponding positions; first feature points corresponding to feature points in the clear area of ​​the non-target image are determined in the target image according to the position of each feature point in the corresponding feature points in the respective images; and feature points in the area of ​​the target image corresponding to the clear area of ​​the non-target image are determined according to the first feature points. In this way, more feature points can be determined in the target image according to the correspondence between feature points in different images to be processed, so as to obtain more accurate image information of the target image.

[0030] The above description is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention, it can be implemented according to the contents of the specification. In order to make the above and other purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] The above and other purposes, features and advantages of the present invention will become more apparent by describing the embodiments of the present invention in more detail in conjunction with the accompanying drawings. The accompanying drawings are used to provide a further understanding of the embodiments of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings, the same reference numerals generally represent the same components or steps.

[0032] Figure 1 A schematic flow chart of an image feature point recognition method according to an embodiment of the present invention is shown;

[0033] Figure 2 A schematic diagram showing a first image and a second image to be processed according to an embodiment of the present invention;

[0034] Figure 3 A schematic flow chart of determining first feature points in a target image that respectively correspond to feature points of a clear area in a non-target image according to an embodiment of the present invention is shown;

[0035] Figure 4 A schematic diagram showing a method of determining first feature points in a target image that correspond to feature points of a clear area in a non-target image, according to an embodiment of the present invention;

[0036] Figure 5A schematic flow chart of determining a first difference in image parameters between intermediate images according to positions of corresponding feature points in every two different intermediate images according to one embodiment of the present invention is shown;

[0037] Figure 6 A schematic flow chart of determining a first difference that minimizes the variance according to an embodiment of the present invention is shown;

[0038] Figure 7 A schematic flow chart of determining a first difference that minimizes the variance according to yet another embodiment of the present invention is shown;

[0039] Figure 8 A schematic flow chart showing determining feature points in an area in a target image corresponding to a clear area in a non-target image according to a first feature point according to an embodiment of the present invention is shown;

[0040] Fig. 9 A schematic block diagram showing an image feature point recognition device according to an embodiment of the present invention; and

[0041] Fig.10 A schematic block diagram of an electronic device according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0042] In order to make the purpose, technical scheme and advantages of the present invention more obvious, the exemplary embodiments according to the present invention will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments of the present invention, and it should be understood that the present invention is not limited to the exemplary embodiments described herein. Based on the embodiments of the present invention described in the present invention, all other embodiments obtained by those skilled in the art without creative work should fall within the protection scope of the present invention.

[0043] After the camera captures the image of the target object, subsequent processing can be performed based on the captured image, for example, camera calibration, camera image distortion correction and other processing processes can be performed based on the captured calibration plate image. When using the captured image to perform these processing processes, it is usually necessary to obtain accurate image information of the image, such as pixel distribution, gradient information, corner point position, etc. However, due to the limitations of shooting conditions, there may be blurred areas in the image of the target object. For example, when a shallow depth of field camera is used to shoot the target object, due to the short depth of field range, there will be blurred areas in the image of the target object, and the image information in these blurred areas cannot provide accurate key information, which will affect the accuracy of subsequent processing of the image.

[0044] In order to at least partially solve the above problems, an image feature point recognition method is proposed. The method focuses on different areas of the target object respectively and shoots a to-be-processed image with a clear area of ​​the target object having a clarity higher than a preset threshold after each focusing. The continuously shot to-be-processed images with overlapping and clear features / clear areas (these feature areas have one or more feature points) are screened out. Then, the corresponding feature points of the to-be-processed image with overlapping areas in the overlapping area are determined, wherein the first image is one of the to-be-processed images, the second image is the to-be-processed image in which there is an overlapping area between the clear area and the clear area in the first image, the corresponding feature points include multiple pairs of feature points, and each pair of feature points in the corresponding feature points includes two feature points in the first image and the second image, which are in the overlapping area and have corresponding positions; according to the position of each feature point in the corresponding feature points in the respective images, the first feature points corresponding to the feature points of the clear area in the non-target image are determined in the target image, wherein the target image is one of the to-be-processed images, and the non-target image is the other image in the to-be-processed image except the target image; according to the first feature points, the feature points in the area in the target image corresponding to the clear area in the non-target image are determined. In this way, more accurate image information in the target image can be obtained when there is a blurred area in the target image.

[0045] Figure 1 FIG. 2 shows a schematic flow chart of an image feature point recognition method according to an embodiment of the present invention. Figure 1 As shown, the image feature point recognition method may include steps S110 to S140.

[0046] In step S110, focusing on different areas of the target object is performed and an image to be processed of the target object is captured after each focusing to obtain a plurality of images to be processed, wherein each image to be processed has a clear area with a clarity higher than a preset threshold.

[0047] Different areas of the target object can be different areas of the entire target object, or different sub-areas in a designated area of ​​the target object. The shooting target of multiple images to be processed is the same area of ​​the target object, so different images to be processed can be images of similar size, and each image to be processed should contain the entire target object or the designated area of ​​the target object to be photographed. Because different areas of the target object are focused on, different clear areas should exist in different images to be processed, and the clear areas of each image to be processed correspond to different areas of the target object. For example, the area of ​​the target object to be photographed can be designated as the first area, and the first area can be the entire target object or a partial area of ​​the entire target object. Because different sub-areas of the first area are focused and photographed, the clear areas of each image to be processed correspond to different sub-areas of the first area.

[0048] The center of the clear area of ​​each image to be processed may be the same as the center of different areas of the target object focused when the image to be processed is captured. The image to be processed may be a color image, a grayscale image, a binary image, or the like.

[0049] For example, for a target object, the distance between the camera and the target object can be fine-tuned to accurately focus and shoot different positions of the target object. In this way, multiple images to be processed can be obtained, each of which contains a local clear area with a clarity higher than a preset threshold.

[0050] The characteristics of a clear area are clear edges and rich texture information. On the contrary, a fuzzy area has unclear edges and missing detail information. Therefore, the clarity of different areas in each image to be processed can be characterized according to parameters such as local contrast, structural similarity index, and mean square error of each image to be processed, and the area with clarity higher than a preset threshold is determined as part of the clear area to obtain a complete clear area in each image to be processed.

[0051] Exemplarily, the clear area of ​​the image to be processed may be determined according to the local contrast and / or gradient information of the image to be processed.

[0052] For example, the gradient of the image area of ​​the image to be processed can be calculated. A high gradient value usually means that there are obvious edges or texture features in the partial image area, so the partial image area is clearer. The image area with a gradient value higher than a preset gradient threshold can be determined as a clear area. For another example, the local contrast of the image area in the image to be processed can be determined based on the variance of the image information in the local area of ​​the image to be processed. The larger the variance, the higher the local contrast of the area. When the local contrast of the image area exceeds the preset threshold, the part of the image area that is not a clear area can be determined.

[0053] Optionally, the frequency domain characteristics of the image to be processed can be quantitatively evaluated by analyzing the Fourier transform of the image to be processed, so as to determine the clear area. In the frequency domain, the image area of ​​the clear area tends to have higher high-frequency components.

[0054] In step S120, corresponding feature points in the overlapping area of ​​the first image and the second image are determined, wherein the first image is one of the images to be processed, and the second image is the image to be processed in which a clear area overlaps with a clear area in the first image, and the corresponding feature points include multiple pairs of feature points (multiple pairs of feature point pairs), and each pair of feature points (feature point pair) includes two feature points in the first image and the second image, respectively, which are in the overlapping area and have corresponding positions.

[0055] The first image and the second image may be a pair of images to be processed in which the clear regions have an overlapping region, and the first image and the second image are respectively one of the images to be processed. The overlapping region is the overlapping region of the first image and the second image.

[0056] Figure 2 FIG. 4 is a schematic diagram showing a first image and a second image to be processed according to an embodiment of the present invention. Figure 2 As shown, different images to be processed may be images of similar or even identical size, and when the image parameters in each image to be processed are consistent, the area of ​​the target object captured in each image to be processed is substantially consistent. Therefore, it is possible to first determine the position of the clear area of ​​the first image and the position of the clear area of ​​the second image in the first image, and then determine the pixel coordinates of each position to determine the overlapping area between the clear area of ​​the first image and the clear area of ​​the second image as the overlapping area of ​​the first image and the second image.

[0057] For example, Figure 2 As shown, the overlapping area between the clear area of ​​the first image and the clear area of ​​the second image can be determined first, and then the first image area corresponding to the overlapping area in the clear area of ​​the first image and the second image area corresponding to the clear area of ​​the second image can be determined. Then, a feature point corresponding to a feature point position in the first image area can be determined in the second image area, and these two points are used as a pair of feature points in the corresponding feature points. In other words, the overlapping area of ​​the first image and the second image can be used to determine the different positions of the same feature point in the first image and the second image, and the same feature point at different positions is called a pair of feature points. Therefore, when there are multiple feature points in the first image area, the corresponding position of each feature point in the second image can be determined, that is, a feature point corresponding to its position can be determined in the second image area based on each feature point, so as to obtain multiple pairs of feature points corresponding to the positions as corresponding feature points.

[0058] Exemplarily, the above-mentioned feature point pair refers to two feature points that are located closest to each other and whose distance is less than a distance threshold.

[0059] Because the area of ​​the target object captured in each image to be processed is basically the same, when there is an overlap between the clear area of ​​the first image and the clear area of ​​the second image, the feature points corresponding to the positions of the first image and the second image in the overlapping area will also be basically the same. Even if it may be affected by the difference in the object distance of the camera, the two feature points closest to each other are very likely to belong to the corresponding feature points (feature point pair). Therefore, the two feature points closest to each other can be regarded as the two feature points corresponding to each other.

[0060] When the distance between the two closest feature points is far, it means that there is a large difference between the regions of the target object captured in the first image and the second image. In this case, it is easy to identify multiple pairs of feature points that should not be corresponding feature points as corresponding feature points. Therefore, a distance threshold can also be set to use the two closest feature points whose distance is less than the distance threshold as the two feature points corresponding to the position.

[0061] It can be understood that for a pair of images to be processed with overlapping areas of any clear areas, the image to be processed can be processed according to step S120 to determine the position of each feature point in the corresponding feature points of the overlapping areas of the images to be processed in the respective images.

[0062] In step S130, first feature points corresponding to feature points of clear areas in non-target images are determined in the target image according to the position of each feature point in the respective images. The target image is one of the images to be processed, and the non-target image is the image other than the target image in the images to be processed.

[0063] Different clear areas correspond to different areas of the target object of the image to be processed, so the image information in the clear area of ​​the non-target image will include the image information of the non-clear area of ​​the target image. The feature points in the non-clear area of ​​the target image can be determined based on the feature points in the clear area of ​​the non-target image.

[0064] For example, when the image parameters of the non-target image and the target image are consistent, the regions of the target object captured in the non-target image and the target image are substantially consistent. Feature points can be determined in the clear region of the non-target image, and then based on the positions of the feature points in the non-target image, points at the same positions in the target image are determined as first feature points.

[0065] For example, the size and direction of the target object areas in each image to be processed may be inconsistent and deflected, and at this time, the position of the feature points of the non-target image cannot be directly determined in the target image. The difference in image parameters between the non-target image and the target image can be determined first, and then the mapping relationship between the area of ​​the target object in the non-target image and the area of ​​the target object in the target image can be determined based on the difference. According to the mapping relationship, when the image parameters of the non-target image and the image parameters of the target image are the same, the corresponding position of the feature points of the clear area of ​​the non-target image in the target image can be determined, and then the point at the same position as the position in the target image can be determined as the first feature point.

[0066] The clear area of ​​the non-target image and the clear area of ​​the target avatar overlap directly or indirectly. For example, when the non-target image is the first image and the target image is the second image, the clear area of ​​the non-target image and the clear area of ​​the target avatar overlap directly. When the non-target image is the first image and the target image is the third image, the clear area of ​​the non-target image and the clear area of ​​the target avatar overlap indirectly. There is an overlapping area between the clear area of ​​the third image and the clear area of ​​the second image, but there is no overlapping area between the clear area of ​​the third image and the clear area of ​​the non-target image. Only in this way can the first feature point be determined in the target avatar according to the position of each feature point in the corresponding feature points in their respective images.

[0067] For example, an image area requiring a first feature point can be determined in the target image according to the requirements. Then, based on the relationship between the image area and the clear area of ​​the image to be processed, part of the image to be processed is selected as a non-target image, and the first feature point in the image area that meets the requirements in the target image is determined only based on the feature points of the clear area of ​​these non-target images. The image area corresponding to the total image area of ​​the clear area of ​​the selected non-target image in the target image should cover the image area outside the non-clear area of ​​the first image where the first feature point needs to be determined.

[0068] In step S140, feature points in an area in the target image corresponding to a clear area in the non-target image are determined based on the first feature points.

[0069] There may be feature points corresponding to the first feature point in the target image, but only one feature point is needed. Therefore, after obtaining the first feature point, the first feature point can be screened according to actual needs to determine the feature points in the target image corresponding to the clear area in the non-target image. For example, the first feature point and its corresponding point in the first image can be merged into one feature point.

[0070] Optionally, first feature points respectively corresponding to feature points of a clear area in a non-target image may be determined only in a fuzzy area of ​​the target image. The fuzzy area is an image area outside the clear area in the target image.

[0071] In the above technical solution, different areas of the target object are focused respectively, and after each focusing, an image to be processed of a clear area of ​​the target object with a clarity higher than a preset threshold is captured. Then, the corresponding feature points of the image to be processed in the overlapping area where there is an overlapping area are determined, and each pair of feature points in the corresponding feature points includes two feature points in the first image and the second image, respectively, in the overlapping area, and at corresponding positions. Then, according to the position of each feature point in the corresponding feature points in the respective images, the first feature points corresponding to the feature points of the clear area in the non-target image are determined in the target image, and finally, according to the first feature points, the feature points in the area in the target image corresponding to the clear area in the non-target image are determined. In this way, more accurate image information in the target image can be obtained.

[0072] Figure 3 FIG. 4 is a schematic flow chart of determining first feature points in a target image that correspond to feature points in a clear area in a non-target image according to an embodiment of the present invention. Figure 3 As shown, the above step S130 may include steps S310 to S330.

[0073] In step S310, for each non-target image, the first difference in image parameters between the intermediate images is determined according to the positions of corresponding feature points in each two different intermediate images, where the intermediate image is an image to be processed whose clear area is an intermediate clear area, and the intermediate clear area is a clear area located between the clear area of ​​the non-target image and the clear area of ​​the target image.

[0074] The corresponding points in the two different intermediate images are located in the overlapping area of ​​the two different intermediate images. When the image parameters of the two different intermediate images are the same, because the same area of ​​the target object is photographed, the area of ​​the target object in the two different intermediate images to be processed is usually the same. At this time, the positions of the corresponding feature points of the two different intermediate images to be processed in the overlapping area should also be the same. For example, if the images to be processed related to the overlapping area are the third image and the fourth image, and the coordinates of the feature point a in the third image are (x1, y1), then the coordinates of the feature point b corresponding to the position a in the fourth image are also (x1, y1).

[0075] However, due to changes in shooting angle and position, the image parameters of two different intermediate images may be different. At this time, the corresponding feature points of the two different intermediate images may be scattered in different positions in the overlapping area. For example, two different intermediate images taken by a camera at two focus planes may have different magnifications due to different focus distances, that is, inconsistent object distances. When the camera focuses, if the direction of the camera focus movement is not parallel to the camera orientation, there may also be pixel translations in the X and Y directions between the two images. If the camera rotates when taking two different intermediate images, it is also necessary to estimate the rotation angle between the two intermediate images.

[0076] Exemplarily, the image parameters include image magnification, image zoom center, and pixel shift. For example, the X-direction magnification of the third image is 1, and the coordinates of the feature point a in the third image are (1,1). The X-direction magnification of the fourth image is 2, and the coordinates of the feature point b corresponding to the position a in the fourth image are (2,1). At this time, there will be a position difference between each pair of corresponding feature points in the corresponding feature points, and the difference in image parameters between the third image and the fourth image will affect the position difference. The difference in image parameters can be determined in turn based on the position difference. Similarly, in addition to the image magnification, the image zoom center and the image shift will also affect the position of the corresponding feature points in the overlapping area.

[0077] An affine transformation matrix representing the first difference in image parameters between the third image and the fourth image can be determined based on the influence of the image parameters on the positions of the corresponding feature points and the position difference between each pair of feature points in the overlapping area between the clear area of ​​the third image and the clear area of ​​the fourth image. The affine transformation matrix can also be used to determine the positions where the feature points in the corresponding feature points located in the third image should be located when the image parameters of the third image are the image parameters of the fourth image.

[0078] Exemplarily, when there is a difference in image magnification and pixel translation between the third image and the fourth image, the affine transformation matrix A may be determined according to the following formula 1:

[0079]

[0080] in, ( x,y ) represents the pixel coordinates of the corresponding feature point in the third image, ( x′,y′ ) represents the corresponding ( x,y ), a x represents the difference in image magnification in the X direction between the third image and the fourth image, a yrepresents the difference in image magnification in the Y direction between the third image and the fourth image, ( x 0 ,y 0 ) is the image zoom center, Δ x represents the horizontal coordinate offset between the third image and the fourth image, Δ y Indicates the vertical coordinate offset between the third image and the fourth image.

[0081] There are 4 unknown elements in the affine transformation matrix A of the above formula 1: a x 、a y 、a x (Δ x-x 0 )+ x 0 、a x (Δ y- y 0 ) + y 0 In order to determine the four unknown elements to obtain the above-mentioned affine transformation matrix A, the corresponding feature points in the overlapping area of ​​the third image and the fourth image include at least four pairs of feature points with corresponding positions.

[0082] The coordinates of the feature points corresponding to multiple pairs of positions can be substituted into the above formula 1 and stacked to obtain the feature points corresponding to the multiple pairs of positions and the mapping equations corresponding to the above formula 1. Then, the unknown elements of the affine transformation matrix A can be fitted using the least squares method.

[0083] The pixel coordinates of the corresponding feature points in the third image after the first difference is compensated can also be determined according to the product between the affine transformation matrix A and the pixel coordinates of the corresponding feature points in the third image. Then, the unknown elements of the affine transformation matrix A are determined according to the difference between the pixel coordinates and the pixel coordinates of the corresponding feature points in the fourth image.

[0084] In step S320, a second difference in image parameters between the non-target image and the target image is determined based on the first difference.

[0085] The two intermediate images may be a non-target image and a target image. When there is an overlapping area between a clear area of ​​the non-target image and a clear area of ​​the target image, the first difference in image parameters between the non-target image and the target image may be directly used as the second difference in image parameters between the non-target image and the target image.

[0086] When there is no overlapping area between the clear area of ​​the non-target image and the clear area of ​​the target image, there are no corresponding feature points for the non-target image and the target image. An intermediate clear area that directly or indirectly overlaps with the clear area of ​​the non-target image and directly or indirectly overlaps with the clear area of ​​the target portrait can be determined, and then the second difference in image parameters between the non-target image and the target image is determined based on the first difference between some intermediate images in the intermediate images where these intermediate clear areas are located. Among them, as few intermediate images as possible can be selected to determine as few first differences as possible and reduce the amount of calculation. It is also possible to select as many intermediate images as possible to determine as many first differences as possible and improve the accuracy of the second difference.

[0087] Exemplarily, when different regions of the target object are arranged along a predetermined direction and there are overlapping regions in adjacent regions of the different regions in step S110, all first differences may be accumulated to determine a second difference in image parameters between the non-target image and the target image.

[0088] It can be understood that, because the clear areas of each image to be processed correspond to different areas of the target object, when there are overlapping areas in adjacent areas of different areas, all intermediate clear areas directly or indirectly overlap with the clear areas of the non-target image, and directly or indirectly overlap with the clear areas of the target image. The first difference between each two intermediate images closest to the intermediate clear area can be determined pair by pair along the direction from the non-target image to the target image, and then all the first differences are directly accumulated to determine the second difference of the image parameters between the non-target image and the target image. In this way, there is no need to screen the intermediate images, reducing the corresponding amount of calculation.

[0089] In step S330 , according to the second difference, the feature points of the clear area in the non-target image are mapped to the target image to obtain first feature points of the target image.

[0090] When determining the second difference, an affine transformation matrix representing the second difference can be determined according to a transformation matrix representing the first difference used to determine the second difference. The feature points of the clear area in the non-target image are then mapped to the target image using the affine transformation matrix representing the second difference to obtain the first feature points of the target image.

[0091] Figure 4 A schematic diagram of determining first feature points in a target image that respectively correspond to feature points of clear areas in a non-target image according to an embodiment of the present invention is shown.

[0092] like Figure 4As shown, there is an overlapping area (first overlapping area) between the clear area of ​​the target image and the clear area of ​​the third image, there is an overlapping area (second overlapping area) between the clear area of ​​the target image and the clear area of ​​the fourth image, there is an overlapping area (third overlapping area) between the clear area of ​​the third image and the clear area of ​​the fourth image, and there is an overlapping area (fourth overlapping area) between the clear area of ​​the non-target image and the clear area of ​​the fourth image.

[0093] For example, the affine transformation matrix A1 representing the first difference between the fourth image and the target image can be determined according to the above formula 1 and the corresponding feature points of the target image and the fourth image in the second overlapping area. Similarly, the affine transformation matrix A2 representing the first difference between the non-target image and the fourth image can be determined. At this time, the affine transformation matrix A = A1 A2 representing the non-target image and the fourth image can be obtained. Using the affine transformation matrix A and the feature points of the clear area in the non-target image, the first feature points corresponding to the feature points of the clear area in the non-target image can be determined in the target image.

[0094] For example, an affine transformation matrix A3 representing the first difference between the third image and the target image can also be determined based on the above formula 1 and the corresponding feature points of the clear area of ​​the target image and the third image in the first overlapping area. Similarly, an affine transformation matrix A2 representing the first difference between the non-target image and the fourth image, and an affine transformation matrix A4 representing the first difference between the fourth image and the third image can be determined. At this time, an affine transformation matrix A = A3 A4 A2 representing the non-target image and the fourth image can be obtained. Using the affine transformation matrix A and the feature points of the clear area in the non-target image, first feature points corresponding to the feature points of the clear area in the non-target image can be determined in the target image.

[0095] In the above technical solution, for each non-target image, the first difference of the image parameters between the intermediate images is determined according to the positions of the corresponding feature points in each of the two different intermediate images, and then the second difference of the image parameters between the non-target image and the target image is determined according to the first difference, and then the feature points of the clear area in the non-target image are mapped to the target image according to the second difference to obtain the first feature points of the target image. In this way, more accurate first feature points can be determined in the target avatar according to the feature points of the clear areas of different images to be processed, which is conducive to accurately determining more image information of the target image.

[0096] Figure 5 FIG. 1 is a schematic flow chart of determining a first difference in image parameters between intermediate images according to the positions of corresponding feature points in each of two different intermediate images according to an embodiment of the present invention. Figure 5As shown, the above step S310 may include steps S510 to S520.

[0097] In step S510, the variance of the coordinate information between all the affine transformed feature points and the feature points corresponding to all the affine transformed feature points in the second image is determined based on the difference in coordinate information between each affine transformed feature point and the feature point in the second image to which the affine transformed feature point corresponds, wherein the coordinate information of the affine transformed feature point is represented by the product of the coordinate information of the corresponding feature point in the second image corresponding to the affine transformed feature point and the affine transformation matrix, and the elements in the affine transformation matrix include the unknown quantity represented by the first difference.

[0098] It can be understood that the corresponding feature points in the first image and the corresponding feature points in the second image are positionally corresponding, and the corresponding feature points in the first image and the feature points after affine transformation are also positionally corresponding, so the feature points after affine transformation and the corresponding feature points in the second image are also corresponding. For example, the corresponding feature point a1 in the first image corresponds to b1 in the second image, and the feature point a2 after affine transformation determined by the product of the coordinate information of a1 and the affine transformation matrix also corresponds to b1.

[0099] The two different intermediate images include a first image and a second image, and the clear area in the first image is farther from the clear area in the first image than the clear area in the second image. In this way, the difference in image parameters of the second image relative to the first image can be determined.

[0100] The affine transformation matrix may be the affine transformation matrix A in the above formula 1, and the unknown quantity represented by the first difference may be the unknown element in the above affine transformation matrix A.

[0101] Exemplarily, the coordinate information of the feature points after affine transformation can be determined according to the affine transformation matrix A and the following formula 2:

[0102]

[0103] in, ( u i ，v i ) represents the pixel coordinates of the corresponding feature point in the first image, ( u" i ，v " i ) represents the pixel coordinates of the feature point after affine transformation corresponding to the feature point in the second image.

[0104] For example, when the corresponding feature points in the overlapping area of ​​the first image and the second image include N pairs of feature points corresponding to positions, after the feature points after affine transformation are determined according to the above formula 2, the variance M can be determined according to the following formula 3:

[0105]

[0106] In step S530, a first difference that minimizes the variance is determined.

[0107] When the variance reaches a minimum value, it means that for all corresponding feature points, the affine transformation matrix representing the first difference can make the pixel coordinates of the feature points after affine transformation closest to the pixel coordinates of the corresponding feature points in the second image.

[0108] In the above technical solution, the coordinate information of the affine transformed feature points corresponding to the corresponding feature points in the second image is represented by the product of the coordinate information of the corresponding feature points in the first image and the affine transformation matrix including the unknown quantity represented by the first difference, and then the variance of the coordinate information between all the affine transformed feature points and the feature points in the second image to which the affine transformed feature points correspond is determined according to the difference in coordinate information between each affine transformed feature point and the feature point in the second image to which the affine transformed feature point corresponds, and then the first difference that minimizes the variance is determined. In this way, the first difference that allows the corresponding feature points of the first image to be mapped to the feature points corresponding to the positions in the second image as much as possible can be determined, and thus the accuracy of the first feature points can be improved.

[0109] Figure 6 FIG. 4 is a schematic flow chart of determining a first difference that minimizes the variance according to an embodiment of the present invention. Figure 6 As shown, the above step S530 may include steps S610 to S630.

[0110] In step S610, an error function is established according to the variance.

[0111] For example, the error function can be determined according to the above formula 3 and the following formula 4: min [ f ( a u ,a v ,u 0 , v 0 , Δ u, Δ v )]:

[0112]

[0113] In step S620, a line search is performed on the error function to determine all elements in the affine transformation matrix, wherein all elements in the affine transformation matrix minimize the value of the error function.

[0114] In order to solve the equation 4 a u ,a v ,u 0 ,v 0 , Δ u, Δ v , the parameter vector for line search can be determined according to the following formula 5: :

[0115]

[0116] Assume the search step is , the search direction is , the minimum point can be found after multiple searches Among them, the minimum point The following formula 6 should be satisfied:

[0117]

[0118] Search step length Δ λ k The selection adopts the line search method, first let , which contains the minimum point . (Δ λ k ), if present ∈[ a,b ], (Δ λ k )exist[ a , ] monotonically decreasing, (Δ λ k )exist[ , b ] increases monotonically. In the interval [ a,b ] Select the initial point Δ λ 0 >0 and initial step size γ 0 >0. From Δ λ 0 Start Search γ 0 ,if (Δ λ 0 +γ 0)< (Δ λ 0 ), then increase γ 0 Continue searching. If in the first step, (Δ λ 0 +γ 0 )> (Δ λ 0 ), then use - γ 0 to search.

[0119] Exemplarily, the quasi-Newton method may be used to determine the search direction for line searching the error function.

[0120] The search direction can be determined according to the following formula 7 :

[0121]

[0122] in, Indicates the descending direction, that is, the current search direction , is the first-order derivative (gradient) at the current point, is the inverse of the approximate Hessian matrix.

[0123] Figure 7 A schematic flow chart of determining a first difference that minimizes the variance according to yet another embodiment of the present invention is shown.

[0124] like Figure 7 As shown, during initialization, the initial search position is selected as , which can be a u and a v Both are 1, ( u 0, v 0) represents the coordinates of the center point of the image, Δ u and Δ v Set to 0, The identity matrix can be used as the initial value of the inverse of the approximate Hessian matrix. If the minimum point is found at this time , then according to the parameter vector corresponding to the minimum point Determine the affine transformation matrix. Otherwise, calculate the descent direction Then follow the direction Search and use the above formula 6 to determine the search step length Δ λ k >0, let . At the same time, correct the parameters To obtain the corrected For example, the DFP (Davidon-Fletcher-Powell) algorithm and the BFGS (Broyden-Fletcher-Goldfarb-Shanno) algorithm can be used to correct If the minimum point is found at this time , then according to the minimum point The corresponding parameter vector Determine the affine transformation matrix. If the minimum point is not found , k value is increased by 1, so that the modified value is used as the new parameter The above line search process is repeated until the minimum point is determined. , thereby determining all elements in the affine transformation matrix that minimize the value of the error function.

[0125] In step S630, a first difference is determined according to all elements in the determined affine transformation matrix.

[0126] After all elements of the affine transformation matrix are determined, the affine transformation matrix representing the first difference can be determined. Thus, the first difference can be determined.

[0127] In the above technical solution, an error function is established according to the variance based on all elements in the determined affine transformation matrix, and then a line search is performed on the error function to determine the element in the affine transformation matrix that minimizes the value of the error function, and then the first difference is determined. In this way, the optimal first difference can be determined more accurately.

[0128] Figure 8 FIG. 1 is a schematic flow chart showing how to determine feature points in an area in a target image corresponding to a clear area in a non-target image based on a first feature point according to an embodiment of the present invention. Figure 8 As shown, the above step S140 may include steps S810 to S830.

[0129] In step S810, for each first feature point, if the first feature point is located in an overlapping area of ​​different images to be processed, a feature point corresponding to the first feature point in the target image is determined as a second feature point.

[0130] The feature points located in the overlapping area of ​​different images to be processed are corresponding feature points. However, due to the difference in image parameters between different images to be processed, each pair of corresponding feature points are two feature points with corresponding positions. Therefore, when the first feature point is located in the overlapping area of ​​different images to be processed, there may be multiple first feature points corresponding to the same position of the first image. Taking the figure as an example, when the non-target image includes the third image and the fourth image, a3 is a feature point in the second overlapping area of ​​the third image, and a4 is a feature point in the second overlapping area of ​​the fourth image corresponding to the position of a3. The first feature point in the target image corresponding to the feature point a3 in the second overlapping area of ​​the third image is a5, and the first feature point in the target image corresponding to the feature point a4 in the second overlapping area of ​​the fourth image is a6. Because a4 and a3 are feature points with corresponding positions, a5 and a6 are also corresponding feature points. At this time, a6 can be used as the second feature point corresponding to the first feature point a5.

[0131] In step S820, the average value of the coordinates of the first feature point and its corresponding second feature point is determined, so that the pixel point at the average value is used as the feature point in the area of ​​the target image corresponding to the overlapping area of ​​different images to be processed.

[0132] For example, for the first feature point a5 with coordinates (m1, n1) and the second feature point a6 with coordinates (m2, n2), we can use The pixel points replace a5 and a6 as the feature points in the target image.

[0133] Optionally, when there are multiple second feature points corresponding to the first feature point, the median of the coordinates of the second feature point corresponding to the first feature point can be determined, so that the pixel point at the median can be used as the feature point in the area in the target image corresponding to the overlapping area of ​​different images to be processed.

[0134] In step S830, if the first feature point is outside the overlapping area of ​​different images to be processed, the first feature point is used as a feature point of the target image.

[0135] When the first feature point is outside the overlapping area of ​​different images to be processed, there is no second feature point corresponding to the first feature point, so the first feature point can be directly used as the feature point of the target image.

[0136] In the above technical solution, for each first feature point, if the first feature point is located in the overlapping area of ​​different images to be processed, the corresponding feature point of the first feature point in the target image is determined as the second feature point, and then the average value of the coordinates of the first feature point and its corresponding second feature point is determined, so that the pixel point at the average value is used as the feature point in the area corresponding to the overlapping area of ​​different images to be processed in the target image. If the first feature point is outside the overlapping area of ​​different images to be processed, the first feature point is used as the feature point of the target image. In this way, it is possible to avoid using multiple first feature points to represent the pixel point at the same position in the target image, and prevent the image information in the determined target image from being inaccurate.

[0137] Fig. 9 FIG. 4 is a schematic block diagram of an image feature point recognition device according to an embodiment of the present invention. Fig. 9 As shown, the image feature point recognition device includes an image acquisition module 910 , a feature point matching module 920 , a feature point recognition module 930 , and a feature point selection module 940 .

[0138] The image acquisition module 910 is used to focus on different areas of the target object and take a to-be-processed image of the target object after each focusing to obtain a plurality of to-be-processed images, wherein each to-be-processed image has a clear area with a clarity higher than a preset threshold.

[0139] The feature point matching module 920 is used to determine the corresponding feature points in the overlapping area of ​​the first image and the second image, wherein the first image is one of the images to be processed, and the second image is the image to be processed in which there is an overlapping area between the clear area and the clear area in the first image. The corresponding feature points include multiple pairs of feature points, and each pair of feature points includes two feature points in the first image and the second image, which are in the overlapping area and have corresponding positions.

[0140] The feature point recognition module 930 is used to determine the first feature points in the target image that correspond to the feature points of the clear area in the non-target image according to the position of each feature point in the corresponding feature points in the respective images. The target image is one of the images to be processed, and the non-target image is the other image in the images to be processed except the target image.

[0141] The feature point selection module 940 is used to determine, based on the first feature point, feature points in an area in the target image corresponding to a clear area in the non-target image.

[0142] Exemplarily, the feature point recognition module 930 includes a first difference determination submodule, a second difference determination submodule and a first feature point determination submodule. The first difference determination submodule is used to determine, for each non-target image, a first difference in image parameters between intermediate images according to the positions of corresponding feature points in each two different intermediate images, wherein the intermediate image is an image to be processed whose clear area is an intermediate clear area, and the intermediate clear area is a clear area located between the clear area of ​​the non-target image and the clear area of ​​the target image. The second difference determination submodule is used to determine, according to the first difference, a second difference in image parameters between the non-target image and the target image. The first feature point determination submodule is used to map the feature points of the clear area in the non-target image to the target image according to the second difference, so as to obtain the first feature points of the target image.

[0143] Exemplarily, the different regions are arranged along a predetermined direction and there are overlapping regions in adjacent regions of the different regions. The second difference determination submodule includes a second difference calculation submodule. The second difference calculation submodule is used to accumulate all first differences and determine a second difference in image parameters between the non-target image and the target image.

[0144] Exemplarily, the first difference determination submodule includes a coordinate difference determination submodule and a first difference calculation submodule. The coordinate difference determination submodule is used to determine the variance of the coordinate information between all the affine transformed feature points and the corresponding feature points in the second image that all the affine transformed feature points correspond to, based on the difference in coordinate information between each affine transformed feature point and the corresponding feature point in the second image that the affine transformed feature point corresponds to, wherein the coordinate information of the affine transformed feature point is represented by the product between the coordinate information of the corresponding feature point in the second image that corresponds to the affine transformed feature point and the affine transformation matrix, and the elements in the affine transformation matrix include unknown quantities represented by the first difference. The first difference calculation submodule is used to determine the first difference that minimizes the variance. The two different intermediate images include the first image and the second image, and the clear area in the first image is farther from the clear area in the target image than the clear area in the second image.

[0145] Exemplarily, the first difference calculation submodule includes an error function construction submodule, a line search submodule and an affine transformation matrix determination submodule. The error function construction submodule is used to establish an error function according to the variance. The line search submodule is used to perform a line search on the error function to determine all elements in the affine transformation matrix, wherein all elements in the affine transformation matrix minimize the value of the error function. The affine transformation matrix determination submodule is used to determine the first difference according to all elements in the determined affine transformation matrix.

[0146] Exemplarily, the first difference calculation submodule further includes a search direction determination submodule. The search direction determination submodule is used to determine the search direction of performing line search on the error function by using the quasi-Newton method.

[0147] Exemplarily, the two feature points corresponding to the position are the two feature points that are closest in position and whose distance is less than a distance threshold.

[0148] Exemplarily, the image feature point recognition device further includes a clear area recognition module. The clear area recognition module is used to determine the clear area of ​​the image to be processed according to the local contrast and / or gradient information of the image to be processed.

[0149] Exemplarily, the feature point selection module 940 includes a first selection submodule, a second selection submodule and a third selection submodule. The first selection submodule is used to determine, for each first feature point, the corresponding feature point of the first feature point in the target image as the second feature point when the first feature point is located in the overlapping area of ​​different images to be processed. The second selection submodule is used to determine the average value of the coordinates of the first feature point and its corresponding second feature point, so as to use the pixel point at the average value as the feature point in the area corresponding to the overlapping area of ​​different images to be processed in the target image. The third selection submodule is used to use the first feature point as the feature point of the target image when the first feature point is outside the overlapping area of ​​different images to be processed.

[0150] According to another aspect of the present invention, an electronic device is provided. Fig.10 FIG. 1 shows a schematic block diagram of an electronic device according to an embodiment of the present invention. Fig.10 As shown, the electronic device includes a processor and a memory, wherein the memory stores computer program instructions, and the computer program instructions are used by the processor to execute the image feature point recognition method as described above when the processor is running.

[0151] In addition, according to another aspect of the present invention, a storage medium is also provided, on which program instructions are stored, and when the program instructions are executed by a computer or a processor, the computer or the processor executes the corresponding steps of the above-mentioned image feature point recognition method according to the embodiment of the present invention, and is used to implement the corresponding modules in the above-mentioned image feature point recognition device according to the embodiment of the present invention. The storage medium may include, for example, a storage component of a tablet computer, a hard disk of a personal computer, a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a portable compact disk read-only memory (CD-ROM), a USB memory, or any combination of the above storage media. The computer-readable storage medium may be any combination of one or more computer-readable storage media.

[0152] According to another aspect of the present invention, a computer program product is provided, comprising computer program instructions, wherein the computer program instructions are used to execute the above-mentioned image feature point recognition method when running.

[0153] A person skilled in the art can understand the specific implementation and beneficial effects of the above-mentioned image feature point recognition device, electronic device, storage medium and computer program product by reading the above-mentioned detailed description of the image feature point recognition method. For the sake of brevity, they will not be repeated here.

[0154] Although example embodiments have been described herein with reference to the accompanying drawings, it should be understood that the above example embodiments are merely exemplary and are not intended to limit the scope of the present invention thereto. Various changes and modifications may be made therein by one of ordinary skill in the art without departing from the scope and spirit of the present invention. All such changes and modifications are intended to be included within the scope of the present invention as required by the appended claims.

[0155] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0156] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of the units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed.

[0157] In the description provided herein, a large number of specific details are described. However, it is understood that embodiments of the present invention can be practiced without these specific details. In some instances, well-known methods, structures and techniques are not shown in detail so as not to obscure the understanding of this description.

[0158] Similarly, it should be understood that in order to streamline the present invention and help understand one or more of the various inventive aspects, in the description of the exemplary embodiments of the present invention, the various features of the present invention are sometimes grouped together into a single embodiment, figure, or description thereof. However, the method of the present invention should not be interpreted as reflecting the following intention: the claimed invention requires more features than the features explicitly stated in each claim. More specifically, as reflected in the corresponding claims, the inventive point is that the corresponding technical problem can be solved with less than all the features of a single disclosed embodiment. Therefore, the claims following the specific embodiment are hereby expressly incorporated into the specific embodiment, wherein each claim itself serves as a separate embodiment of the present invention.

[0159] Those skilled in the art will understand that, except for mutually exclusive features, all features disclosed in this specification (including the accompanying claims, abstract and drawings) and all processes or units of any method or device disclosed in this specification may be combined in any combination. Unless otherwise explicitly stated, each feature disclosed in this specification (including the accompanying claims, abstract and drawings) may be replaced by an alternative feature that provides the same, equivalent or similar purpose.

[0160] In addition, those skilled in the art will appreciate that, although some embodiments described herein include certain features included in other embodiments but not other features, the combination of features of different embodiments is meant to be within the scope of the present invention and form different embodiments. For example, in the claims, any one of the claimed embodiments may be used in any combination.

[0161] The various component embodiments of the present invention may be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. Those skilled in the art will appreciate that a microprocessor or digital signal processor (DSP) may be used in practice to implement some or all of the functions of some modules in the image feature point recognition device according to an embodiment of the present invention. The present invention may also be implemented as a device program (e.g., a computer program and a computer program product) for executing part or all of the methods described herein. Such a program for implementing the present invention may be stored on a computer-readable medium, or may be in the form of one or more signals. Such a signal may be downloaded from an Internet website, or provided on a carrier signal, or provided in any other form.

[0162] It should be noted that the above embodiments illustrate the present invention rather than limit it, and that those skilled in the art may devise alternative embodiments without departing from the scope of the appended claims. In the claims, any reference symbol between brackets shall not be construed as a limitation on the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "one" or "an" preceding an element does not exclude the presence of a plurality of such elements. The present invention may be implemented by means of hardware comprising a number of different elements and by means of a suitably programmed computer. In a unit claim enumerating a number of devices, several of these devices may be embodied by the same hardware item. The use of the words first, second, and third, etc., does not indicate any order. These words may be interpreted as names.

[0163] The above is only a specific embodiment of the present invention or an explanation of a specific embodiment. The protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. The protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A method for identifying image feature points, characterized in that: The method comprises: For a target object, focusing on different areas of the target object and taking an image to be processed of the target object after each focusing, so as to obtain a plurality of images to be processed, wherein each image to be processed has a clear area with a clarity higher than a preset threshold; Determine corresponding feature points in an overlapping area of ​​a first image and a second image, wherein the first image is one of the images to be processed, and the second image is an image to be processed in which a clear area has the overlapping area with a clear area in the first image, and the corresponding feature points include a plurality of pairs of feature points, each pair of feature points includes two feature points in the first image and the second image, respectively, in the overlapping area and at corresponding positions, and the two feature points at corresponding positions are two feature points that are closest in position and whose distance is less than a distance threshold; Determine, in the target image, first feature points respectively corresponding to feature points of clear areas in the non-target image according to the position of each feature point in the corresponding feature points in the respective images, wherein the target image is one of the images to be processed, and the non-target image is another image in the images to be processed except the target image; According to the first feature points, feature points in an area in the target image corresponding to a clear area in the non-target image are determined.

2. The method according to claim 1, characterized in that The step of determining, in the target image, first feature points respectively corresponding to feature points of clear areas in the non-target image according to the position of each feature point in the corresponding feature points in the respective images comprises: For each non-target image, Determining a first difference in image parameters between the intermediate images according to positions of corresponding feature points in each of two different intermediate images, wherein the intermediate image is an image to be processed whose clear area is an intermediate clear area, and the intermediate clear area is a clear area located between a clear area of ​​the non-target image and a clear area of ​​the target image; Determining a second difference in image parameters between the non-target image and the target image based on the first difference; According to the second difference, the feature points of the clear area in the non-target image are mapped to the target image to obtain the first feature points of the target image.

3. The method according to claim 2, characterized in that The different regions are arranged along a predetermined direction and there are overlapping regions in adjacent regions of the different regions; Determining a second difference in image parameters between the non-target image and the target image according to the first difference includes: All the first differences are accumulated to determine a second difference in image parameters between the non-target image and the target image.

4. The method according to claim 2, characterized in that: The step of determining a first difference in image parameters between the intermediate images according to positions of corresponding feature points in each of two different intermediate images comprises: Determining the variance of coordinate information between all affine transformed feature points and corresponding feature points in the second image to which all affine transformed feature points respectively correspond based on the difference in coordinate information between each affine transformed feature point and a corresponding feature point in the second image to which the affine transformed feature point corresponds, wherein the coordinate information of the affine transformed feature point is represented by the product of the coordinate information of the corresponding feature point in the second image corresponding to the affine transformed feature point and an affine transformation matrix, and the elements in the affine transformation matrix include the unknown quantity represented by the first difference; Determine the first difference that minimizes the variance; The two different intermediate images include the first image and the second image, and a clear area in the first image is farther from a clear area in the target image than a clear area in the second image.

5. The method according to claim 4, characterized in that The determining the first difference that minimizes the variance includes: establishing an error function based on the variance; Performing a line search on the error function to determine all elements in the affine transformation matrix, wherein all elements in the affine transformation matrix minimize the value of the error function; The first difference is determined according to all elements in the determined affine transformation matrix.

6. The method according to claim 5, characterized in that The method further comprises: The quasi-Newton method is used to determine the search direction for line searching the error function.

7. The method according to any one of claims 1 to 6, characterized in that: The method further comprises: Determine a clear area of ​​the image to be processed according to the local contrast and / or gradient information of the image to be processed.

8. The method according to any one of claims 1 to 6, characterized in that: The determining, based on the first feature points, feature points in an area in the target image corresponding to a clear area in the non-target image comprises: For each first feature point, In the case where the first feature point is located in an overlapping area of ​​different images to be processed, determining a feature point corresponding to the first feature point in the target image as a second feature point; Determine an average value of the coordinates of the first feature point and the second feature point corresponding thereto, so as to use the pixel point at the average value as the feature point in the area of ​​the target image corresponding to the overlapping area of ​​the different images to be processed; If the first feature point is outside the overlapping area of ​​the different images to be processed, the first feature point is used as the feature point of the target image.

9. An image feature point recognition device, characterized in that: include: An image acquisition module, for focusing on different areas of a target object and taking an image to be processed of the target object after each focusing, so as to obtain a plurality of images to be processed, wherein each image to be processed has a clear area with a clarity higher than a preset threshold; a feature point matching module, configured to determine corresponding feature points in an overlapping area of ​​a first image and a second image, wherein the first image is one of the images to be processed, and the second image is an image to be processed in which a clear area overlaps with a clear area in the first image, and the corresponding feature points include a plurality of pairs of feature points, each pair of feature points includes two feature points in the first image and the second image, respectively, in the overlapping area and at corresponding positions, and the two feature points at corresponding positions are two feature points that are closest in position and whose distance is less than a distance threshold; A feature point recognition module, used to determine, in a target image, first feature points respectively corresponding to feature points of a clear area in a non-target image according to the position of each feature point in the corresponding feature points in the respective images, wherein the target image is one of the images to be processed, and the non-target image is another image in the images to be processed except the target image; The feature point selection module is used to determine the feature points in the area of ​​the target image corresponding to the clear area in the non-target image based on the first feature points.

10. An electronic device comprising a processor and a memory, characterized in that: The memory stores computer program instructions, which are used by the processor to execute the image feature point recognition method according to any one of claims 1 to 8 when the processor runs the computer program instructions.

11. A storage medium having program instructions stored thereon, characterized in that: The program instructions are used to execute the image feature point recognition method according to any one of claims 1 to 8 when running.

12. A computer program product comprising computer program instructions, characterized in that The computer program instructions are used to execute the image feature point recognition method according to any one of claims 1 to 8 when running.

Images